2015-03-19

On this day 8 years ago Apache Camel was born

So its March 19th which is 8 years ago the first public commit of Apache Camel was pushed to ASF by James Strachan in 2007.

Using the lovely github browser its easy to have a flashback and take a look at what this first commit contains.

So James included APIs and concepts that end users of Apache Camel today would find familiar.

Lets start with the unit test of a Camel route, which is as shown below. The unit test includes three tests, one is a test of a simple route, that is just a from -> to as shown below:

RouteBuilder from the very first commit - sometimes we hit the nail from the first try

If you look at the code above, then that code would be 100% the same today. Its a route using the Java DSL using the same class name, RouteBuilder, where the routes are defined in the configure method. And this route is a simple from -> to using the seda endpoints.

The other two unit tests are using the content based router, aka choice in Camel.

Camel route using choice and predicates
These routes would almost be the same today, its only the predicate in the choice that has changed from:

headerEquals("foo", bar")

to a more rich DSL

header("foo").isEqualTo("bar")

If we keep looking in the first commit, we see that it includes APIs and concepts we use today

org.apache.camel.Endpoint
org.apache.camel.Exchange
org.apache.camel.Predicate
org.apache.camel.Processor
org.apache.camel.builder.RouteBuilder

It sure was a great start from the very first commit.

In summer 2007 Apache Camel 1.0 was released, which I had a flashback in 2012 and blogged about what I though was a great start for Camel.

And here 8 years later Apache Camel is as strong as ever.
Just take a look at the latest impressive Camel 2.15 release.


2015-03-15

Apache Camel Eclipse Tooling with Debugger

About 2 months ago Lars Heineman blogged about improved Apache Camel Eclipse tooling on the way as part of the JBoss Tool stack

In the upcoming release they have integrated the Camel debugger with the native Eclipse debugger, so you get the Eclipse debugging experience, when you use breakpoints, single step through the Camel routes. And you can of course see all the details of the Camel Exchange and Message on the way. And as well change the data on the fly.

A screenshot is shown below, which I have borrowed from Lars's blog.

Eclipse debugging your Apache Camel routes at runtime.
Lars also worked on adding support for editing Camel endpoints uri's using a properties panel, so each option is provided individually. That work was based on Apache Camel 2.14 that has some support for this.

But since we have improved this tremendously in Apache Camel 2.15, which I recently blogged about. And therefore Lars is currently working on upgrading to Camel 2.15 so the Eclipse Tooling becomes even better.

Now imaging that we take the properties panel based on Camel 2.14 and add all the extra information and documentation we have from Camel 2.15, which will allow Eclipse to present a similar enriched properties panel as hawtio can do.

Using Camel 2.14 showing an empty properties panel (no documentation, no information about defaults, required, limited enums, etc, consumer vs producer option etc.)

Eclipse Camel properties editor based on Camel 2.14

And below the enriched Camel 2.15 which has all of above information, show currently in hawtio.

Hawtio Camel properties viewer based on Camel 2.15

So imagine that the Eclipse properties panel will be able to include out of the box:

  • documentation
  • default values
  • enum types (eg choices to select among)
  • required vs optional
  • deprecated
  • simple and java type
  • option as part of uri path or query parameter
  • consumer only option
  • producer only option
  • custom category for the option (eg security, advanced, etc.)

And with Camel 2.15, we are able to do this for all the components - they all provide all this information.

Camel 2.15 also brings to the table, that it would allow the Eclipse tooling to dynamic generate the EIP palette, as Camel include information about all the EIPs and their options as well. So imagine the Eclipse tooling is able to adjust to which version of Camel you are currently using in the project. And yeah all the EIP options is now documented as well, which the tooling can present to you.

I am really exited about the possibilities that Camel 2.15 brings to the table in terms of tooling and also runtime experience we can enhance.

We also work on JBoss Forge commands that allows to add Apache Camel to existing projects, to dockerize and/or fabric8 enable the projects - that is something for another blog. But as part of this work, we are working on commands to add/edit Camel components/endpoints. So the idea would be they can show all the endpoints uris in your project, and present a nice properties editor for you to have "type safe" editing.

Going back to the title of this blog. Yeah great work Lars and the Eclipse team, we now have a great Apache Camel debugger. And its using the same Camel debugging API that hawtio uses also - no magic trick. In fact this week I was in talk with a company that has built their data integration platform on top of Apache Camel and also leverages its debugging api, to allow its developers and users to debug the deployed Camel routes, on the platform.


2015-03-11

In the year 2015 - Apache Camel 2.15.0 was released

So it will be known to mankind that in the first quarter of the year 2015, the awesome Apache Camel 2.15.0 was released.

Apache Camel 2.15.0 was released in 2015

Top 10 of great new features

I will try to do a top 10 of new functionality this release brings to the table.

#1 - Self documented

This release brings together, the steps we have taken over the last couple of released, in terms of being self documented. So what does this mean?

I have previous blogged about the work we have done


So what does this mean? It means that the same documentation Camel provides for all its:
  • Java and XML DSLs
  • Enterprise Integration Patterns
  • Camel Components
  • Camel Data formats
  • Camel languages
Is now included at both design and runtime. So for example if you design Camel routes in either Java or XML then the documentation is at your fingertips. For Java its documented as javadoc, and for XML that same documentation is now injected into the Camel XSD schemas, which allows IDE tooling to show you that documentation.

Here is a screenshot of IDEA editing a Camel route in XML and showing the documentation for the splitter EIP.

Camel XML now includes EIP documentation out of the box
At runtime your Camel applications and routes can present the documentation as well, but with the extra twist that the documentation can overlay with the runtime configuration in use. In other words we can now show what all those EIP options, component options, endpoint options, and whatnot means, and what their current values are, and what their default values are, whether they are required or options, which type they are, and if they are enum based, what options the can be, and much more.

All that comes together nicely if using the latest hawtio web console, which can present this nicely as the next screenshot shows

hawtio showing EIP endpoints at runtime with current values and documentation
To access the documentation at runtime, then Camel offers Java, JMX and Camel commands as the APIs. hawtio uses the JMX API.

#2 - Categorized components

All the Camel components have now been categorized, which means we can group the components, for example to know which Camel components there is for SQL, social, messaging, file, and so on. That is not all, we have also categorized all the EIPs, data formats as well.

And that information is available at both design and runtime, just as with the documentation. And this information can be reached using Java, JMX, and Camel commands. For example the Camel commands now has a set of Camel catalog commands that can be used, to find out which components we have for social, database, etc. The following screen shot shows the Camel commands in action using the new Camel Catalog component to do just that.

Camel commands to list Camel components by group

#3 - Camel Catalog for Tooling

There is a new standalone camel-catalog JAR which is intended for third party tool developers (and also Camel itself). The JAR contains a list of all Camel eips, components, data formats, languages, Camel XSD schemas, and the Maven archetypes. And also full documentation for all those as well. In other words the camel-catalog JAR is an offline JAR that has all the information we have talked about in #1 and #2. The JAR has Java and JMX API, as well .properties, .xsd, and .json schema files for all that. So its easy for tools to tap into that information.

Apache Camel uses the catalog itself, in the Camel commands as shown in the screenshot above.

As mentioned before there is a set of new Camel commands being developed for JBoss Forge, which is intended for making development with Apache Camel easier. You can say that this is an example of a third party developing Camel tooling. There is commands to make it easier to add new Camel components, configure endpoints, and whatnot. What makes JBoss Forge awesome is that it allows to run the same binary commands (forge addon) in any of its supported environments, which are

- command line (eg CLI)
- Eclipse
- Netbeans
- IDEA
- web

So for example we have a Camel command to add a new component that allows to filter by groups. The following screenshot shows which components we have that is about files.

Camel Forge commands running in Eclipse, as a wizard to add a Camel component to your project
And the same command used from web (work in progress)

Camel Forge command running in web browser, as a wizard to add a Camel component to your project

#4 - Reuse Camel commands

In previous releases of Apache Camel, the Camel commands was only for Karaf users. But in this release we have made these commands reuseable, in the camel-commads-core module. And there is an out of the box implementation for Karaf and Jolokia. 

And for third party there is JBoss Forge Camel commands as part of the fabric8 project. By using these commands you can manage Camel applications at runtime, for example to check the state of your routes, and so on.

The screenshot below shows how to use the camel-routes command from JBoss Forge. The same command can be run from Apache Karaf as well.

Camel commands connecting to a remote JVM using jolokia to check the route status
The Karaf commands only works in the same JVM as Karaf itself. But the jolokia commands allows to manage remote JVMs as well. In the screenshot above I have a local running JVM with Apache Tomcat and deployed the standard Camel Tomcat Servlet example. And from a JBoss Forge JVM I use the commands to connect to the Tomcat JVM and list all its running Camel routes. If you are familiar with hawtio then its the same principle, as hawito is able to do that same as well.


#5 - Camel Boot / Spring Boot

For the micro services trend we have Camel boot supported, though most noticeable by the camel camel-spring-boot module. Henryk Konsek did a great job implementing and documenting this module, so there is plenty of material there to run Camel and Spring happily together.

For Camel Boot we are improving on alternatives with CDI, Guice, and plain Java in the upcoming release. For CDI in particular there is works in a much improved and working camel-cdi, which is expected to be included in Camel 2.16.


#6 - Improved swagger 

The camel-swagger module now supports any kind of runtime (before it was tied to Spring). In the next release we will add support for hosting multiple Camel applications from the same camel-swagger module, so you can deploy it once, and use it for all your Camel apps in the same JVM.

We are also hoping the swagger team finish their highly anticipated pure Java implementation, so we can drop Scala and make camel-swagger more friendly to use / and much lighter without the Scala libraries.


#7 - Improved Rest DSL

We have improved and fixed some issues with the new Rest DSL that was introduced in Camel 2.14 based on community feedback. For example its easier to configure using custom jackson modules for json. We also made it easier to return custom error messages when routing has failed etc.

We will continue to do so as we have still some outstanding work. 


#8 - Routing Engine optimizations

We also optimized Camel itself in particular two areas:

  • Optimized the case insensitive headers, but dropping the need for an internal shadow map.
  • Optimized the type converter to avoid conversion attempts that was not needed, which in some cases reduced the factor by 2-4 x during routing.

#9 - More detailed information about inflight messages

Camel now tracks inflight messages with more details, which allows to pin point exactly where in the routes the inflight messages current is being processed at. This information can be accessed from Java and JMX. And there is also a Camel command to list inflight messages. Also hawtio has a web page that list all the inflight messages with all that information as well.

hawtio showing inflight exchanges for the selected route. Notice the message is currently at "delay1" in the route

And the Route JMX MBean now also include a JMX attribute that contains the oldest inflight duration, which makes it easier for monitor tooling to quickly identify which routes may have messages that may "take too long time".

Also during shutdown of Camel itself, in case of the graceful shutdown was not able to shutdown and complete all inflight exchanges, then Camel now logs which exchanges are still inflight, and where in the routes they currently are.

#10 - Configuring endpoints in XML with long attributes can now break lines

When using XML for configuring Camel routes, you may use endpoints that has many options you need to configure, which then may make the endpoint uri long and less human readable. What you can do now, is to use line breaks in the attribute as shown below

Configuring endpoints in XML can now use line breaks (you can have multiple options in the same line)
An alternative is also that the less known can be used to configure endpoints, which now support style as well, as shown:

Configuring endpoints using property style


#11 - And as always a bunch of new components

And as usual we have a bunch of new components. This time there is 16 new components, for example for the talk of the town, Docker. And for Spring 4.0.x users there is a working camel-spring40-test module for testing. As camel-spring-test requires Spring 4.1.x.




You can find more details in the Apache Camel 2.15.0 release notes. And make sure to read the bottom of the release notes that details what to consider when upgrading.

The release can be downloaded from maven central, and the Apache Camel website.