Apache Camel please explain me what these endpoint options mean

In the upcoming Apache Camel 2.15, we have made Camel smarter. It is now able to act as a teacher and explain to you how its configured and what those options mean.

The first lesson Camel can do is to tell you how all the endpoints have been configured and what these option mean.

Lessons we are working on next is to let Camel explain the options for the EIPs are.

Okay a picture is worth a thousand words, so let me show a screenshot from Apache Karaf, where you can use the new endpoint-explain command to explain how the endpoints have been configured.

The screenshot from Apache is from the SQL example which I have installed in Karaf. This example uses a number of endpoints, and among those a timer to trigger every 5 seconds. As you can see from above, the command list the endpoint uri: timer://foo?period=5s and then explain the option(s) below. As the uri only has 1 option, there is only one listed. We can see that the option is named period. Its java type is a long. The json schema type is integer. We can see the value is 5s, and below the description which explains what the value does.
So why is there two types listed? The idea is that there is a type that is suitable for tooling etc, as it has a simpler category of types accordingly to the JSonSchema specification. The actual type in Java is listed as well.
The timer endpoint has many more options, so we can use the --verbose option to list all the options, as shown below:

The explain endpoint functionality is also available as JMX or as Java API on the CamelContext. For JMX each endpoint mbean has an explain operation that returns a tabular data with the data as above. This is illustrated in the screenshot below from jconsole:

In addition there is a generic explainEndpointJson operation on the CamelContext MBean, this allows to explain any arbitrary uri that is provided. So you can explain endpoints that are not in use by Camel.

So how does this works?

During the built of the Apache Camel release, for each component we generate a HTML and JSon schema where each endpoint option is documented with their name, type, and description. And for enums we list the possible values.

Here is an example of such a json schema for the camel-sql component:

Now for this to work, the component must support the uri options, which requires to annotation the endpoint with the @UriEndpoint. Though the Camel team has not migrated all the 160+ components in the Camel release yet. But we plan to migrate the components over time.

And certainly now where we have this new functionality, it encourages us to migrate all the components.

So where do we get the documentation? Well its just java code, so all you have to do is to have getter/setter for an endpoint option. Add the @UriParam annotation, and for the setter you just add javadoc. Yes we grab the javadoc as the documentation. So its just documented in one place and its in the source code, as standard javadoc.
I hope we in the future can auto generate the Camel website documentation for the components, so we do not have to maintain that separately in its wiki system. But that would take hard work to implement. But eventually we should get there, so every component is documented in the source code. For example we could have a readme.md for each component that has all the component documentation, and then the endpoint options is injected from the Camel built system into that readme.md file automatic. Having readme.md files also allow github users to browse the Camel component documentation nicely using github style ;o

So what is next?

The hawtio web console will integrate this as well, so users with Camel 2.15 onwards have that information in the web console out of the box.

And then its onwards to include documentation about the EIP in the XML schemas for Spring/Blueprint users. And improve the javadoc for the EIPs, as that then becomes the single source of documentation as well. This then allows tooling such as Eclipse / IDEA / Netbeans and whatnot to show the documentation when people develop their Camel routes in the XML editor, as the documentation is provided in the XSD as xsd:documentation tags.

We have captured some thoughts what else to do in the CAMEL-7999 ticket. If you have any ideas what else to improve or whatnot, then we love feedback from the community.


66th Apache Camel release is out - its the 2.14 release

Today Apache Camel 2.14.0 hit the streets. Its our 66th release (include all patch releases) since the project was created over 7 years ago.

There is a bunch of great stuff in this release, so let me try to distill a list of 10 highlights (in no particular order)

1) Java 8 support
The 2.14.x branch marks the 1st release where we officially support Java 8. We made sure the code compiles and all tests passes on Java 8 also. We also support Java 7.

Now you may ask about all the coolness from Java 8 about lambda's and whatnot. Yeah we are looking into that for the next release to provide examples and a first cut of a Java 8 based DSL.

2) Spring 4 support
The same goes here. We now support Spring 4.x also. There is one migration effort, as Spring 3.x and 4.x is not compatible when using spring testing. So we had to introduce a camel-test-spring3 module for Spring 3.x users, and camel-test-spring is for 4.x users. Also there is a known issue in camel-test-spring about Spring 4.1.x support which we will fix in the next patch release.

3) Rest DSL
This is likely one of the most exciting new features in this release. We now offer a REST styled DSL which can be used with Java or XML. The intention is to allow end users to define REST services using a REST style with verbs such as get, post, delete etc.

Here is a small taste of it in action:


The DSL can be used in both Java and XML. As this is the 1st release with the Rest DSL we will of course continue to improve on this, and certainly based on the feedback from the community.

4) Netty 4.x support
We now have support for both Netty 3.x and 4.x. As we take backwards compatibility seriously we decided to leave camel-netty as-is for Netty 3.x users, and port the code to a new camel-netty4 for Netty 4.x users. The same goes for the camel-netty-http and camel-netty-http4 modules.

5) API component
Dhiraj Bodke worked on a great new API component support in camel-core, that makes creating new Camel components for APIs much easier and faster. We use this to integrated with new components such as box, linkedin, google-drive, and salesforce and what else comes in the future. He wrote a great blog about this component and how to use it. So expect many more of these API components in upcoming releases.

6) More JMX MBeans
We now also enlist producers in JMX, and as well have a registry to capture runtime usage of endpoints, which allows you to "see" which endpoints are in use, and as well how endpoints may be shared across routes, e.g. sending to queue A in route 1 and consume from queue A in route 2. All based on dynamic usage such as dynamic patterns like the recipient list.

The aim is to provide better understand how your Camel routes are being used at runtime. We will continue to improve tooling in this area such as hawtio, and the Camel Karaf commands, etc.

7) Codahale metrics
And continued from #6 we also have a new camel-metrics component to integrate with the excellent codahale metrics library. On top of that we also allow to use this to capture Camel routing statistics using codehale. This can then be used with existing codahale tooling and also with hawtio. I wrote a blog about this earlier.

The screenshot below illustrates routing information captured by camel-metrics and displayed by hawtio in the new route metrics tab which supports codahale.

camel-metrics used to capture routing statics which can be displayed in hawtio using the new route metrics tab.

8) Circuit breaker EIP
We introduce a new EIP using the circuit breaker pattern.

9) Swagger integration
We have a new camel-swagger module that in combination with the Rest DSL allows to expose an API documentation of your REST services. This can then be used with existing swagger tooling and web consoles. For example hawtio has a swagger console out of the box, such as shown below

Camel example using the new Rest DSL which offers swagger API documentation, which can be viewed using a swagger console such as offered by hawtio

10) And the usual stuff
We added 15 new components, 1 new EIP, 1 new data format, and 1 new language. And as usual a bunch of bug fixes, improvements and hardening.  And we dropped support for Java 6.

If you want to see some of the new stuff in action, then you can try the new examples. We have the camel-example-servlet-rest-tomcat. This is a plain WAR file which you can run in Apache Tomcat, Jetty, Wildfly, fabric8 or any other web container. Then you can see both the new Rest DSL, Swagger and codehale metrics in action, all in the same application. And for hawtness then try deployed hawtio as well and see what it can do as well.

The same example is also available for Karaf users by the camel-example-servlet-rest-blueprint.

You can find more details about this release in the 2.14 release notes. And if you are upgrading from a previous release, then make sure to read about the changes that may effect you.

You can get this release from Maven Central and download from the Apache Camel website.


More metrics in Apache Camel 2.14

Apache Camel 2.14 is being released later this month. There is a slight holdup due some Apache infrastructure issue which is being worked on.

This blog post is to talk about one of the new functions we have added to this release. Thanks to Lauri Kimmel who donated a camel-metrics component, we integrated with the excellent codehale metrics library.

So I took this component one step further and integrated it with the Camel routes so we have additional metrics about the route performances using codehale metrics. This allows end users to seamless feed Camel routing information together with existing data they are gathering using codehale metrics.

Also take note we have a lot of existing metrics from camel-core which of course is still around.

What codehale brings to the table is that they have additional statistical data which we do not have in camel-core.

To use the codehale metics all you need to do is:

  1. add camel-metrics component
  2. enable route metrics in XML or Java code

To enable in XML you declare a as shown below:

  &;t;bean id="metricsRoutePolicyFactory"

And doing so in Java code is easy as well by calling this method on your CamelContext

  context.addRoutePolicyFactory(new MetricsRoutePolicyFactory());

Now performance metrics is only useable if you have a way of displaying them, and for that you can use hawtio. Notice you can use any kind of monitoring tooling which can integrate with JMX, as the metrics is available over JMX. The actual data is 100% codehale json format, where a piece of the data is shown in the figure below.

Sample of the route metrics JSON data
The next release of hawtio supports Camel 2.14 and automatic detects if you have enabled route metrics and if so, then shows a sub, where the information can be seen in real time in a graphical charts.
hawtio have detected that we have route metrics enabled, and shows a sub tab where we can see the data in real time

The screenshot above is from the new camel-example-servlet-rest-tomcat which we ship out of the box. This example demonstrates another new functionality in Camel 2.14 which is the Rest DSL (I will do a blog about that later). This example enables the route metrics out of the box, so what I did was to deploy this example together with hawtio (the hawtio-default WAR) in Apache Tomcat 8.

With hawtio you can also build custom dashboards, so here at the end I have put together a dashboard with various screens from hawtio to have a custom view of a Camel application.

hawtio dashboard with Camel route and metrics as well control panel to control the route(s), and the logs in the bottom.


Apache Camel 2.14 on the way

This is just a little blog update to say that we are working on the final touches for the next release of Apache Camel which is version 2.14.

We have a discussion thread on the Apache Camel mailing list where you can provide feedback and follow the process.

There is a lot of great stuff in this release which I frankly need to blog about when I get a chance and spirt to write.

After getting some bug fixes into the Camel codebase today I plan to work on finishing the codehale metrics UI in hawtio. This allows Camel end users to easily add additional metrics about their route performances, using the metrics from codehale. We then use a metrics javascript ui, which we then integrate in hawtio, which makes it possible to render live charting which is shown in a screenshot below:

Camel routes using codehale metrics statistics, rendered in real time
Another great new functionality in Camel 2.14, is the new DSL for rest services, though that warrants a blog entry by itself. And we also integrated with Swagger, so these REST services can provide an API out of the box.

The Apache Camel 2.14 release is expected out in September 2014. We will try to see if we are ready next week to start cutting the release.

If you have anything you want included or fixed then its time now to speak up.


Developer Interview with me about Apache Camel

Today I had a nice talk with Markus Eisele whom run a series of developer interviews. I was lined up today.

The main topic was of course Apache Camel which is my primary area of interest and work for the last 6 years or so. But we do have a bit of time to talk a little about other cool projects such as hawtio, jolokia, vertx and whatnot. I also talked about my background and how I discovered and got involved with the Apache Camel project about 6,5 - 7 years ago.

As I talked a bit too much then the video is about 35 minutes, so it requires 2 cup of coffee to watch it end to end. But half way through I share my deskop and do some demoing. I have to talk Bilgin Ibryam for creating his lovely Camel Olympics demo, which I used as the 2nd demo.

You can find Markus blog here, to keep an eye on his developer interview series, and also find the past recordings.


Meet Fabric8 and video of fabric8 provisioning Apache Tomcat

Recently my fellow colleague Christian Posta wrote two great blog entries about fabric, which I want to share to you.

The new fabric8 logo
In his Meet Fabric8: An open-source integration platform based on Camel and ActiveMQ, Christian introduces us to fabric8, and talks about what fabric8 brings to the table from a DevOps perspective, and some of the other value-adds. Christian also talks about the history of the project.

In his 2nd blog DevOps with Apache Tomcat/TomEE and Fabric8, Christian talks and demonstrates some of the upcoming functionality in the next 1.1 beta7 release, where fabric8 adds support for provision and manage Apache Tomcat, and other containers. Christian recorded a video demonstrating this in action.

We are currently working hard on getting the last pieces into the 1.1 beta7 release which ought to be released this week. We have also worked on revamping the fabric8 website, and working on having the documentation in a nice bookish readable format, and as in PDF as well. The documentation is nicely readable now on handheld devices. I am also putting my touches on improving the quickstarts and examples to provide a lot more documentation and how-to. So expect the documentation, quickstarts, and other examples to have out attention, leading up to the 1.1 release.

We want to do a bunch of beta releases, as this makes it much easier for the community to try out fabric8, and help contribute to the project, before we reach 1.1 final release (ETA sometimes after the summer vacation).


Apache Camel, Groovy and hawtio at gr8conf

Today I presented Apache Camel at the gr8conf, which is a Groovy conference. So in my talk I presented what Camel can do with Groovy today.

Giving a talk about Apache Camel and Groovy at gr8conf 2014 in Copenhagen

The talk was recorded and will be later published by the gr8conf organizers. So keep an eye on their website when that happens.

Being here at the gr8conf is fun and a great event. I love how the groovy community and ecosystem has been able to establish this annual conference, running at its 6 year now. Also I take my hat off for the great people of the groovy team that do the great work on this language, that is now over 10 years old.

In my talk I also demonstrated hawtio which is a great web console, and a project that is fun to work on. So to prepare for gr8conf I developed a plugin example for hatwtio that is a Groovy Shell, so you can execute groovy scripts from the web console.

The source code for the example is in github at: https://github.com/hawtio/hawtio/tree/master/hawtio-plugin-examples/groovy-shell-plugin

And the plugin can easily be installed in Apache Tomcat, Karaf, JBoss Fuse, fabric8 etc.

Custom plugin installed in hawtio running on Apache Tomcat

If you are interested in learning how to build custom plugins for hawtio, then I think the groovy shell plugin is a good example, as it has a bit of backend service, and a simple web page to go with that.

So in my day time work I have great fun working with great technologies like Apache Camel, hawtio, and fabric8. And also a bit fun when I get a chance to work a bit with Groovy too. Anyway I just wanted to say that all these four awesome technologies has something in common. They are created/co-created by a great entrepreneur in our industry.

A slide with 4 great technologies that has something in common ...

Okay there is almost beer'o'clock here at gr8conf, as the organizers have brewed their own groovy beers we are going to taste soon in the scroll bar ;)