Introducing the new Apache Camel based iPaaS

I just got back from CamelOne 2013 which was the 3rd annual Camel conference.

At the conference, James Strachan, introduced the new open source Apache Camel based iPaaS at the ending keynote on the first day.

You can find more details from James blog where he put out this information with the slides and a cool 20 minute video demonstrating the iPaaS in action. Its all a single page HTML5 web console, using the awesome hawio project.

In the video you will also see cool stuff such as the new Camel visual debugger, where you can set breakpoints, and single step the message in the routes etc.

And we also reveal for the first time the new web UI for log analytics (insight search). So you easily search in the logs from all the containers, and correlate the log messages, and have all that visualized with timeline, graphs etc.

I guess a few screenshot is worth a 1000 words in this case.

Lets start with the new insight plugin in action ...

Fabric Insight in action - Live log analytics using Kibana web UI, to aggregate and visualize log from all your containers. Notice the Camel tab with first class Camel integration.

And the debugger in action ...

Camel debugger in action - Live debugging of Camel routes, set breakpoint, single step, see message details. And as well edit the message on the fly (to be added)

And we also got time to add a new Camel route creator / editor that is purely web based ...

Camel route designer - A full Camel editor in the web console with wizards, making it easier for casual Camel developers to use.

Though the video shows all this in action, and there is a funky theme song to go along. The selection of the song was done after careful consideration in the bar, after a long day at CamelOne ;)

To see the video I suggest to click this link to get to James blog, or use this link for the direct link to Vimeo where the video is hosted.

On James blog you will also find instructions how to run this from the source code and try this on your own.

Fuse Fabric and hawtio are both 100% open source projects, with the ASL 2.0 license, and their source code hosted on github. We love contributions and collaboration. So feel free to fork and send pull requests. And if you send too many, then you can become a member of these projects, and commit directly ;)

See you soon at CamelOne 2013

So CamelOne 2013 is starting on sunday evening with a reception with drinks and beers. What a great way to start. You can find details when and where at the agenda from the website. So make sure to attend on sunday evening if you can.

The conference then follows on monday and tuesday. The full agenda is up on the website at

This is the 3rd annual CamelOne conference, which is focused on the Apache integration projects such as Apache Camel, ActiveMQ, ServiceMix, Karaf and CXF. The speakers at the conference includes the founders of these projects, leaders, committers, long term end users, book authors, and as well end users who are using these projects in their companies.

We have a mix of speakers who are the people who hack on the projects all day long, and as well people who have used these projects in large projects at their companies. So you get information from both sides of the fence.

Rob Davies have blogged about what to look forward to at the conference. And as well giving a teaser for his talk about connecting applications everywhere with ActiveMQ.

Christian Posta who has dived deep into the internal of ActiveMQ is giving a talk about ... well the internals of ActiveMQ.

I will give a getting started with Apache Camel presentation, which is aimed for beginners or new users of Camel. And on the 2nd day I give a rundown of the common Camel components, and as well how to build your custom components. Both my talks is a mix with slides and practical live coding sessions.

I am much looking forward to James Strachan's ending keynote on the first day about using the hawtio web console with Apache Camel. There is gonna be some cool stuff revealed for the first time :)

Long stacktraces in Apache Camel no more

In the upcoming Apache Camel 2.12 we have improved the internal routing engine to dramatically reduce the stacktraces, our end users will see when an exception occurred and was logged.

During the development of the improvement we had a small sample route for benchmarking.
It was basically a simple route with:

from("seda:start")
  .to("log:foo")
  .to("log:bar")
  .to("log:baz")
  .process(new Processor ...);

And in the processor we forced an exception being logged. In Apache Camel 2.11.x the stacktrace would be around 40 lines. The Camel team managed to reduce this down to 15 lines. And the bottom 6 lines is 3 from the JVM and 3 from the seda consumer. So in between the routing engine is executing; and that's where we have optimized the code. So we went from (40-6 = 34) to (15-6 = 9). Eg before we had 34 lines during the routing, and that is down to 9 now.

Camel is still executing the same functionality. As an end user there is no functionality turned off or missing now. Its all still there and being executed; just in a different way. What we did was refactor most of the internal cross cutting functionality into a single entity CamelInternalProcessor (name subject for change), and fold that into a single stack-frame being executed.

This improvement is internal changes only, and all end users Camel application is upgrade compatible.

For our end users who have been debugging the Camel source code, this is now also simpler, as we added code comments in the CamelInternalProcessor with pointers how to do this easier, and which part you can safely skip over, to make your debugging experience easier and faster.

And speaking of debugger. Then we also added a new BacklogDebugger, which is a Camel routing debugger JMX MBean which is ready to use for tooling and end users. For example you can use it easily set breakpoints, single step a message in the routes, and edit/view its message content. Now for the graphical UI, then James Strachan is hacking on hawtio to add this in the upcoming hawtio 1.2 release. And since its JMX based you can always use it from a JMX console such as JConsole or JVisualVM. And for the brave a plugin to Eclipse / IDEA could be developed to integrate it with their native debugger; but thats a big task to take on.

For further details and background on the stacktrace improvement then you can take a look at the ticket CAMEL-6077 and our discussion on the Camel developer mailing list. As well we have a sample of the before and after stacktraces in the CAMEL-6077 ticket.

Webinar on May 16th 2013 - Using Apache Camel connectors for external connectivity

I have the pleasure of speaking at a webinar on thursday focusing on Apache Camel components.

Webinar about Apache Camel components

Any integration requires connectivity to applications, services, or databases. Connectivity could be a file drop via FTPor sFTP, a JDBC call to a database, or a webService call to an application. Apache Camel comes with 100+s Camel components or connectors that enable this functionality.

To archive that Apache Camel comes with 130+ components out of the box, ready for you to use. In this webinar we take a look at some of the popular components, as well how you can write your custom components.

The webinar is a mix between slides and live demonstration. The session is scheduled for one hour, where we leave about 10 minutes at the end for Q and A.

Register for this webinar and learn how:

• Apache Camel speeds connectivity.
• To use Apache Camel components to create faster integration solutions.
• To build custom Camel components.

When
Thursday, May 16, 2013 at:

• 15:00 UTC
• 11:00 am (New York)
• 4:00 pm (Paris)
• 8:30 pm (Mumbai)

How to register
The webinar is free and you can register for the event from this link.

Video recordings (in Danish) from my latest getting started with Apache Camel presentation

Last week I visited Århus, Denmark to give a presentation about Apache Camel. This event was organized by JavaGruppen, and OpenMinds was kind to host the event, in their beautiful office centrally located in Århus.

Thanks to Thomas Salling whom was so kind to pick me up at the train station, I arrived safely and in time for the event.

Claus presenting Apache Camel at Javagruppen meeting in Århus, Denmark -  Photo by Søren Glasius

At the event we had a great turnup, of about 25 people. And we were almost out of free chairs, and space in the room, as you can see from the photos.

Attendees at the event - Photo by Søren Glasius

After the event we went for beers at Sct. Clements which is a brewery inside the restaurant.

Having beers at Sct Clemens after the event.

My presentation was 2 hours and with a mix of slides and live demonstrations. And as well taking in questions. We had a 10 minutes break at the one hour mark. Giving such a long presentation is of course a challenge. The response from the attendees was positive and they didn't get bored etc. And we had about 2/3 of the attendees whom didn't know about Apache Camel.

I also had the pleasure of meeting with two former colleagues whom I haven't seen in over 10 years. It was great to see you Michael and Simon.

Søren Berg Glasius, whom is known in the Groovy and Grails communities  is head of organizing the GR8Conf conferences. The next GR8Conf takes place in a couple of weeks in Copenhagen, Denmark. So he is testing his video and presentation equipment, so we used this opportunity to video tape my entire presentation. Unfortunately there was some problems with the HDMI connection so we had to settle for VGA. But the end result is still very great. The video features both the slides and the camera feed in the same picture. This makes it very easy to follow the presentation.

Soren was so kind to invite me as a guest to the GR8Conf, so I will drop by in Copenhagen on the 23rd of March. Looking forward to see how far the Groovy and Grails have reached. We do see more and more people use Grails and Camel. Likewise the gradle build system seems to be cool as well.

The videos of the presentation is already online and you can find them here.
And the slides is also uploaded to slideshare here.

Thanks to Javagruppen and OpenMinds for hosting such a great event. I had a great time visiting Århus.

Apache Camel 2.11 released

Last week Apache Camel 2.11 was released.

Apache Camel 2.11 has been released

This blog post is a summary of the most noticeable new features and improvements.
For a detailed description, see the Camel 2.11 release notes.

1) New components
As usual each new release contains a number of new components, contributed by our large user base. Thanks guys.

For example there is camel-cmis which allows to integrate with content management systems, such as Alfresco, or any of the systems supported by Apache Chemistry, which is what we use in camel-cmis.

We also got a new camel-couchdb for integrating with our fellow Apache CouchDB project.

Also very exiting is the new camel-elasticsearch component, to integrate with the excellent elastichsearch project.

In the hawt new project hawtio we are also working on some great new stuff with elastichsearch around log aggregation and analytics, with a shiny HTML5 graphical user interface using Kibana. If you haven't see hawtio yet, then make sure to check it out.

Then James Strachan created the new camel-rx component, for integrating Camel beautifully, as Eric Maijer said, with the fantastic Netflix port of Reative Extensions (RX) library. And when we got Java8 as well, this is going to rock.

Then I created the new camel-servletlistener component to allow bootstrapping Apache Camel applications in web applications with no other dependencies. Usually people would need to use Spring or other library to do this. To demonstrate this in example, we have a new servlet-tomcat-no-spring example. As well my blog entry - Camel web applications without Spring, earlier this year about this new functionality. For the upcoming Camel 2.12 we will work on an alternative example using the new blueprint-web module that allows Spring XML like configurations but with only two JARs as dependencies (blueprint-noosgi, blueprint-web). This works beautifully, and we already use it hawtio project to easily bootstrap hawtio web application from a blueprint XML file.

Scott Sullivan stepped up and created the new camel-sjms component for a light-weight JMS component which only depends on the JMS API. This component is expected to be further improved and hardened in the upcoming releases. The camel-sjms component is not a 1 to 1 replacement for the existing camel-jms component. They are two independent components. We want the freedom in camel-sjms to implement the functionality we think is best needed, and as well to avoid the many many options that Spring JMS exposes, and hence creeped into camel-jms as well.

We also have a new component for integration with Redis with the camel-spring-redis component.

And last year I created the camel-urlrewrite component, which allows people to do Camel routes for proxying HTTP services with URL rewrites. I have previously blogged about this as well.

We also created a new control bus component, which allows you to send messages to a control-bus endpoint to control routes. This may make it easier for people to start/stop their Camel routes. This component is expected to be improved in the future, so you for example can get performance statistics and other information as well.


2) SQL component can now consume as well
The SQL component has been improved, so you can now consume as well. This allows you to pickup new data from table(s) and route the data in Camel routes. This is best illustrated by the new camel-example-sql that we created. Oh and we also added support for using named parameters in the SQL queries.


3) Groovy DSL
The Groovy DSL in Camel has been totally overhauled, thanks to community contributions. The DSL is now fully up to date and uses the Groovy'sh style that makes it much more Groovy like. We also added a new Camel Maven Archetype to create a new Camel groovy project.


4) CDI improvements
In the earlier phase of development of Camel 2.11 we worked on improving the camel-cdi component.
We are not there yet but its a big step in the right direction. We are also waiting a bit for Apache DeltaSpike project to do new releases so we can finish the last pieces. So expect this to be improved in upcoming releases as well.


5) camel-netty scales better
We also worked on improving the camel-netty component to be faster. Most noticeable the netty producer that now pools channels for reuse.


6) JAXB controlling namespace prefixes
For people stuck in XML land and using JAXB, then we made it easier to control namespace prefixes, so you 100% can control the prefix names in use. This allows to conform the XML to a naming style, or if you must use a specific prefix name; usually if a legacy system expects prefix names to be hardcoded.


7) Guice 3.0
People who are fan of Guice, would be glad to hear we have upgraded camel-guice to use Guice 3.0 as is. The old guicyfruit dependency, which was needed when using Guice 1.x is now gone.


8) Backlog tracer
We introduced a new backlog tracer, which allows tooling to trace Camel messages at runtime, on-demand. There is new camel-backlog-tracer command(s) for Apache Karaf / ServiceMix, which allows you to trace messages on your running Camel applications at runtime. You can even enable a predicate filter, to only trace matched messages etc.


9) OSGi upgrades
Apache Camel 2.11 now requires OSGi 4.3 and Apache Aries 1.0 if you use the camel-blueprint component. This means that you should use Apache Karaf 2.3 or better as the container.


10) Miscellanies
We have improved the startup of Apache Camel a bit, as well the simple language is now faster when invoking OGNL like expressions. And we disabled the type converter utilization statistics as there is a slight performance impact under heavy load. And we managed to let the camel-jms component re-create temporary queues when doing request/reply over JMS with temporary queues, and the connection has been re-connected (eg automatic self-heal). The camel-cxfrs component has a simpler binding, making it easier to use.

And as usual we have a ton of bug fixes, minor improvements and new features. See the release notes for full details.

Apache Camel 2.11 is available for download from the Apache web site, and Maven users can get it from maven central.

The Camel team is now busy working on Apache Camel 2.12, where we have some exiting new work on improved documentations for Camel components.

Apache Camel web dashboard with hawtio

I have been waiting patiently for James Strachan to write a blog to introduce us to his latest project - hawtio. This project surely has the potential to become as great or even greater than Apache Camel.

In a one liner, hawtio is:

hawtio is a lightweight and modular HTML5 web console for managing your Java stuff.

Today James is busy doing the release of the hawtio 1.1. This is a very hawt release, that has great functionality already, so we should start spread the word about this project. We have also talked about doing small videos, as recoding what you do in a web browser is easy.

Anyway this blog entry today of mine, is not to steal the thunder from James. I would let him do the introduction of hawtio, and his vision for creating this project.

So in hawtio we have a Camel plugin, that allows you to gain insight into your running Camel applications. So the screenshot below is from hawtio running as a plain web application deployed to JBoss AS 7.1.1. You can use any web container of choice, such as Tomcat, Jetty etc. Though I was tasked to give the JBoss AS a little test spin, and thus had the opportunity to take a screenshot. I have also deployed the Apache Camel example camel-example-servlet-tomcat in the container (its also just a WAR application).

Dashboard in hawtio, with a live Camel diagram and table, updating in real time.

hawtio allows you to create custom dashboards, where you can add any part of a view to the dashboard, and position the views as you please. So I created a Camel dashboard, which has a live diagram of the route from the camel-example-servlet-tomcat. As well a table with the running routes and their state and some key metrics. All the data on the dashboard updates in real time, which is very cool.

And the dashboards can be saved to a git repository, and you can share them with your team. And as its just a URL you can easily access the dashboard, by just copying the url from the browser address bar.

Anyway hawtio has a lot more cool stuff already. And we are having fun hacking on it. Its of course a 100% open source project, ASL licensed, and hosted at github. Everybody is welcome to contribute and hack on it.

Stay tuned, more blogs and videos should be coming in the future.