2015-03-19

On this day 8 years ago Apache Camel was born

So its March 19th which is 8 years ago the first public commit of Apache Camel was pushed to ASF by James Strachan in 2007.

Using the lovely github browser its easy to have a flashback and take a look at what this first commit contains.

So James included APIs and concepts that end users of Apache Camel today would find familiar.

Lets start with the unit test of a Camel route, which is as shown below. The unit test includes three tests, one is a test of a simple route, that is just a from -> to as shown below:

RouteBuilder from the very first commit - sometimes we hit the nail from the first try

If you look at the code above, then that code would be 100% the same today. Its a route using the Java DSL using the same class name, RouteBuilder, where the routes are defined in the configure method. And this route is a simple from -> to using the seda endpoints.

The other two unit tests are using the content based router, aka choice in Camel.

Camel route using choice and predicates
These routes would almost be the same today, its only the predicate in the choice that has changed from:

headerEquals("foo", bar")

to a more rich DSL

header("foo").isEqualTo("bar")

If we keep looking in the first commit, we see that it includes APIs and concepts we use today

org.apache.camel.Endpoint
org.apache.camel.Exchange
org.apache.camel.Predicate
org.apache.camel.Processor
org.apache.camel.builder.RouteBuilder

It sure was a great start from the very first commit.

In summer 2007 Apache Camel 1.0 was released, which I had a flashback in 2012 and blogged about what I though was a great start for Camel.

And here 8 years later Apache Camel is as strong as ever.
Just take a look at the latest impressive Camel 2.15 release.


2015-03-15

Apache Camel Eclipse Tooling with Debugger

About 2 months ago Lars Heineman blogged about improved Apache Camel Eclipse tooling on the way as part of the JBoss Tool stack

In the upcoming release they have integrated the Camel debugger with the native Eclipse debugger, so you get the Eclipse debugging experience, when you use breakpoints, single step through the Camel routes. And you can of course see all the details of the Camel Exchange and Message on the way. And as well change the data on the fly.

A screenshot is shown below, which I have borrowed from Lars's blog.

Eclipse debugging your Apache Camel routes at runtime.
Lars also worked on adding support for editing Camel endpoints uri's using a properties panel, so each option is provided individually. That work was based on Apache Camel 2.14 that has some support for this.

But since we have improved this tremendously in Apache Camel 2.15, which I recently blogged about. And therefore Lars is currently working on upgrading to Camel 2.15 so the Eclipse Tooling becomes even better.

Now imaging that we take the properties panel based on Camel 2.14 and add all the extra information and documentation we have from Camel 2.15, which will allow Eclipse to present a similar enriched properties panel as hawtio can do.

Using Camel 2.14 showing an empty properties panel (no documentation, no information about defaults, required, limited enums, etc, consumer vs producer option etc.)

Eclipse Camel properties editor based on Camel 2.14

And below the enriched Camel 2.15 which has all of above information, show currently in hawtio.

Hawtio Camel properties viewer based on Camel 2.15

So imagine that the Eclipse properties panel will be able to include out of the box:

  • documentation
  • default values
  • enum types (eg choices to select among)
  • required vs optional
  • deprecated
  • simple and java type
  • option as part of uri path or query parameter
  • consumer only option
  • producer only option
  • custom category for the option (eg security, advanced, etc.)

And with Camel 2.15, we are able to do this for all the components - they all provide all this information.

Camel 2.15 also brings to the table, that it would allow the Eclipse tooling to dynamic generate the EIP palette, as Camel include information about all the EIPs and their options as well. So imagine the Eclipse tooling is able to adjust to which version of Camel you are currently using in the project. And yeah all the EIP options is now documented as well, which the tooling can present to you.

I am really exited about the possibilities that Camel 2.15 brings to the table in terms of tooling and also runtime experience we can enhance.

We also work on JBoss Forge commands that allows to add Apache Camel to existing projects, to dockerize and/or fabric8 enable the projects - that is something for another blog. But as part of this work, we are working on commands to add/edit Camel components/endpoints. So the idea would be they can show all the endpoints uris in your project, and present a nice properties editor for you to have "type safe" editing.

Going back to the title of this blog. Yeah great work Lars and the Eclipse team, we now have a great Apache Camel debugger. And its using the same Camel debugging API that hawtio uses also - no magic trick. In fact this week I was in talk with a company that has built their data integration platform on top of Apache Camel and also leverages its debugging api, to allow its developers and users to debug the deployed Camel routes, on the platform.


2015-03-11

In the year 2015 - Apache Camel 2.15.0 was released

So it will be known to mankind that in the first quarter of the year 2015, the awesome Apache Camel 2.15.0 was released.

Apache Camel 2.15.0 was released in 2015

Top 10 of great new features

I will try to do a top 10 of new functionality this release brings to the table.

#1 - Self documented

This release brings together, the steps we have taken over the last couple of released, in terms of being self documented. So what does this mean?

I have previous blogged about the work we have done


So what does this mean? It means that the same documentation Camel provides for all its:
  • Java and XML DSLs
  • Enterprise Integration Patterns
  • Camel Components
  • Camel Data formats
  • Camel languages
Is now included at both design and runtime. So for example if you design Camel routes in either Java or XML then the documentation is at your fingertips. For Java its documented as javadoc, and for XML that same documentation is now injected into the Camel XSD schemas, which allows IDE tooling to show you that documentation.

Here is a screenshot of IDEA editing a Camel route in XML and showing the documentation for the splitter EIP.

Camel XML now includes EIP documentation out of the box
At runtime your Camel applications and routes can present the documentation as well, but with the extra twist that the documentation can overlay with the runtime configuration in use. In other words we can now show what all those EIP options, component options, endpoint options, and whatnot means, and what their current values are, and what their default values are, whether they are required or options, which type they are, and if they are enum based, what options the can be, and much more.

All that comes together nicely if using the latest hawtio web console, which can present this nicely as the next screenshot shows

hawtio showing EIP endpoints at runtime with current values and documentation
To access the documentation at runtime, then Camel offers Java, JMX and Camel commands as the APIs. hawtio uses the JMX API.

#2 - Categorized components

All the Camel components have now been categorized, which means we can group the components, for example to know which Camel components there is for SQL, social, messaging, file, and so on. That is not all, we have also categorized all the EIPs, data formats as well.

And that information is available at both design and runtime, just as with the documentation. And this information can be reached using Java, JMX, and Camel commands. For example the Camel commands now has a set of Camel catalog commands that can be used, to find out which components we have for social, database, etc. The following screen shot shows the Camel commands in action using the new Camel Catalog component to do just that.

Camel commands to list Camel components by group

#3 - Camel Catalog for Tooling

There is a new standalone camel-catalog JAR which is intended for third party tool developers (and also Camel itself). The JAR contains a list of all Camel eips, components, data formats, languages, Camel XSD schemas, and the Maven archetypes. And also full documentation for all those as well. In other words the camel-catalog JAR is an offline JAR that has all the information we have talked about in #1 and #2. The JAR has Java and JMX API, as well .properties, .xsd, and .json schema files for all that. So its easy for tools to tap into that information.

Apache Camel uses the catalog itself, in the Camel commands as shown in the screenshot above.

As mentioned before there is a set of new Camel commands being developed for JBoss Forge, which is intended for making development with Apache Camel easier. You can say that this is an example of a third party developing Camel tooling. There is commands to make it easier to add new Camel components, configure endpoints, and whatnot. What makes JBoss Forge awesome is that it allows to run the same binary commands (forge addon) in any of its supported environments, which are

- command line (eg CLI)
- Eclipse
- Netbeans
- IDEA
- web

So for example we have a Camel command to add a new component that allows to filter by groups. The following screenshot shows which components we have that is about files.

Camel Forge commands running in Eclipse, as a wizard to add a Camel component to your project
And the same command used from web (work in progress)

Camel Forge command running in web browser, as a wizard to add a Camel component to your project

#4 - Reuse Camel commands

In previous releases of Apache Camel, the Camel commands was only for Karaf users. But in this release we have made these commands reuseable, in the camel-commads-core module. And there is an out of the box implementation for Karaf and Jolokia. 

And for third party there is JBoss Forge Camel commands as part of the fabric8 project. By using these commands you can manage Camel applications at runtime, for example to check the state of your routes, and so on.

The screenshot below shows how to use the camel-routes command from JBoss Forge. The same command can be run from Apache Karaf as well.

Camel commands connecting to a remote JVM using jolokia to check the route status
The Karaf commands only works in the same JVM as Karaf itself. But the jolokia commands allows to manage remote JVMs as well. In the screenshot above I have a local running JVM with Apache Tomcat and deployed the standard Camel Tomcat Servlet example. And from a JBoss Forge JVM I use the commands to connect to the Tomcat JVM and list all its running Camel routes. If you are familiar with hawtio then its the same principle, as hawito is able to do that same as well.


#5 - Camel Boot / Spring Boot

For the micro services trend we have Camel boot supported, though most noticeable by the camel camel-spring-boot module. Henryk Konsek did a great job implementing and documenting this module, so there is plenty of material there to run Camel and Spring happily together.

For Camel Boot we are improving on alternatives with CDI, Guice, and plain Java in the upcoming release. For CDI in particular there is works in a much improved and working camel-cdi, which is expected to be included in Camel 2.16.


#6 - Improved swagger 

The camel-swagger module now supports any kind of runtime (before it was tied to Spring). In the next release we will add support for hosting multiple Camel applications from the same camel-swagger module, so you can deploy it once, and use it for all your Camel apps in the same JVM.

We are also hoping the swagger team finish their highly anticipated pure Java implementation, so we can drop Scala and make camel-swagger more friendly to use / and much lighter without the Scala libraries.


#7 - Improved Rest DSL

We have improved and fixed some issues with the new Rest DSL that was introduced in Camel 2.14 based on community feedback. For example its easier to configure using custom jackson modules for json. We also made it easier to return custom error messages when routing has failed etc.

We will continue to do so as we have still some outstanding work. 


#8 - Routing Engine optimizations

We also optimized Camel itself in particular two areas:

  • Optimized the case insensitive headers, but dropping the need for an internal shadow map.
  • Optimized the type converter to avoid conversion attempts that was not needed, which in some cases reduced the factor by 2-4 x during routing.

#9 - More detailed information about inflight messages

Camel now tracks inflight messages with more details, which allows to pin point exactly where in the routes the inflight messages current is being processed at. This information can be accessed from Java and JMX. And there is also a Camel command to list inflight messages. Also hawtio has a web page that list all the inflight messages with all that information as well.

hawtio showing inflight exchanges for the selected route. Notice the message is currently at "delay1" in the route

And the Route JMX MBean now also include a JMX attribute that contains the oldest inflight duration, which makes it easier for monitor tooling to quickly identify which routes may have messages that may "take too long time".

Also during shutdown of Camel itself, in case of the graceful shutdown was not able to shutdown and complete all inflight exchanges, then Camel now logs which exchanges are still inflight, and where in the routes they currently are.

#10 - Configuring endpoints in XML with long attributes can now break lines

When using XML for configuring Camel routes, you may use endpoints that has many options you need to configure, which then may make the endpoint uri long and less human readable. What you can do now, is to use line breaks in the attribute as shown below

Configuring endpoints in XML can now use line breaks (you can have multiple options in the same line)
An alternative is also that the less known can be used to configure endpoints, which now support style as well, as shown:

Configuring endpoints using property style


#11 - And as always a bunch of new components

And as usual we have a bunch of new components. This time there is 16 new components, for example for the talk of the town, Docker. And for Spring 4.0.x users there is a working camel-spring40-test module for testing. As camel-spring-test requires Spring 4.1.x.




You can find more details in the Apache Camel 2.15.0 release notes. And make sure to read the bottom of the release notes that details what to consider when upgrading.

The release can be downloaded from maven central, and the Apache Camel website.

2015-02-02

Your Apache Camel applications now includes out of the box documentation

A couple of months back I blogged about the upcoming 2.15 release includes functionality to obtain fine grained information about each property configured on your endpoints in the blog titled - Apache Camel please explain me what these endpoint options mean.

We have continued on this path, and today we have extended this from endpoints to cover EIPs, data formats, and languages as well. In a nutshell almost all what you use and configure in your Camel applications now has the capability of including documentation out of the box at runtime. In addition all of those are associated with label(s) which we use as a way of categorizing them. So you can list all components for database, cloud, http, rest, messaging, etc etc.

As the EIPs is provided in the camel-core those all their options is fully documented, including details about their default values, if the option is required, whether an option is deprecated, and of course type information etc. The EIP itself is although currently only documented with a brief summary.

As Camel includes about 175 components then it will take some time to include documentation about each one of them. But we will setup in the build process of Apache Camel a report that outputs which components has missing documentation.

Okay a picture is worth a thousand words, so let me include a bit of eye candy of this in action using hawtio - the web console.

First an example of the EIP such as the aggregate EIP which has many options.

Showing properties of the Aggregate EIP in our Camel application,
including documentation out of the box 
Notice in the top there is checkboxes to:

  • show/hide documentation (the documentation is always provided as a tooltip)
  • show/hide default values (not in current screenshot, as just added)
  • show/hide unused values

And another screenshot showing the properties of the endpoints:

Showing properties of an endpoint in our Camel application,
including documentation out of the box
This page is similar to the EIP page. It will also include the same checkboxes to filter.
Also notice those grey badges, they represent the label(s) this endpoint has been associated with. As we can see this is an endpoint from the core, and its scheduling.

All this information can be accessed at both runtime and design time. At runtime there is Java and JMX API from camel-core. And at design time the information is included in a new camel-catalog JAR that has json schema files for each EIP, component and so on. Likewise these json schema files can be retrieve at runtime using the Java and JMX API from camel-core as well.

So in other words this allows both end users and tooling to access this information.

For more information, read my previous blog, which I refer to in the top of this blog post.

What's Next

We have yet to implement a maven plugin as part of the build process of Apache Camel, that injects the EIP documentation in the XSD schema for the Spring and Blueprint XML DSL. When this is in place, then any IDE tooling should be able to leverage this, so while you edit those XML files, then the tooling can assist you and present the documentation etc.


2015-01-20

Webinar - Integrating microservices with Apache Camel

Tomorrow on wednesday 21st of january, our fellow Camel rider, Christian Posta is giving a webinar about microservices and Apache Camel.
Microservices architectures are all the rage these days, but we have to balance hype with reality. Microservices make it harder to manage your deployments and makes for complex inter-service communications patterns. How do you balance the tradeoffs and focus on getting the most out of your investment in highly scalable, decoupled systems?
Join this webinar to learn how to use patterns from SOA to build out intelligent routing systems with Apache Camel, and centralized management, service discovery, versioning, and tooling support from Red Hat JBoss Fuse for managing complex integrations using a microservice approach.
Date: Wednesday, January 21, 2015
Time: 16:00 UTC | 11:00 am (New York) | 5:00 pm (Paris) | 9:30 pm (Mumbai)
Duration: 60 minutes

You can find more details and register for the webinar using the following link.

2015-01-08

IDC Report on Business Value of using JBoss Fuse (with Apache Camel)

This is just a blog post that has more commercial nature, but you can't have one without the other. In fact this is what keeps Apache Camel alive and doing so well, due also to its commercial success.

This report may be of interest to people who are looking for hard evidence of the value add of using Apache Camel commercially, from products such as JBoss Fuse.
IDC interviewed 6 organizations that report achieving significant business value by using Red Hat® JBoss® Fuse, in particular, and making their application integration and development efforts more efficient and productive. These 6 organizations are achieving a 3 year average return on investment (ROI) of 488% and earning back their investments in JBoss Fuse in 8.2 months.
The report is downloadable as-is (no registration) from Red Hat website at: http://www.redhat.com/en/resources/value-red-hat-integration-products

Some of the highlights I see in relationship to Apache Camel are:


  • JBoss Fuse allows developers to program in Java, which becomes more of an extension to the application rather than a separate development experience. This resulted in making JBoss Fuse easier to adopt. As one customer explained, "One of the biggest advantages that we see with this product is that the integration tool is provided as a domain language. So it's like a natural extension of the Java language. You write integrations as if you're writing Java code. That means that all of the skills you need to debug in Java are the same skills required when you write integrations in this language. You don't have to go to another IDE or another toolset to understand how to write it. It's just Java code."
  • Another customer cited the team's desire to actually know and understand what the code is as a reason for adoption: "Before, we had a problem with the software, and actually being able to prove to the vendor that there was a problem before they would get around to fixing it is a lot harder if you can't actually see any of the code. And this is not a small issue … In order to lodge a request, we would often have to prove to them by writing a sample program and prove that they are the cause of the issue — in a way that it's reproducible in their environment."
  • A shipment and logistics company's application integration and development efforts benefit from the flexibility of JBoss Fuse's use of Apache Camel: "JBoss gives us the opportunity to help the developers in ways that wouldn't necessarily be possible with a graphical user interface tool."
  • Customers also said that they benefit from Red Hat's support and the ability to have access to and understand the code used for their applications. One customer praised Red Hat's support: "The customer service support from Red Hat has been exceptional. They gave us access directly to the developers who are writing the code, whereas it can be hard to actually get access to developers with other solutions."

  • Just a note to the last bullet. Our support program allowed the customer direct access to myself and other Camel core committers, so we were able to help them quickly, and also fix and improve Apache Camel based on their issues and findings.

    The report has more details, and hard numbers stating the "before" vs "after" and their gains (development time, performance, and many others) and cost benefits, such as pretty graphs as shown below:



    2014-12-31

    Some great Hawtio videos by Robin Huiser

    Recently Robin Huiser recorded three videos of demonstrating hawtio in action. I just wanted to help spread the word of his videos, as I enjoyed watching them.

    hawtio videos by Robin Huiser

    hawtio is a true agnostic web console that supports anything running in a Java JVM. Robin uses Talend ESB as his container for hosting Camel applications. You can of course run Camel anywhere as well, so you can also use Apache Karaf / ServiceMix / JBoss Fuse / Wildfly or any other platform of choice, and hawtio "just works".

    The three videos is about 10 minutes give or take, so grab yourself a cup of coffee and sit back and enjoy.

    Links to the videos on youtube.

    Have a great new years even and see you in 2015 for a great year of Camel and hawtio. There is a ton of new stuff coming in Camel 2.15 that makes hawtio and Camel even more awesome.