An introduction to Apache Camel - Blog Series

Recently on twitter I noticed some tweets about Apache Camel, which covers getting started with Apache Camel.

So far there are three blogs, and there are more coming. If you want to get more familiar with what Camel is, as well how to get started with a new project, and kick the tires yourself. Then the blog series is well worth a read.

Part 1 - Introduction to Apache Camel
Part 2 - Maven project setup for a sample camel spring dsl project
Part 3 - Writing Camel Routes Using Spring DSL based XML

Happy reading. And happy new year.

Apache Camel - A little Scala DSL example

So we have a Scala DSL in Apache Camel for many years now, and I guess its about time I wrote a little blog entry about this (has been on my todo list for a while). 

So the Scala DSL is of course using the Scala programming language which has many bells and whistles over plain Java. However the uptake of the Scala DSL is not very high, as the Java and XML DSL's is good enough for most people.

Anyway I guess one of the nice thing about the Scala DSL would be using closures as expressions and predicates etc. So let's do a little example using the Filter EIP pattern and use a closure as the filter predicate.

When using the Scala DSL you should use the org.apache.camel.scala.dsl.builder.RouteBuilder, which is the Scala empowered DSL. 

So in the following we have a FilterRoute class where we use the Scala DSL, when we define the createMyFilterRoute function.

class FilterRoute {

  def createMyFilterRoute = new RouteBuilder {

    from("direct:start")

      .filter(_.in("gold") == "true")

        .to("mock:gold")

  }

}

As you can see from the code above, inside the scope of RouteBuilder we have the Scala DSL at our disposal. Then we use the Filter EIP which accepts a function with the Exchange as parameter, which gets defaulted into the _ symbol. The result of that function is the evaluated as a predicate, using the Scala powerful (but a bit scary) type system with implicit type converter and case matching etc. 

The in("gold") is a function on a Exchange wrapped we have in the Scala DSL which adds additional methods on the Camel Exchange (RichExchange), the in is a function that look up a header.

To unit test this route I wanted to use the existing and powerful camel-test module. This module offers the CamelTestSupport class you can extend for your unit tests. 

So the unit tests can be almost like in Java, but you would need to add the trait org.apache.camel.scala.dsl.RouteBuilderSupport which helps bridge the Scala RouteBuilder with the Java RouteBuilder, that the CamelTestSupport expects and uses. The code below shows an unit test example.

class FilterRouteTest extends CamelTestSupport with RouteBuilderSupport {

  override def createRouteBuilder() = new FilterRoute().createMyFilterRoute

  @Test

  def testFilterRouteGold() {

    getMockEndpoint("mock:gold").expectedMessageCount(1)

    template.sendBodyAndHeader("direct:start", "Hello World", "gold", "true")

    assertMockEndpointsSatisfied()

  }

  @Test

  def testFilterRouteNotGold() {

    getMockEndpoint("mock:gold").expectedMessageCount(0)

    template.sendBodyAndHeader("direct:start", "Hello World", "gold", "false")

    assertMockEndpointsSatisfied()

  }

}

As you can see we use the RouteBuilderSupport trait, and then override the createRouteBuilder function to return the Scala DSL empowered RouteBuilder we created previously. The rest of the code is standard and plain Java code with JUnit @Test annotations.

This example is provided in the source code of the camel-scala module, as part of an unit test.

If you are a Scala fan and interested in Camel as well, then the Camel community could use people who are dedicated to Scala and help with the Scala DSL. The Camel team is often busy with other issues in our lives, so we only have a bit time to have fun and play with Scala.

Upcoming webinar - Getting started developing with Apache Camel and Fuse IDE

I am giving a webinar next week on getting started developing with Apache Camel with Fuse IDE.


Date: December 15th
Speaker: Claus Ibsen
Time: 11:00AM EST


This webinar is a hands-on practical tutorial on how to get started integrating applications with Apache Camel using Fuse IDE.

The webinar begins with a brief overview of Apache Camel, and why it makes integration tasks much easier. We then quickly get practical and develop our first integration application using Camel and Fuse IDE. Slowly we progress and show how to implement integration solutions using, but not limited to, files, ftp, jms, web servies, csv, xml, objects.

We also work with a number of Enterprise Integration Patterns such as Content Based Router, Recipient List, Splitter, Aggregator, Message Translator, Service Gateway and Idempotent Consumer. This webinar also touches on how you can gain insight into running Camel applications, to better understand what's happening, using message tracing. All together this is a very practical webinar with fewer slides and more live code and development.

Accompanying this webinar we have an eBook with sample code to ensure that after the webinar, you have material to successfully get started integrating applications with your Apache Camel journey.

You can register here.

Java Day Riga

I was invited to speak at the Java Day Riga Conference.

The conference took place at the Tallin Riga Hotel, which is a very nice hotel, just opened a couple of years ago. The hotel is also placed in the city centre, with walking distances to the nice areas.

I would like to thank Dmitry, for inviting me, and Oracle for sponsoring this event. According to Dmitry, the event would not be possible without the good faith from the Oracle. The event was free, so we had about 200+ attendees.

I like the badges, having a big and easy to read name.

Conference badget with easy to read names

The evening before the conference, we went out on a guide tour to see Riga. We hired a history professor, whom was very enthusiastic and knew a lot about what has happened to Riga over the many years. Riga is a very old city, named after the river, and it has been taken over many times between Germans, Polishes, Swedes, Russian etc.

Monument of freedom in Riga centre

The big monument in the city centre is the monument of independence, and on top it holds three stars, one for each of the three regions of Latvia.

Unfortunately after one hour it become too cold to pay attention, and we have already seen 4 churches, and many historical buildings. We found a place selling mulled wine, so we could get a bit of warm.

History Tour in Riga. Drinking mulled wine to get a bit warmth

After the tour we went for a later dinner, and just a few beers. The conference starts the next morning.

The keynote talks was good talks about the upcoming lambda expressions in Java 8. As well a talk about JEE 7, which has focus on the cloud.

Martijn Verburg preparing for his talk

Martijn Verburg, did a great talk titled "Diabolical Developer". It's a sort of pep-talk with common sense, where Martin says what some people may think is controversial. But what he really want to say, is that "think for yourself".

Martijn thinks developers are awesome

I gave a talk about Enterprise Integration Patterns and Apache Camel. In the talk I also demonstrated live coding, how to setup a new Camel project, and get that up and running in Eclipse etc. As well some of the powers of the Fuse IDE, which has graphical notation of your Camel routes, using the industry standard EIP icons. Likewise Fuse IDE is capable of doing runtime insights into running applications, which helps you pin point what is going on, as well being able to see performance statistics, with graphical representation etc.

In the end of the talk, the audience was giving a choice of either talking about Apache ServiceMix or the Camel roadmap. The audience chose the latter, so I gave a round about what to expect in the next Camel 2.9 release. Then we ran out of time. A talk for 50 minutes, is frankly a bit on the short side. 1h is usually the minimum.

In the airport on my way home. I spotted the Camel.

Camel spotting at the airport

Splitting big XML files with Apache Camel - Part 2

In my previous blog about splitting big files with Apache Camel, I said we were working on another solution, which is a new camel-stax component. The work is now complete, and the component will be part of the next Apache Camel 2.9.0 release. Thanks to Romain for his contribution.

The stax component
The stax component allows you to split big XML files as well, but it requires using JAXB and StAX. This means you need to define a POJO class(es) with JAXB annotations, to bind to the XML schema.
However the benefit is that you then work with the POJO classes in Camel.

For example the records example from the previous blog could be written with camel-stax as follows in the Java DSL


from("file:inbox")
  .split(stax(Record.class)).streaming()
    .to("activemq:record");


Where Record is the POJO class which has the JAXB annotations. And stax is a static import from the class org.apache.camel.component.stax.StAXBuilder.

If you are using XML DSL, then consult the camel-stax documentation which has such an example.

A little test
So I run the equivalent test from the previous blog as well with 40.000 elements, and the memory usage delta was about 8mb.
The test logs the time: Processed file with 40000 elements in: 55.962 seconds.

Running the test with 200.000 elements results in 9mb memory usage delta, and test time of: 250 seconds.

The unit test is in camel-stax component as the org.apache.camel.component.stax.StAXXPathSplitChoicePerformanceTest class.

Apache Camel 2.8.3 Released

This is just a quick blog entry to say that Apache Camel 2.8.3 has been released. The JIRA tracker has about 60 tickets resolved for this release.

Apache Camel 2.9.0-RC1 Released

The Camel team is working hard on the last pieces for the upcoming Apache Camel 2.9.0 release. In the mean time we decided to cut a release candidate; due to some larger changes like core API refactorings, Spring dependency changes, rewritten simple expression language, etc.

We would highly appreciate any feedback from the community in terms of any upgrade glitches, or other issues discovered in the release candidate.

The release is available to download from Apache, and as well from Central Maven repo.

For the release notes we suggest to take a look at the current in-progress release notes for the 2.9.0 release.

Coffe Machine and Camel in Action

Jonathan and I got published at the very end of last year when Manning announced that the Camel in Action book was available in print. As authors we are entitled to royalties of the sales of the book. Before you ask, we only get pocket changes compared to the amount of work we put into the book.

So what's the story with coffee then? Well as I have a home office I have to make my own coffee, as there is no fancy coffee machines around I can use. So for years I have been living off regular filtered, instant or stempel -coffee.

I made a promise to myself that I would buy a coffee machine when I get my royalty cheque. Well the cheque has arrived a month ago.

There is a lot of different types and brands of coffee machines. So I spend a while reading the web and watching you tube reviews of the machines. I also got advice from Johan Edstrom, who has a machine.

As I do not want to be my own barrista I was looking for a full automatic machine. At first I got my eyes on a machine from Gaggia, but recently spotted a new machine from Jura. I wanted a small machine as the daily use would be at my home office for a single person; our dog don't dring coffee, and my wife only drinks coffee in the weekends. The latest Jura ENA 9 Micro seemed like a great machine. It's small and compact, full automatic, and has a nosel to take in milk directly from the carton (or from a thermal bottle). So basically it's a one push button machine making coffee, expresso, cappuccino or latte.

So last wednesday I put in the order, and this morning the machine arrived at my doorstep.

My new coffee machine

The machine costs about 1300$, which would have been out of my normal price range. Well I could of course take the cash out of my regular pay cheque, but I wanted to stick to what was affordable from the royalties. That would also justify going for a higher priced model.

If I have to guess how many hours I have spend working on the book, then 1300 hours would be a good guess. So that's 1$ per hour. And that's before tax.

Just wanted to share this with the readers, so you know how I spend my royalties.

Thank you, the readers, for making my coffee a pleasure to drink from this day forward.

Splitting big XML files with Apache Camel

In the upcoming Apache Camel 2.9 we have improved the support for splitting big XML files using streaming and very low memory footprint.

In previous versions and examples provided on the Camel website, often showed examples of using XPath to split XML files using the Splitter EIP pattern.

Unfortunately the underlying XPath framework do not support an iterator based result, as its limited to the types defined by the JDK in the XPathConstants. That means a NODESET would be used as result type, which causes the XPath framework to return a NodeList instance which contains the entire XML payload in memory. There is nothing you can do about this despite using StAXSource, SAXSource or other stream types as input to the XPathExpression. Regardless what it would return a NodeList as result.

Tokenizer solution
So the Camel team have two solutions in the works. The first is already implemented in the upcoming 2.9 release. Its based on the tokenizer language which supports an iterator stream based. This means we can split any big file one a piece by piece without causing the entire content loaded into memory. So I enhanced the tokenizer to support two additional modes:

• pair
• xml

pair mode
The pair mode is to be used when you need to grab piece by piece and you have a known start and end tokens to denote a record. For example if you have a [START] and [END] markers in the content.


from("file:inbox")
  .split(body().tokenizePair("[START]", "[END]")).streaming()
    .to("activemq:record");


xml mode
This was used as the foundation for the xml mode as well, as the idea is similar. So you define a child tag name as the record to grab. For example to split by then you do as follows:
from("file:inbox")

  .split(body().tokenizeXML("record")).streaming()
    .to("activemq:record");

The XML content may look like
<records>
  <record id="1">
    <!-- record stuff here -->
  </record>

  <record id="2">
    <!-- record stuff here -->
  </record>
  ...
  <record id="99999">
    <!-- record stuff here -->
  </record>
</records>

Now what about namespaces? Suppose you have a common namespace in the parent/root tag as follows:
<records xmlns="http://acme.com/records">

  ...
</records>

Then you can instruct the tokenizeXML to inherit namespaces from a parent/root tag by providing the name of the tag as the 2nd parameter as shown:
from("file:inbox")

  .split(body().tokenizeXML("record", "records")).streaming()
    .to("activemq:record");

Which means each splitted message will contain the namespace included:
<record id="1" xmlns="http://acme.com/records">

  <!-- record stuff here -->
</record>



What I like about the tokenizer is fully stream based and returns data as String content, which means there is no intermediate DOM or POJO objects or anything like that. Which mean it can split any kind of XML payload without having any model of it in the java code. So if you just need as in these examples to split the big XML file and send each splitted message to a JMS queue, then that is fast as there is no unnecessary to/from object marshaling.

A little test
I ran a little test on my laptop to process 40.000 records and the memory usage delta was about 4mb.
The test logs the time: Processed file with 40000 elements in: 53.676 seconds

Running the same test with XPath reneders a memory usage delta about 100mb.
The test is in fact a little faster: Processed file with 40000 elements in: 49.941 seconds

The reason is that after all content is loaded into memory, then its a pure CPU processing, where as the tokenizer does load the content from disk piece by piece.

That was just a small XML file with 40.000 records with a file size of about 7mb.
Now image if the XML file was a 500mb size with a million records. The XPath will be very slow and most likely cause a OOME exception on your server.

The unit tests is in camel-core which you can play with in the src/test/org/apache/camel/language directory.

What about the other solution
Its a community effort together with Romain who have created a Camel StAX component with a stream based iterator as well. However it requires a POJO model that has been JAXB annotation to be used. We will continue working on this and have his work contributed into the Apache Camel distribution.

Todays Webinars about Database Integration with Camel

Today my colleague Charles Moulliard is giving a webinar about database integration with Apache Camel. Charles will give the presentation twice today, one time slot suited for EMEA and another for US.

Enterprise integration projects often include databases, and creating Apache Camel message routes in the context of database connection and transaction management are nontrivial. The first part of this 2-part webinar series will introduce you to several integration components and demonstrate how easy it is to configure them to give access to the information persisted in a database. Particular attention will be given to the pros and cons of each component, and when to use each. We will also cover how to avoid losing messages during database connection calls using transactions.

Links for more details

• EMEA URL: 9;00AM GMT
• NA URL: 11:00AM EST

If you miss the webinar, then its recorded and will be available later on demand.

About Camel 2.7.4, 2.8.2 and 2.9 Releases

Today the Camel team announced the availability of the Apache Camel 2.7.4 and 2.8.2 releases. These release are mostly bug fix releases to ensure stability on those branches. Albeit 2.8.2 has more than 100 tickets resolved. So I suggest to take a bit extra care if upgrading to 2.8.2 due the many changes.

In terms of the upcoming Apache Camel 2.9.0 release, then we have started discussing this at Apache. The plan is to cut a RC1 release first, to give the community a chance to give it a test spin and provide feedback. The reason is because Camel 2.9.0 have a number of API changes and cleanup in the camel-core. Therefore we would love for people in the community to download and try out RC1, especially if you have plans to upgrade to Camel 2.9.0 or later. We would like feedback on the upgrade issues you may encounter, and we will try to correct them before the actual 2.9.0 release.

If everything goes smoothly then hopefully Camel 2.9.0 is GA in end of november. This release has some great wins for the community (such as the improved simple language), which I will blog about later.

Fish and Chips in London, and Oysters in Paris

Last week I was speaking at the Fuse Community Days in London and Paris. This is a blog entry of my impressions from that trip.

So this was my 2nd time I attend and speak at the London Fuse Community Days, which is located in the Skills Matters facility. The facility is a bit rustic and gives an impression of more informal and hands-down atmosphere. This is what I really like about these community days. It gives every body attending a great opportunity to meet and great and share ideas and solutions.

Fuse Community Day in London, at Skills Matters

It was a great day, and I spoke to many Camel end users, who are not necessary FuseSource customers, but they all spoke well of Camel - there is a lot of love for Camel. For example Julien from BBC who I helped by implementing the done file function for the file and ftp components.

However this blog title is about Fish and Chips, so the day before I went on a stroll in London, and find myself a nice little restaurant where I could enjoy the trademark dish of England - Fish and Chips.

Fish and chips with peas in Lodon

After the Lodon event, we traveled to Paris for the upcoming community day. The day before the event we went out with fellow speakers to a fancy restaurant in Paris. Because we was a large party we were only given two choices for starters, main and desert. As I really dislike four grass I had to go for the oysters. I prefer my oysters spiced or with soya etc, not raw, the seafood taste is not a favorite. As I did not finish up all my oysters the waitress though I disliked them, and asked if I wanted a salad or something. Well the oysters was fine, but I cannot eat 10 for starters. The main dish was a big steak, with french friese. Not as fancy as I would have thought. The desert was some italian dish, which is bonds with ice cream and hot chocolate on top.

Debbie pouring hot chocolate on my desert. Its Larry Alston or CEO next to me.

It was time for a photo of the desert, so Debbie setup and poured more chocolate on my desert. After the restaurant we had a late night beer at the hotel, before turning in.

The next day we attended the Fuse Community Day in Paris. It was another successful day. This time we have a mix of talks in English and French. Even though it was my 2nd time in Paris, my French haven't improved beyond "bonjour". I am ending this blog entry with "au revoir".

CamelOne 2011 Videos Available

I have previously blogged about my thoughts about the CamelOne 2011 conference, where I said all the videos will become available.

Word of mouth is that more videos is now online, which you can find here.
I had a great time at CamelOne and looking forward to next year.

Upcoming Fuse Community Days in London and Paris

I am going to be speaking in London on October 11th and Paris on October 13th speaking at free Fuse events -  if anyone would like to meet-up for a beer and chat then come meet us.

I have the pleasure of doing a co-shared talk with James Strachan about Camel, Enterprise Integration Patterns and Fuse IDE. Knowing James I am sure you will get a glimpse of the latest installment of the Fuse IDE which has some remarkable new features and ... yes looks much nicer.

Skills Matter has provided a site with an agenda and more details you can check out here.

FuseSource UK Community Day
Tuesday 11th October
Skills Matter, London, EC1V 7DP


FuseSource FR Community Day
Thursday 13th October
La Défense, Paris

So if you are nearby those locations, then I suggest to stop by. Its free and ad-free.

Todays webinar - Getting Started with Apache Camel Fuse IDE

Today FuseSource is hosting a 1 hour webinar on getting started with Apache Camel with Fuse IDE.

FuseIDE is a graphical, Eclipse-based tool for integrating software components that work with Apache ServiceMix, ActiveMQ and Camel and can help you with your messaging and integration projects.

Developing Camel applications with Fuse IDE in Eclipse

If you are not currently using Apache Camel you might not know that it is a powerful and flexible integration toolset that allows users to create integration routes between software components using the Camel domain-specific language (DSL). Fuse IDE goes one step further and allows users to connect components visually. With a simple drag and a drop developers and architects can create, debug, and deploy simple or complex routes and integrate systems quickly and easily.

The webinar starts at 5pm central european time.

You can find more details here, and how to register for the webinar.

Camel 2.9 - Much Improved Simple Language

I\m sitting outside and enjoying the last sun and summer we have here in Scandinavia. So I might as well take time to write this blog entry which has been on my todo list for a while.

Camel has a built-in expression language called Simple. It has been around for a very long time. It started out as a "templatish" (String interpolation) language, which allows you to define dynamic strings, using token placeholders.

For example you could set a mail subject header as follows:
.setHeader("subject", "Your order from ${header.company}")

 In Camel 2.0 we added support for operators in the Simple language. This was a good addition, as it allows end users to use Simple languages for predicates. In other words you have a built-in language in camel-core, which you can use in the Camel routes with EIPs that uses predicates.

For example you could route messages depending on the content using the Content Based Router:
.choice()
  .when().simple("${header.priority} > 10")
    .to("activemq:queue:high")
  .when().simple("${header.priority} > 5")
    .to("activemq:queue:med")
   .otherwise()
     .to("activemq:queue:low")
  .end()

As the roots of the Simple language was based on its templatish behavior, there parser was implemented using regular expressions. However as people may know, using regular expressions comes with a cost of complexity and being able to read and understand the fairly long patterns. So we have probably reached how far we could go with the Simple language.

Another problem with using regular expression is that, as an end user, you must enter the input correctly. If you had some typos in the input, then the regular expressions was not always capable of detecting those errors, and being able to report this as an error. Instead the predicate would fallback and pass as true. This leads to confusion and frustration by the end users, as they do not understand, why all their messages goes to the first predicate in their content based router etc.

Knowing this, I took the time to work on a new and improved Simple language. To avoid disturbing the existing source code in the trunk. I setup a new project at github, and asked people in the community to participate. Taariq responded and we worked on this project. A reason for using a repository outside the Camel trunk, was that this was an experiment, and we did not knew when we set sail, if this would become useable and better.

The new improved Simple language is based on the principles of a recursive descent parser, with a grammar, syntax parser, and ast (abstract source tree). We set a goal to not depend on any 3rd party frameworks such as ANTLR or the likes. The Simple language is simple, and do not need to overhead and complexity of ANTRL or Eclipse XText. There is already sufficient 3rd party languages you can use instead such as JavaScript, Groovy, Mvel etc. However maybe in the future we may develop a new Camel language outside camel-core, that uses ANTRL/XText if the community is looking for this kind, and its feasible to implement.

With the new syntax parser we are capable of detecting all syntax errors, and give a precise error report what is the problem. For example if you forgot to use == in the equals operator, and only have =, then you get a error as follows:

org.apache.camel.language.simple.SimpleIllegalSyntaxException:

unexpected character symbol at location 15

${header.high} = true

               ^

And there is a ^ pointer, which points you to the location of the problem.

This re-architecture of the Simple language, also allows us to add new operators and functions to the language without the implications of the old regular expression approach. The community was asking for being able to increment a counter using the Simple language purely from the XML DSLs. So this is now possible as follows:

<setHeader header="myCounter">

  <simple>${header.myCounter}++</simple>

</setHeader>

We have support for unary operators specified on the postfix side of the operator. So we could potential add other operators, if it makes sense.

I have also experimented with ternary operators, so you can do the conditional operator such as:

<setHeader header="bigSpender">

  <simple>${header.amount} > 1000 ? true : false</simple>

</setHeader>

Likewise we may add support for the elvis operator (?:) as a shorthand for the conditional operator:

<setHeader header="name">

  <simple>${header.username} ?: "Anonymous"</simple>

</setHeader>

Anyway what I wanted to say with this blog entry, is that the Simple language is improved in the upcoming Camel 2.9 release, so it will give you better error details when you have syntax errors, and all your troubles with why the content based router is rouging all messages to the first predicate is solved as well.

Video using Splitter EIP and aggregate with Camel

Today a colleague of mine from FuseSource posted a new 8 minute video on YouTube how to use the Splitter EIP with aggregator, so it composes the Composite Message Processor EIP pattern. The video covers the use case shown in the figure.

Use Case in Video

The video also demonstrates how you can trace messages at runtime, and see how the message is changed during routing, using the Fuse IDE tooling.

Apache Camel 2.8.1 Released

Today the Apache Camel team announce the release of Apache Camel 2.8.1. This release is mainly a bugfix release over 2.8.0, resolving 45 tickets. There is one new improvements, which is the camel-soap component now supports using multiple parameters.

The release notes is here, and a list of JIRA tickets resolved is here. You can download the release from Apache, as well the JARs is synced to Apache Maven central.

JavaZone 2011 - Cool Badge

In my previous blog covering my impressions from the recent JavaZone conference I forgot to tell you about the cool badges used.

JavaZone Speaker Badge - A 3.5 inch floppy disk

Its a 3.5 inch floppy disk. The speakers had yellow disks, and attendees black.

At the JavaBin stand (the organizers) there was an old computer with a disk drive. There was a rumor about a lottery, so by inserting the disk you were told if you won or not. The computer told me that I was a looser, using old scholl ASCII arts. This reminds me back in the good old Amiga days with the demo scene and whatnot.

JavaZone 2011 Conference

I am sitting in my hotel room in Oslo, Norway, and thought I ought to write this blog entry while the memories of the JavaZone conference is still fresh in my mind.

JavaZone is possible one of the largest conferences here in Scandinavia, and so far I am impressed. The venue is Oslo Spektrum, which is big multi arena with a huge exhibition floor. The speaker rooms is however a bit out of the ordinary. My presentation was in room five, which was up on the "second floor". The seatings in the arena is very steep and thus your audience is seated high above the podium.

Room 5, having steep seatings

My presentation was similar to some of the previous talks I have been touring with. So you may have seen the presentation at a webinar, or from the CamelOne video, or the likes. The video of my talk is available here. And the slides is here.

After my presentation, a 2nd Camel presentation was given by Rune Peter Bjørnstad, and Bjørn Nordlund. The presentation was in Norwegian, however as a dane I was able to understand almost all of it.

The 2nd Camel talk by Rune and Bjørn

This was perfect timing as Rune and Bjørn's presentation was naturally in line of my presentation. They covered a real life use case used by Nets.no, their employer. The story is how their implemented a solution to process and validate payment transactions in a brach of 1000 pieces. The story and how they presented it was very well done. They did live coding in the Camel Java DSL in IDEA and had a easy to understand web page which they used to test their application. They showed how deal with transactions, avoid losing messages, and how to handle in case the server crashed etc. All together a very good presentation. The video for their presentation is available here.

Later in the evening The Java Posse team went on stage.

JavaPosse on stage

The audience was packed and we had a good time. It was their regular show of having polls of your favorite language, mobile platform etc. And taking in questions from the audience. And thanks to Atlassian for sponsoring the beers.

The JavaZone conference is a very good conference. There is plenty of space in the exhibition hall, although it may seem a little crowded.

Exhibition Hall

There is a overflow room at the 2nd floor, which have 6 big monitor screens. Then you can sit there with a headset and pick and chose your sessions. This is very neat and popular.

Overflow room on the 2nd floor

What I particular like about the JavaZone conference is the excellent food and beverage. Its free and available all the time. So if you are hungry then you just go to a venue of your choice. You can have lasagne, sushi, tappas, fish, vegetarian, ice cream, and there is plenty of expresso machines so you can get a good cup of coffee. And the best, there is no big line of queue at the lunch break. This is something any other conference organizer ought to consider.

There was also a book store in the exhibition hall and I managed to spot Jonathan and mine book Camel in Action. By the way FuseSource is offering an exclusive 40% discount on the book, which you can find details here.

Camel in Action book at the book store

On the 2nd day in the afternoon I took a break from the conference and did a walk in the city of Oslo. The weather showed itself from the best side with sun and a nice temperature. The day before we had a all day of rain and clouds.

I went on my walk to the Munch Museum to see the arts of this excellent and famous norwegian painter. However his most famous painting - Skriget. Is not on display here but at the national museum which is a 30 min distance away using public transport.

Munch Museum in Oslo

When my wife sees the Skriget painting, it reminds her of a well known Danish politician - Willy Søvndal.

Villy Søvndal

I then purchased a poster of the painting, so we can have it hanging in my home office.

Skriget by Munch

If you think you know this painting from somewhere, but can't quite put it from where, then it may be from The Simpsons.

Homer Simpson as Skriget

I went back to the conference to catch a few talks such as the excellent talk about Coffee script. Which reminds me that I am in need of coffee, so I better end this blog post now.

CamelOne 2011 Flashback

I have had this blog article on my ever growing todo list. Now I am sitting in the airport waiting for a plane bound to Oslo, for the JavaZone conference. I guess airports manifest itself as good opportunities to write blog articles as you often tend to be without wifi, and have hours to kill.

So CamelOne is the first conference which is solely focused on the integration space and having the Camel project as the "star of the show". The conference took place in May 2011 in Washington DC. It was a full day event with 3 tracks, each having 5 sessions, and 3 key notes as well. So there is plenty of choices and great talks to go to. The agenda is online here.

CamelOne 2011

I attended the conference as a speaker, giving a talk, well of course, about Apache Camel. All the sessions was video recorded, and they are being put out in the public over the next course of time. The first number of videos is online, which you can find here.

From a Camel point of view, we had the pleasure of having Gregor Hohpe giving the key note.

Gregor giving the key note at CamelOne 2011

Gregor is the "main" co-author of the definitive Enterprise Integration Patterns book, which is a record selling computer book. I snapped a photo of Gregor and James sitting in the lobby of the conference center.

James Stachan and Gregor Hohpe 

In fact these two individuals is the reason why we have the Apache Camel project today. Gregor wrote the EIP book, and James wrote Apache Camel. Well the story is that Gregor and James know each other. Gregor mentioned the background of why he got started on the EIP book, as well as the story how James got the idea of Camel, at a party they had at Gregor house, back then in San Francisco. James basically said that someone should write a software implementation of the EIP patterns. And well yes a bit later he said ... well ***** ** I am going to do it. Gregors keynote is online here. Gregor also talks about the possibility of a EIP II book (fingers crossed).

The day before the conference took place, we had dinner and drinks. On the picture we have Debbie, Scott, James, Gregor, and Jonathan from my table. I forgot to snap a photo of the other tables.

Speakers Dinner

The day after the conference I had the pleasure of accompanying Gregor for a sightseeing tour in Washington. We started having a grand american breakfast at IHOP. So we got a stack of pancakes, hash-browns, egg and bacon.

American Dinner Breakfast at IHOP

Getting to the city center is easy as we caught the subway. Gregor had a little dispute with a subway attendee whom disliked the fact Gregor was taking photos of the subway station. Anyway we managed to get into the centre and was soon out in the nice sunny weather. We made our ways to the Washington monument. I was impressed with the height of the sphere, and the sheer size of the bricks.

Washington Monument

After that we walked down past the white house, but needless to say, Mr Obama was not at home, inviting us in for a coffee and a talk about integration and open source in the public sector.

The White House

So we walked down the alley and found the Smithsonian museums. At first we walked a couple of parks having modern sculptures on display.

Some famous sculpture, but some famous artis

The aviation and space museum was just next door, so we did our last stop here. I am really impressed with this museum. There are countless of space rockets, the moon lunar lander, space suites, and whatnot. You can easily spend hours walking and seeing something new. We only managed to catch a glimpse of what the museum had to offer by walking the ground floor.

Gregor looking at the return landing capsule

So in the afternoon we made our way back to the hotel as Gregor had to catch an earlier flight than I. So I spend the next couple of hours at the hotel, talking to a number of conference attendees, who was on the 2-day training sessions. Among others I had the honor of a drawing the raffle for the prices of four Camel in Action books as free giveaways, which Manning was sponsoring. So the winners was able to have me sign their book as well. At about 6pm I hailed a cap, and went for the Dulles Airport, on my evening flight back to Europe.

The CamelOne videos is being released online over the course of time. You can find them here.

Hope to see you next year at CamelOne 2012.

Cloud integration with Apache Camel

I actually just want to bring the attention of Kai Wähner who posts some great blog entries about Apache Camel. One of his last posts is about Amazon cloud integration with Apache Camel.

You can read his blog entry here.

Keep the good blogs coming Kai.

Apache Camel - Bean integration improved

I might as well continue this month with a blog post about a new feature in the upcoming Apache Camel 2.9 release I implemented a couple of weeks ago.

Apache Camel has always had great integration for using beans. You had all kind of options to pick and chose according to your needs and liking.

When Camel invokes a bean, it uses a strategy to adapt to the bean. It introspects the bean and using an algorithm if figures out the method to invoke and how to bind to the parameters of the method signature.
This algorithm is detailed in chapter 4 in the Camel in Action book, figure 4.5 and 4.6.

So what have we added in Camel 2.9? Well one area where the previous logic was a bit lacking was binding to methods with multiple parameters and by which some parameters would have to be instructed using annotations how to bind.

Imagine this method signature

public void doSomething(String payload, boolean highPriority)

What we would like to do from a Camel route is to invoke this method, with the message payload as the first parameter, and true for the 2nd parameter.

This is now more easy in Camel 2.9 as you can declare the binding directly in the Camel route as shown:

.bean(OrderService.class, "doSomething(${body}, true)")

This applies as well when using .to

.to("bean:myOrderService?method=doSomething(${body}, true)")

In fact Camel uses the Simple language in the syntax which allows you to bind message headers, properties, invoke other beans, property placeholder values etc.

In fact you can even invoke methods using the OGNL support from Simple. Now suppose the message body is a Java object which has a method called asXml(), and you would pass the result of this method invocation to the doSomething method. This can simply be done as follows:

.to("bean:myOrderService?method=doSomething(${body.asXml()}, true)")

You can omit the () and this time for the boolean parameter we bind to a message header with the key high

.to("bean:myOrderService?method=doSomething(${body.asXml}, ${header.high}")

You can read more details in the parameter binding using method option section in the Apache Camel documentation.

Upcoming webinar - Apache Camel - How to go from EIPs to production

Next week I am speaking at a webinar on Apache Camel hosted by FuseSource.

Based on the integration patterns developed by Gregor Hohpe and Bobby Woolf, Apache Camel originated four years ago. James Strachan, technical director and software fellow at FuseSource and co-founder of the project with Rob Davies – now CTO of FuseSource – posted the first commit of the project on March 19, 2007.

In the years since, Apache Camel has emerged as one of the most popular integration frameworks and a singular example of open source technology that gives developers the freedom they need to solve real world business problems cost effectively and quickly – all while benefiting from and contributing to one of the most vibrant and innovative open source communities, the Apache Software Foundation.

“This webinar is merely another reflection of our commitment to help more developers and architects realize the proven superiority and value of Apache Camel,” added Ibsen. “If you don’t use it, you are working harder at application integration than you should.”

The webinar takes place on tuesday August 23rd. You can see more details here and register for the event.

Two new Fuse IDE videos showing Camel development and runtime message tracing

The Fuse IDE, which is our developer tooling for Apache Camel, ServiceMix and ActiveMQ was recently released as 2.0. This new major versions introduces major new features in the area of insight into running applications. For example you can browse endpoints, JMX trees, view any Camel routes as graphical EIPs, and trace messages.

Browsing endpoints and JMX

At FuseSource we have created two videos to show this tooling in Action. The first video shows how you can create a new Camel project that throttles a web service between two business.

Throttling a service between two business

The 2nd video shows the Camel insight in action, where you can browse endpoints, send messages to endpoints, and trace messages as they are being routed. That means you can see a graphical representation of the route and step through the route and see the content of the message as it changes.

Tracing messages in Camel routes (single step)

You can see the videos from the following links

• Throttler Enterprise Integration Pattern Video
• Browsing and Tracing Video

We will create and publish new videos in the future to demonstrate other features of the Fuse IDE tooling. So stay tuned. You can read and see more details about Fuse IDE here. If you have feedback and issues using the tooling you can use our online forum to get in touch with us and talk to other users.

Apache Camel 2.9 - Reduced dependency on Spring JARs

One of the goals for Apache Camel 2.9 is to reduce dependency on Spring JARs in both the camel-core and other components.

In the core we used the Spring JMX annotations to more easily export Camel MBeans into JMX. This dependency is an optional dependency on Spring JARs as you would need those in case you want to manage Camel using JMX. Unfortunately Spring does not provide a light weight API for the JMX pieces, and you end up loading in 6+ Spring JARs. This is of course not desirable, so we moved forward a goal for Camel 3.0 to the upcoming Apache Camel 2.9 release, which is to provide a Camel set of JMX annotations and use pure Java JMX logic to enlist those.

This work has been carried out, and you can now use Camel JMX with no Spring JARs at all at runtime.

To be compatible with the Spring JMX annotations, we moved the old logic from camel-core to camel-spring component. So you can still use Spring JMX annotations in your custom Camel components / beans / processors etc. and have them enlisted in JMX, together with the other Camel MBeans.

The camel-core JAR in Camel 2.9 onwards now only depend on the SLF4j logging API. This is as low dependency we want to go.

The dependency tree report by Apache Maven is now simply:

[INFO] org.apache.camel:camel-core:bundle:2.9-SNAPSHOT
[INFO] +- org.slf4j:slf4j-api:jar:1.6.1:compile


A number of Camel components used the Spring IO Resource to load template files from the file system or classpath. This didn't work in OSGi blueprint containers, so we decided to use existing code from camel-core instead. This means a total of 11 components now do not depend on Spring JARs either. We also identified a component in camel-spring, which could be moved to camel-core. For example the XSLT component has now been moved from camel-spring to camel-core.

So what's left? Which components depend on Spring JARs? Well there is only camel-mail and camel-jms left. In the former we only use Spring to send emails, and there is a contribution from the community to use plain Java Mail API to send emails. So it should be a matter of time before camel-mail is migrated as well. The camel-jms component is built on top of spring-jms, especially its MessageListenerContainer abstraction. Therefore it is not feasible to remove Spring dependency. However we may consider creating a new light weight JMS component in the future, having less bells and whistles as the current component, but keeping it simpler and leaner.

So does this mean we dont like Spring? Well of course we do, but Spring is not the center of the universe. Apache Camel is used in a lot of different systems and environments. Its important that Camel can easily fit into your environment. For example if you use JEE then JEE containers offers a lot of the same capabilities as Spring does. Now you can more easily use JEE and Camel without dragging Spring along.

Apache Camel 2.9 will still be as powerful integrated with Spring as it has always been. There is no changes for any existing users who use Spring and Camel together.

Apache Camel 2.8 released

Its been a while since we released Apache Camel 2.7.0, on march 22nd 2011.
In between we have done three patch releases 2.7.1, 2.7.2 and 2.7.3.

So I am glad to say that Apache Camel 2.8.0 has been released. The binaries is synchronized to Apache central maven. And the Camel website will be updated in the near future with download links and release notes.

This release has a ton of great new features and improvements (as usual). A few things to point out. I have previously blogged about how we managed to speedup unit testing with Camel.

Two weeks ago James Strachan and I looked into improving how Camel discover type converters on startup. So we identified an improvement to avoid package scanning if not needed, which further optimizes Camel startup. This also helps run Camel on Java web start, and JBoss without the need for a special adapter.

David Valeri a new Camel committer (welcome onboard David), worked hard to make it much easier to setup TLS security in the various Camel components. Now you can configure SSLContext in the DSL.

We also improved testing using mocks, as you can replace a component directly with a MockComponent, and the uri of the endpoints will no longer complain about unknown parameters, as they previously did.

The restlet component has been upgraded to restlet 2.x and it supports the asynchronous routing engine.

Internal caches in Camel uses a soft reference cache which ensures the JVM can claim the memory if its running low on memory.

We have added two new maven archetypes for creating a new Camel project with Spring-DM or OSGi blueprint.

We have added more features to the AdviceWith feature you can use when unit testing Camel routes. Its now easier to add / remove / manipulate routes for testing. For example to tweak production like routes a tad before unit testing.

We added the option shareUnitOfWork to the Splitter EIP. This allows you to treat the entire split operation as a single unit of work, so in case of a rollback, the entire Exchange will rollback.

Exceptions thrown while handling other exceptions in OnException will now caught by a fallback error handler logging to 2nd exception and propagating the 2nd exception on the Exchange and causing that Exchange to break out processing and fail immediately. This avoids complications, what to do if a exceptions occur while handling a previous exception. For example you could end up going in circles. Now its predictable (Log the 2nd and fail).

And we optimized the internal routing engine in Camel to reduce the memory footprint used during routing.

Jean Baptiste a new Camel rider (welcome onboard Jean) added Camel shell commands to Apache Karaf, so you can view details about your Camel applications running in Karaf.

All the HTTP related components now support multi value HTTP headers.

And we have added 6 new components, and 2 new examples.

And for the Scala lovers we upgraded to latest Scala release, and adjusted the DSL slightly to use camel case naming, in the DSL.

The release notes has a ton more details about fixes, improvements and new features etc. As always make sure to read the notes before upgrading, as they contain information about API changes, and other changes that is important to know when upgrading.