Last weekend, I was supposed to write on chapter 10 for the Camel in Action book, but I got side tracked with the prospect of Camel meeting Elvis.
A FUSE customer is in due time to upgrade to from Apache Camel 1.x to 2.x and they are currently using the Spring XML to define routes. They are using the Content Based Router EIP with predicates. As Camel 1.x is not as rich in its Spring XML they had to write those predicates in Java code. In Camel 2.x the Camel EL, known as the Simple language, can be used for most common use case for defining predicates.
However they are using predicates which accessing a header which is a Map value, which they then index. This is currently not possible to do with Apache Camel 2.2, as the simple language can only, retrieve the header value.
In this light we could easily add special support for using a syntax like this: header.name[key] to support their use case. This was easy to implement, in a matter of minutes, ah okay, lets say 1 hour at most.
However then it occurred to me as I have previously thought of adding a OGNL to the Camel Simple language. Its not a full blown OGNL like: Mvel, OGNL or the dynamic programming languages: Groovy, JuEL, JavaScript etc. But still a simple OGNL that will raise the bar that the Camel EL can solve out of the box, from 80% to 95% :).
So in the weekend, my wife had other plans, leaving me with plenty of time to give it a go. After having implemented a fair bit of the OGNL so you can invoke methods, access Map/List etc. it occurred to me that Camel must meet Elvis.
Its of course not the king himself, but the mr Elvis Operator. Which means you can do ?. in the OGNL notation.
Suppose you have a User object on a Message and want to test whether an address has a zip code in a certain range.
For example you can do:
${body?.address?.zip} between 5000..5999
Notice how we can use the elvis operator to avoid the NPE. If the User object do not have an address the predicate will return false.
I have updated the Camel Simple wiki page with a few more examples, if you want to learn more.
And btw the Bean language also supports this new OGNL.
I have though of adding it to the Bean component as well, so you can do in an endpoint
to("bean:myBean?method=doSomething.doEvenMore")
But currently it cannot do this. Any thoughts?
This new feature will be available in Apache Camel 2.3.
PS: I have started on chapter 10
2010-03-17
2010-03-03
Apache Camel 2.3 - Overhauled Aggregator and introducing HawtDB
For the last couple of weeks I had the time to work on recoding the Aggregator EIP in Apache Camel.
Over the time we have gathered a list of issues with the old code, which you can find here.
The works is now complete and we got a much improved Aggregator in Apache Camel 2.3 onwards.
The new and improved aggregator
First of all it allows you to use multiple and independent ways of triggering a completion. For example you can say the say the completion should happen when 5 messages have been aggregated. Then that will wait until that condition occurs. So if that takes 4 seconds or 17 hours don't matter. You may also add an additional completion trigger such as an inactivity timeout. So if you think that waiting 17 hours is to long you can set the timeout to e.g 1h. The aggregated message will also contain a property which tells you which of the completion conditions triggered. So you can see if it was the timeout or the size etc.
On top of that the inactivity timeout is per correlation key. So if you aggregated with multiple keys then those are separated. In the old logic, the timeout was not based on inactivity, but really just a timer for a background thread to wake up, and start aggregating the messages which have arrived since last time.
This is really a great improvement as it works much more in the way end users will expect, that the timeout is based on inactivity and its per correlation key. So for example the timeout is not reset just because a new messages arrived on a another correlation key.
You can also add a 3rd completion trigger such a predicate. So you can use that to check if a special message arrived, or the aggregated message contains some special data that would trigger completion.
On top of that we have added support for letting the aggregator use parallel processing. That means you can have concurrent messages being send out of the aggregator.
And another easy of use change is that the AggregationStrategy is now mandatory, which means than the UseLastAggregationStrategy is no longer default. That should fix up some confusing when using it for the first time - e.g. why does the aggregator just spit out the last message, etc.
Other minor improvements is that you can chose what to happen if the correlation key is bad, and as well you can now "close" a correlation key to deny late messages.
Introducing HawtDB
Well that is not all, this blog entry is also about a new feature which I am a bit exited about.
With the help from the very talented people from ApacheMQ, such as Hiram Chirino, we have a new Camel component, called camel-hawtdb.
HawtDB is a lightweight and very fast key/value database. Its based on the sound knowledge from ApacheMQand its KahaDB persistent store.
So what we have done now is to integrate HawtDB with the Camel aggregator. Which means we now support persistence out of the box. That means the aggregated messages is persistent to a file store, which means you avoid loosing messages in case your server crashes, or is reliable shutdown.
This is just a start. We will in the future add support for using HawtDB in other areas with Camel, such as the resequencer EIP, SEDA queues, DeadLetterChannel, StreamCaching big streams, idempotent repository and elsewhere. If you got good ideas for use cases then let us know.
I have put together a small example which demonstrates the new aggregator and the persistence support. If you don't want to run the example, the link showed console output which is kinda like the sexiest we can do with integration software.
Over the time we have gathered a list of issues with the old code, which you can find here.
The works is now complete and we got a much improved Aggregator in Apache Camel 2.3 onwards.
The new and improved aggregator
First of all it allows you to use multiple and independent ways of triggering a completion. For example you can say the say the completion should happen when 5 messages have been aggregated. Then that will wait until that condition occurs. So if that takes 4 seconds or 17 hours don't matter. You may also add an additional completion trigger such as an inactivity timeout. So if you think that waiting 17 hours is to long you can set the timeout to e.g 1h. The aggregated message will also contain a property which tells you which of the completion conditions triggered. So you can see if it was the timeout or the size etc.
On top of that the inactivity timeout is per correlation key. So if you aggregated with multiple keys then those are separated. In the old logic, the timeout was not based on inactivity, but really just a timer for a background thread to wake up, and start aggregating the messages which have arrived since last time.
This is really a great improvement as it works much more in the way end users will expect, that the timeout is based on inactivity and its per correlation key. So for example the timeout is not reset just because a new messages arrived on a another correlation key.
You can also add a 3rd completion trigger such a predicate. So you can use that to check if a special message arrived, or the aggregated message contains some special data that would trigger completion.
On top of that we have added support for letting the aggregator use parallel processing. That means you can have concurrent messages being send out of the aggregator.
And another easy of use change is that the AggregationStrategy is now mandatory, which means than the UseLastAggregationStrategy is no longer default. That should fix up some confusing when using it for the first time - e.g. why does the aggregator just spit out the last message, etc.
Other minor improvements is that you can chose what to happen if the correlation key is bad, and as well you can now "close" a correlation key to deny late messages.
Introducing HawtDB
Well that is not all, this blog entry is also about a new feature which I am a bit exited about.
With the help from the very talented people from ApacheMQ, such as Hiram Chirino, we have a new Camel component, called camel-hawtdb.
HawtDB is a lightweight and very fast key/value database. Its based on the sound knowledge from ApacheMQand its KahaDB persistent store.
So what we have done now is to integrate HawtDB with the Camel aggregator. Which means we now support persistence out of the box. That means the aggregated messages is persistent to a file store, which means you avoid loosing messages in case your server crashes, or is reliable shutdown.
This is just a start. We will in the future add support for using HawtDB in other areas with Camel, such as the resequencer EIP, SEDA queues, DeadLetterChannel, StreamCaching big streams, idempotent repository and elsewhere. If you got good ideas for use cases then let us know.
I have put together a small example which demonstrates the new aggregator and the persistence support. If you don't want to run the example, the link showed console output which is kinda like the sexiest we can do with integration software.
Subscribe to:
Posts (Atom)
