2008-10-30

Camel 1.5 a nifty new feature - Delay Interceptor

It's just a matter of hours before the votes passes and Camel 1.5.0 is GA.

However this blog entry is about a new nifty feature we have added in Camel 1.5. The delay interceptor. The idea is basically to slow things down, or to be more precise to slow down the routing. Camel do this by delaying the passing of the message from node to node in the route graph.

During either development or testing you are often in a situation where you are debugging or testing an issue and you push in some messages to your ESB platform. Then they are processed but things happens so fast in modern computers that the logging out is hammering with zillions of log lines and you kinda lost the picture what happened.

So instead of having to add a lot of Thread.sleep() or .delay() (yes we also have a delayer DSL) in the route we have this light weight AOP interceptor that delays the routing for you.

To enable it in for instance the Spring DSL you set the delay attribute on the camelContext. As shown below we set a delay of 3 seconds (3000 millis).
<camelContext id="camel" delay="3000" xmlns="http://activemq.apache.org/camel/schema/spring">
...
<camelContext>
This kind of interceptor has a sister so to speak, we have the Tracer that enables logging of the exchanges that are routed. Combining these two you have a powerful combination to easily help you at runtime to investigate what is happening.

2008-10-27

File consumer gotchas with CAPS

In my day job I get the pleasure of working with $$$ ESB platforms from large vendors such as SUN and IBM. Mostly SUN but I get the deployer role from time to time on the IBM platform.

Well this time I had some trouble implementing a file based resend feature. The idea is to drop the backup file in a certain folder and then CAPS will pickup the file and resend it as if it was the original request (webservice).

While working on this one I got side tracked as CAPS once in a while would drop this error in the log:
[#|2008-10-27T11:41:21.407+0100|SEVERE|IS5.1.2|STC.eGate.CMap.Collabs.CM.srvBackupFiles._3180_prjBrugerSikkerhed_3180_10_BACKUPFI_1417966593.jcdBac
kupFiles2CAPS|_ThreadID=149; ThreadName=Worker: 74;|[mammatus.rh.dk] [3180_10_BACKUPFILES2CAPS] [] [ERROR] [9] [] [3180_10_BACKUPFILES2CAPS] [CAPS]
[Batch Local File eWay error when doing file operation in [get()], message=[LocalFileClientImpl.get: Failed to get file to the payload.Nested exce
ption follows: ---
com.stc.eways.common.eway.standalone.streaming.StreamingException: LocalFileClientImpl.requestInputStream: Exception occurred.Nested exception foll
ows: ---
com.stc.eways.batchext.BatchException: Unable to find a file matching the Target Location configuration settings.Nested exception follows: ---
java.io.FileNotFoundException: Directory Name: /caps/projectdata/3180_EUP/resend/ File name: 8C803FD4-C7D8-A2BF-BA0B-5BF045012BA7.20081022-12145726
4-87-eup.xml--- End of nested exception.
--- End of nested exception.
--- End of nested exception.
].Nested exception follows: ---
After a while I realized it's the fact that CAPS will rename in progress files with a GUI prefix but keeping the original filename as postfix.

Original name: 20081022-121457264-87-eup.xml
In progress filename: 8C803FD4-C7D8-A2BF-BA0B-5BF045012BA7.20081022-12145726
4-87-eup.xml

So what is the problem here? Well the fact is that I have setup CAPS to monitor this folder and consume .xml files. So what CAPS will do is to also consume it's in progress file ;)

I haven't found a setting where I can change this to eg. let CAPS use a sub folder or a different file extension for in progress files.

Now I gotta change the regexp for matching the files to a avoid the CAPS in progress files.

2008-10-14

File Language - A new feature in upcoming Camel 1.5.0

A favorite feature of mine in the next Apache Camel is a new feature to use file name patterns in the file component. What we have introduced is a new language, the file language, so you can leverage this feature anywhere in Camel where it uses Expressions, Languages etc. The file language currently support both the file and the FTP component as well.

As a lot of integrations is file based you need to be able to read (consume) or write (produce) files. Then you need to be able to specify file names. What's new in Camel 1.5.0 is that you can express this using patterns, for instance directly in the configuration of your endpoints. Prior you had to do this in Java code.

Simple example
Lets image you need to move consumed files into a backup folder after processing.

Now you can express this using the pattern: backup/${file:name}
Well then you need to use .bak as extension, and the pattern is: backup/${file:name}.bak

Okay next issue is that the backup folder should be grouped by dates using the yyyyMMdd pattern. Well the expression support the java.text.SimpleDateFormat patterns also.
The pattern is: backup/${date:now:yyyyMMdd}/${file:name}.bak where now means current date. You can substitute this with different commands such as file for using the timestamp on the file instead.

Advanced example
In this sample you need to produce files (save) and you need to save the file using a generate unique filename. We use the divide and conquer pattern for this so you create a POJO class that generates the unique filename, then you as the developer have the full power how to do this. And then its very easy to unit test as its plain POJO that can be tested very easily with JUnit. Next step is to express this as a file pattern. As your POJO is a bean we use the bean langauge to invoke your pojo.

So the pattern is: myfile-${bean:myguidgenerator.generateid}.txt
Where
myguidgenerator is the bean id of your POJO and generateid is the method name. You can omit the method name if there is no ambiguity which method Camel should invoke.

Camel routes
This feature is configurable directly in camel routing on your endpoints. So what we can do now is:

from("file://inbox?expression=
backup/${date:now:yyyyMMdd}/${file:name}.bak").to("bean:processFile");

The route will consume files and process the files in the processFile bean, that is a plain POJO class. After processing the files is moved (renamed) using the pattern in the expression. So the file is moved to the backup subfolder, for example: backup/20081014/report-october-2008.bak

More to come
Well there is a ton of new features and improvements in Camel 1.5.0. Check out the current release note.