Camel in Action 2nd edition - To be done by start of January 2018

We just wanted to share the latest status of the Camel in Action 2nd edition book here a few days before Christmas.

We had hoped the book would have been done today. However there has been unexpected holdup due to the indexing of this big book is causing longer time to process than usual. In light of this Manning has pushed the schedule so the book is expected to be done by end of first week in January 2018.

This gives us, the authors, an extra opportunity to review all the pages of the book, yet one more time, which would be our 5th review. It's unusual that a book has as many final reviews as we end up doing with this book. Its all good in the end as it means that higher quality and that we will correct cosmetic details that may otherwise have been missed.

For example we spotted a glitch in figure 17.9, where the dashed box is not 100% horizontal in the bottom as shown in the following before vs after screenshots.

Figure 17.9 - Before with the glitch in the dashed line in the bottom not being 100% horizontal line
Figure 17.9 - After with the dashed lined in the bottom fixed

Manning has told us that they expect the page total in the print book to be around 900. In addition we have two bonus chapters that are available for download of readers, which are 26 and 18 pages long. All together is nearly a double of the number of pages of the first edition which was already a big book.

So hold on a little longer and we will be on a great start of 2018 with a up-to-date modern book covering all aspects of Apache Camel. The book uses the latest Camel release 2.20.1 at this time of writing, and we have some tips, what is coming in Camel 2.21 release, whenever relevant to our readers.

Merry x-mas and happy new-year

Claus Ibsen
Jonathan Anstey


Camel in Action 2nd edition - First layout done for print book chapters

I just wanted to capture a blog with the news that the Camel in Action 2nd edition book, first 19 chapters and 2 appendixes (the content for the print book) now has passed the 1st layout phase.

I took a screendump of the last page of chapter 19, which has the page count of 822. That means you get 822 pages of extensive details about Apache Camel in a single book.

Last page of chapter 19 with page count 822

At this moment Jonathan and myself are doing our author reviews on every single page of the those 822 pages to ensure code formatting, figures, technical errors, adding minor TIPs about upcoming/very latest Camel change, updating to Camel 2.20.1 release, updating screenshots in high-res resolution etc. All together to make sure you get the latest information in your hands, and that we have done our best all the way. I currently need to review chapter 18, 19 and 1 which is the outstanding, then I have read every single chapter from cover-to-cover for the N'th million time.

There is more content as we have 2 bonus chapters that will follow layout phase as well and made freely available on the book's website when the book is finished.

So we are on track to finish the book before x-mas, just in time for you to ask your significant other to buy you the best present ever to put under the tree ;)

Also we appreciate if you consider following the advice that we put at the end of the last page (see screenshot) on chapter 19.


List of people to thank for help with our book Camel in Action 2nd edition

The Camel in Action 2nd edition book is getting real close soon. Today the first three chapters just returned from the first layout process.

Here is a screenshot of the first page of chapter 1:

In the book we want to say thank you to people whom have helped us on the way in any form of shape, such as by sending us emails with reviews, feedback, or submitted fixes to the source code etc.

We have gathered a provisional list of people to thank.

The following people have been in direct touch with us about reviewing and feedback on the early access material:

  • Antoine Dessaigne
  • Bilgin Ibryam
  • Christian Posta
  • Clement Escoffier
  • Grzegorz Grzybek
  • Jospeh Kampf
  • Kevin Earls
  • Lars Heinemann
  • Luca Burgazzoli
  • Nicola Ferraro

The following people have helped with the book's source code:

  • Aurelien Pupier
  • Babak Vahdat
  • Christoph Deppisch
  • Grzegorz Grzybek
  • Kevin Earls
  • Luca Burgazzoli
  • Morten Erik Banzon
  • Ryota Sato
  • Scott Cranton
  • Willem Jiang

During the development of the book, Manning held 3 official reviews where anonymous readers provided feedback. At this time of writing we, the authors, do NOT have this list of persons. However Manning will insert the names of the reviewers in the final book.

In case you have provided us with help over the last 2,5 years of work on the 2nd edition book, then you are welcome to reach out to us so we can ensure to include your name as well. And we are sorry if we haven't included your name from the beginning, but we only started gathering these names over a year ago.

You are also welcome to contract us if you have helped us, but do not want to have your name included in the final book.

Claus Ibsen and Jonathan Anstey


Working with large messages using Apache Camel and ActiveMQ Artemis improved in upcoming Camel 2.21 release

Historically the Apache ActiveMQ message broker was originally created in a time where large messages was measured in MB and not in GB as you may do today.

This is not the case with the next generation broker Apache ActiveMQ Artemis (or just Artemis) which has much better support for large messages.

So its about time that the Camel team finally had some time to work on this to ensure Camel work well with Artemis and large messages. This work was committed this weekend and we provided an example to demonstrate this.

The example runs Camel with the following two small routes:

The first route just route files to a queue on the message broker named data. The 2nd route does the opposite, routes from the data queue to file.

Pay attention to the 2nd route as it has turned on Camel's stream caching. This ensures that Camel will deal with large streaming payloads in a manner where Camel can automatic spool big streams to temporary disk space to avoid taking up memory. The stream caching in Apache Camel is fully configurable and you can setup thresholds that are based on payload size, memory left in the JVM etc to trigger when to spool to disk. However the default settings are often sufficient.

Camel then uses the JMS component to integrate with the ActiveMQ Artemis broker which you setup as follows:

This is all standard configuration (you should consider setting up a connection pool as well).

The example requires to run a ActiveMQ Artemis message broker separately in a JVM, and then start the Camel JVM with a lower memory setting such as 128mb or 256mb etc which can be done via Maven:

  export MAVEN_OPTS="-Xmx256m"

And then you run Camel via Maven

  mvn camel:run

When the application runs, you can then copy big files to the target/inbox directory, which should then stream these big messages to the Artemis broker, and then back again to Camel which will then save this to the target/outbox directory.

For example I tired this by copying a 1.6gb docker VM file, and Camel will log the following:
INFO  Sending file disk.vmdk to Artemis
INFO  Finish sending file to Artemis
INFO  Received data from Artemis
INFO  Finish saving data from Artemis as file

And we can see the file is saved again, and its also the correct size of 1.6gb

$ ls -lh target/outbox/
total 3417600
-rw-r--r--  1 davsclaus  staff   1.6G Oct 22 14:39 disk.vmdk

I attached jconsole to the running Camel JVM and monitored the memory usage which is shown in the graph:

The graph shows that the heap memory peaked at around 130mb and that after GC its back down to around 50mb. The JVM is configured with a max of 256mb.

You can find detailed step by step instructions with the example how exactly to run the example, so you can try for yourself. The example is part of the upcoming Apache Camel 2.21 release, where the camel-jms component has been improved for supporting javax.jms.StreamMessage types and has special optimisation for ActiveMQ Artemis as demonstrated by this example.

PS: The example could be written in numerous ways, but instead of creating yet another Spring Boot based example we chose to just use plain XML. In the end Camel does not care, you can implement and use Camel anyhow you like.


Apache Camel 2.20 released - What's new

Apache Camel 2.20 has been released today and as usual I am tasked to write a blog about this great new release and what's the highlights.

The release has the following highlights.

1) Java 9 technical preview support

We have started our work to support Java 9 and this release is what we call technical preview. The source code builds and runs on Java 9 and we will continue to improve work for official support in the following release.

2) Improved startup time

We have found a few spots to optimise the startup time of Apache Camel so it starts 100 - 200 milli seconds faster.

3) Optimised core to reduce footprint

Many internal optimisations in the Camel routing engine, such as reducing thread contention when updating JMX statistics, reducing internal state objects to claim less memory, and reducing the number of allocated objects to reduce overhead on GC etc, and much more.

4) Improved Spring Boot support and preparing for Spring Boot 2

We have improved Camel running on Spring Boot in various ways.

We also worked to make Apache Camel more ready and compatible with the upcoming Spring Boot 2 and Spring Framework 5. Officially support for these is expected in Camel 2.21 release.

5) Improved Spring lifecycle

Starting and stoping the CamelContext when used with Spring framework (SpringCamelContext) was revised to ensure that the Camel context is started last - when all resources should be available, and stopped first - while all resources are still available.

6) JMS 2.0 support

The camel-jms component now supports JMS 2.0 APIs.

7) Faster Map implementation for message headers

If you include camel-headersmap component on the classpath, then Camel will auto detect it on startup and use a faster implementation of case-insenstive map (used by camel message headers).

8) Health-Check API

We have added experimental support for a new health-check API  (which we will continue to work on over the next couple of releases).  The health checks can be leveraged in in cloud environments to detect non healthy contexts.

9) Cluster API

Introduced an experimental Cluster SPI (which we will continue to work on over the next couple of releases) for high availability contexts, out of the box Camel supports: atomix, consul, file, kubernetes and zookeeper as underlying clustering technologies through the respective components.

10) RouteController API

Introduced an experimental Route Controller SPI (which we will continue to work on over the next couple of releases) aimed to provide more fine-grained control of routes, out of the box Camel provides the following implementations:

  • SupervisingRouteController which delays startup of the routes after the camel context is properly started and attempt to restart routes that have not been starter successfully.
  • ClusteredRouteController which leverages Cluster SPI to start routes only when the context is elected as leader.

11) More components

As usual there is a bunch of new components for example we have support for calling AWS lambda functions in the camel-aws component. There is also a new json validator component, and camel-master is used with the new Cluster API to do route leader election in a cluster. There is 13 new components and 3 new data formats. You can find more details in the Camel 2.20 release notes.

We will now start working on the next release 2.21 which is scheduled in start of 2018. We are trying to push for a bit quicker release cycle of these bigger Camel releases, so we can go from doing 2 to 3 releases per year. This allows people to quicker pickup new functionality and components etc.

Also we want to get a release out that officially support Java 9, Spring Boot 2 and all the usual great stuff we add to each release, and what the community contributes.


Apache Camel route coverage tooling on the way

Last weekend I found some time to hack on new tooling for doing Apache Camel route coverage reports. The intention is to provide APIs and functionality out of the box from Apache Camel that other tooling vendors can leverage in their tooling. For example to show route coverage in IDEA or Eclipse tooling, or to generate SonarType reports, etc.

I got as far to build a prototype that is capable of generating a report which you run via the camel-maven-plugin. Having a prototype built in the came-maven-plugin is a very good idea as its neutral and basically just plain Java. And make its possible for other vendors to look at how its implemented in the camel-maven-plugin and be inspired how to use this functionality in their tooling.

I wanted to work on the hardest bit first which is being able to parse your Java routes and correlate to which EIPs was covered or not. We do have parts of such a parse based on the endpoint validation tooling which already exists in the camel-maven-plugin, which I have previously blogged about. The parser still needs a little bit more work however I do think I got it pretty far over just one weekend of work. I have not begun adding support for XML yet, but this should be much easier to do than Java, and I anticipate no problems there.

I recorded a video demonstrating the tooling in action.

I forgot to show in the video recording, that you can change the Camel routes, such as inserting empty lines, adding methods to the class etc, and when you re-run the camel-maven-plugin it will re-parse the route and output the line numbers correctly.

Anyway enjoy the video, its about 12 minutes long, so go grab a cup of coffee or tea first.

The plan is to include this tooling in the next Apache Camel 2.21 release which is scheduled in January 2018.

The JIRA ticket about the tooling is CAMEL-8657

Feedback is much welcome, and as always we love contributions at Apache Camel, so you are welcome to help out. The current work is upstream on this github branch. The code will be merged to master branch later when Apache Camel 2.20.0 is officially released.

When I get some more time in the future I would like to add support for route coverage in the Apache Camel IDEA plugin. Eclipse users may want to look at the JBoss Fuse tooling which has support for Apache Camel, which could potentially also add support for route coverage as well.


Camel in Action 2nd edition goes into production phase

We are moving forward with the Camel in Action 2nd edition book. Manning just informed us that they have moved the book into production phase.

All chapters and appendixes except for Ch21 have now completed the copyedit and review cycle and have moved into the proofreading and indexing stages of production.

The proofing is done by a professional, whom has not seen the material before (new fresh set of eyes). Once the proofreading and indexing stages are complete, the final Word manuscript chapters will be sent on to the typesetter for layout, along with prepped graphics from the illustrator.

When we react this stage Jonathan and myself have chances to review PDFs of the formatted book pages, to ensure all the code examples are formatted correctly and so on.

At present time Jonathan and I are now working on the front matter, and writing our preface and what else. We are also reaching out to the foreword writers to ensure they hand in their material (Gregor Hohpe and James Strachan are yet to return their forewords).

And when Henryk Konsek has provided his updates, then chapter 21 can move along and also be part of the book.

So it starts to look really good and we can start to see the finish line. Its been a long run this time which has taken us almost double the time to complete the 2nd edition than what we did for the 1st edition. But that is possible expected, we are getting older, and the book is also almost twice the size.

To celebrate this, Manning has the book as deal of the day. This means you get 50% discount if you order the book on September 24th using the couple code: dotd092417au

The book can be ordered from Manning website at: https://www.manning.com/books/camel-in-action-second-edition