Apache Camel 3.1 - More camel-core optimizations coming

Hope all is good and you had a safe entry into 2020. 

The Camel team is already busy working on the next Camel 3.1 version. One of the goals is to continue optimize camel-core, and this time we have had some time to look into finding some hot spots in the routing engine.

One of the aspects we have looked at is also the object allocations that occurs per message that Camel routes. The JVM itself is great at allocation objects and garbage collecting them when they are no longer in use. However there are room for improvements if you can identify a number of objects that is unnecessary per EIP in the route.

So today I found several of these by just running a basic Camel route that is


Which basically routes 1000 messages per second. And prints each message to the log.

One of the bigger culprits in object allocations turned out to be human logging for the reactive executor which logs at TRACE level. So by avoiding this we can reduce a great deal of allocations, and string building for logging messages.

Other aspects we have optimised is the to EIP (the most used EIP) which is now smarter in its startup to avoid creating caches that was not necessary. And this goes together with areas where we now lazy creates some features in Camel that were very rarely in use that would otherwise also setup and create some caches. 

We also identified as part of the Camel 3 work, then the LRUCache was not pre warmed up as early as before, which meant Camel would startup a bit slower than it otherwise are capable of. So by moving this warmup to an earlier phase then Camel can startup faster by doing concurrent work on startup until the LRUCache is warmed up (its caffeine cache that needs this).

The log component has also been optimised to reduce its object allocations which building the logging message.

So all together a great day and if we compare startup up a Camel 3.0.0 vs 3.1.0-SNAPSHOT with the Camel route as shown above, then we have an awesome reducing in object allocations per second (thanks to YourKit for profiler).

The profile says that in Camel 3.0.0 then Camel would roughly generate about 22.000 objects per second (routing 1000 messages). And that has been reduced to about 6.000 objects per second in Camel 3.1. That is fantastic, and is almost a 4x reduction.


Apache Camel 3 - Whats New Top 10

Apache Camel 3 was released last thursday November 28th 2019, which also happens to be the day of the US Thanksgiving. This was not intentionally but we can say its a big thanks from us to the community with a brand new major version of Camel - this does not come by often. In fact it's 10 years since Camel 2 hit the streets. So this 3rd generation is long overdue.

This blog post highlights the noteworthy new features and improvements in Camel v3.

Apache Camel, is now a family of projects (3 at this time of writing):

  • Camel 3: Integration Framework Swiss knife of integration
  • Camel K: Lightweight Serverless Integration Platform Camel on Kubernetes & Knative
  • Camel Quarkus: Camel extensions for Quarkus Optimised JVM & Native compiled Java (GraalVM)

The Camel code-base is very large, and we have setup sub-projects for new innovative projects using Camel. The first sub-project was to run Camel as cloud-native on Kubernetes in a serverless manner which became Camel K. Then Camel Quarkus came to make Java and Camel with very fast startup and very small memory footprint primary for container based deployments.

A major goal for Camel 3 was to finally revamp the old aging website to use modern technologies and be able to auto-generate content from the source code. This has taken years to get to this point as we have built tools over the last many Camel 2.x releases that could take us closer. At end of 2019 then the Camel community and others stepped up and provided the new art-work, logo, and look and feel for the new website - thank you very much!.

For Camel 3.x we will continue to improve the website and the documentation. This is much easier for us to do, and also for people to contribute changes as its just a regular github PR to provide updates. We love contributions.

Zoran had some fun with the new look and feel and he added a little gem; if you stare at the front page, then you should see a little animation of the curved bezel ;)

3) JAVA 11
Camel 3 is the first official release that supports Java 11. Java 8 will still be supported for the first number of 3.x releases, but is expected to be dropped later in 2020. However we wanted to provide Java 8 support to help migrate Camel 2.x users whom may be restricted to Java 8 for some time to come.

The camel-core has been modularized from 1 JAR to 33 JARs. The core functionality has been splitup into:


For Camel end users then only a few JARs is relevant.

camel-api contains the public API for Camel (eg interfaces such as CamelContext, Endpoint, Exchange, Message, and so on).

camel-support contains the base classes and RouteBuilder which you would use to build Camel routes and applications. This JAR is also contains necessary base classes for building custom Camel components, and other kinds of plugins.

The components that resided in camel-core has also be externalized into individual components:


Camel end users can then pick and choose exactly only what they need, or keep using everything.

Therefore we have camel-core and camel-core-engine as two starting dependencies. You can use camel-core which gives you all the JARs which is similar to Camel 2.x. When you use camel-core-engine you get the minimum set of JARs that makes a functional Camel.

camel-core contains 33 JARs and camel-core-engine contains 12 JARs.

We have reduced the size of core Camel and the number of classes loaded. For example in Camel 2 about 5200 classes was loaded, which has been reduced to about 4300 loaded classes in Camel 3.

We have also done many smaller optimizations in the core, to reduce the number of allocated Java objects, and speeup initialization and other means. We have used JVM profiling tools to assist and find the bottlenecks.

Another area of improvement is to reduce Java reflections. In Camel 2 then all the configuration of Camel components, endpoints, and routes are reflection based. In Camel 3 we have source code generated Java code for configuration that allows us to use direct Java calls instead of reflections.

Another similar area is Camel’s type converters which in Camel 2 are Java reflection based (you could build custom type converts that were not reflection based). In Camel 3 we also generate Java source code which means that type converting is direct Java calls at runtime.

We have also moved initialization logic to earlier phases when it was possible. For example there is a new build phase which allows Camel to do special initialization during building your project (this requires Camel Quarkus).

All this optimization improves the startup performance of Camel and reduces the memory overhead. With Camel Quarkus you can natively compile your Camel application and make it startup in 30 milli seconds and consume only 10mb of memory (RSS) with a full blown HTTP REST server and health-checks and metrics.

There are still a few items on the agenda that we want to work on in Camel 3.x to further optimize Camel core.

Camel end users whom has configured endpoints using URI strings, would all have experienced the problem when you make a configuration mistake in the endpoint, which then makes Camel fail on startup.

In Camel 3, we have a new type-safe DSL for endpoints which you can use in Java routes. You can continue to use the classic URI strings, but if you want to try the endpoint DSL, then you need to add camel-endpointdsl to your classpath. Then you should extend EndpointRouteBuilder instead of RouteBuilder to access the endpoint DSL.

Here is a basic example without and with the endpoint DSL:



You can also find a little example in the source code.

The routing engine in Camel has internally been reactive’fied and all EIPs has been retrofitted to work in a reactive manner. However this is internal only, and the Camel API for both end users and component developers are based on existing callback behavior.

We will later introduce and work on a client-side facing reactive API after we have jumped to Java 11 as minimum version (then we can support Java 9 flowable API).

Camel already have integration with reactive frameworks such as Vert.X, RxJava and Reactor Core in the dedicated Camel components.

We have introduced camel-main as a standalone JAR that makes it easier to run just Camel. There are a couple of examples with the source code that demonstrates how to do that.

We also use camel-main to have common code to configure and bootstrap Camel for standalone, Spring Boot, Camel K, and Camel Quarkus. This allows us to share the same code, and configuration options.

Camel 3 now integrates better with Eclipse Microprofile and we have Camel components for Microprofile configuration, metrics, health checks, and fault tolerance (on the way).

More components to come in upcoming Camel releases. These microprofile components are also used by Camel Quarkus.

Camel 3 now supports JUnit 5 for unit tests, with the test components that have -junit5 as suffix.

The Camel Registry is now also writeable, so you can add beans to the registry at runtime, or from unit tests etc.

You can also configure endpoints (producer) to lazy start. By default Camel works in a fail-fast mode, which means that Camel components that fails to connect to external systems during startup may cause the route to fail on startup. For Camel 3 you can now configure these endpoints to lazy start, which means the route will startup and they will first fail when a message is routed to the endpoint.

Camel also allows to configure your routes to be supervised during startup, which allows Camel to more intelligently start routes in a more safe manner, by restarting routes that failed.

We have of course cleaned up the code base, such as removing all deprecated APIs and components. We have also adjusted some APIs to make them easier to use from end users, and more Java 8 lambda friendly.

Internally we have also adjusted the route model, to make it easier to extend into new DSLs; and there is a YAML DSL on the way which was initiated in Camel K.

In terms of backwards compatibility then Camel 3 is mostly compatibility for regular Camel applications. However if you are using some of the more advanced features and other plugins in Camel then migration is needed. Also custom components must be migrated and recompiled. There are other adjustments such as Spring Boot users must use org.apache.camel.springboot as groupId instead of org.apache.camel etc. All details can be seen in the migration guide.

Good luck with your migration if you decide to continue your Camel journey. And for new users to Camel then good luck getting onboard.

There are 30 net new components in Camel 3, such as more stuff for Amazon AWS, and with GraphQL, and also worthwhile to mention is integration with Debezium, which is a change data capture project to grab change events from databases. 


Upcoming Tech Event in Copenhagen - What does the system integration toolbox anno 2019 looks like

On October 24th 2019, I am presenting at a full day tech event in Copenhagen.

My session is the first of the day, with a total of 90 minutes covering all the latest about Apache Camel 3, Camel K, and Camel Quarkus. I will have up to date slides, and demos ready for the talk.

The event also hosts other speakers such as Jeppe Cramon whom will talk about EDA and microservices. And I have been told that we should look forward to Syed Shaaf's session where he has a big tech demo with many moving parts.

In the afternoon you can choose to listen to breakout sessions or attend a full 2,5 hours hands-on workshop. The breakout sessions will among others cover use-cases with Apache Camel and Apache Kafka from the real world.

You can find more details about the event, the location, and how to register here.

Hope to see you there.

Cloud-native integration with Apache Camel on Kubernetes

Cloud-native applications of the future will consist of hybrid workloads: stateful applications, batch jobs, microservices, and functions, wrapped as Linux containers and deployed via Kubernetes on any cloud.

In this session, we will explore key challenges with function interactions and coordination, addressing these problems using Enterprise Integration Patterns (EIP) and modern approaches with the latest innovations from the Apache Camel community:

  • Apache Camel 3
  • Camel K
  • Camel Quarkus

Apache Camel is the Swiss army knife of integration, and the most powerful integration framework. In this session you will hear about the latest features in the brand new 3rd generation.

Camel K, is a lightweight integration platform that enables Enterprise Integration Patterns to be used natively on any Kubernetes cluster. When used in combination with Knative, a framework that adds serverless building blocks to Kubernetes, and the subatomic execution environment of Quarkus, Camel K can mix serverless features such as auto-scaling, scaling to zero, and event-based communication with the outstanding integration capabilities of Apache Camel.

We will show how Camel K works. We'll also use examples to demonstrate how Camel K makes it easier to connect to cloud services or enterprise applications using some of the 300 components that Camel provides.


Upcoming Webinar - Integration Patterns in a Serverless World

Next Thursday, October 10th, I am giving a 35 minutes webinar about Serverless Integration (yeah it covers Apache Camel, Camel K, Knative and Quarkus).

The webinar is part of a full day event (virtual event) where there are keynotes and a parallel tracks with 12 breakout sessions. 

My talk is focused purely on the upstream work we do at Apache Camel with the next generation Camel 3.

The abstract for my talk 

Cloud-native applications of the future will consist of hybrid workloads: stateful applications, batch jobs, microservices, and functions, wrapped as Linux containers and deployed via Kubernetes on any cloud.

In this session, we'll explore the key challenges with function interactions and coordination, addressing these problems using classic integration patterns and modern approaches with the latest innovation from the Apache Camel community: Camel K, a lightweight integration platform that enables enterprise integration patterns to be used natively on any Kubernetes cluster.

When used in combination with Knative, a framework that adds serverless building blocks to Kubernetes, and the subatomic execution environment of Quarkus, Camel K can mix serverless features such as auto-scaling, scaling to zero, and event-based communication with the outstanding integration capabilities of Apache Camel.

We will show how Camel K works. We'll also use examples to demonstrate how Camel K makes it easier to connect cloud services or enterprise applications using some of the 250+ components that Camel provides.


The event is free to attend, and you can signup using this link.

Date: October 10th 2019
Time: 5 pm CEST (event starts with keynotes)


Apache Camel 3 is only 2 months away

The Camel team is busy working on the last set of work for Apache Camel 3. Today the 2nd release candidate was built and published on a staging repository for early adapters to give it a try.

As I am busy myself then I just wanted to write a short blog post to keep the community posted that Apache Camel 3 is on the way, and that we expect it to be released by end of this year (sometime in November or December).

The latest deadline is to release Camel 3.0 before December 19th 2019 as it was exactly on that day 1 year ago, we switched the master branch to become the work branch for Apache Camel 3. That means the total development time for Camel 3 would be 1 year.

Here is a illustration that highlights the timeline of Camel 3:

That is not all Apache Camel, is now a family of 3 projects (at this moment). So working on Camel 3 is not all we do. Camel K and Camel Quarkus are very promising for cloud-native integration (microservices and serverless).

These projects have their own lifecycle. Will will post more details about these projects, and what's new in Camel 3 in the following months leading up to the final release of Camel 3.  So stay tuned.

PS: If you are migrating Camel 2.x applications to Camel 3, then read the migration guide.


New website for Apache Camel

Yesterday we published the new website for Apache Camel.

It's really fantastic to finally see a new website. And now with a modern tech for the website we can much easier update and keep it more modern for the future, and also it should help encourage users to contribute to its content to help build a better user guide.

Huge thanks to all involved.


Apache Camel 3 - camel-core vs camel-core-engine (smaller core)

The Camel team is very busy currently working on Apache Camel 3. A lot of work has already been implemented and we have released 3 milestone releases so far. The next milestone release number 4 has some great new innovative features which I will blog about in the following months.

The topic of this blog is the work we have been doing on splitting up camel-core into smaller modules which you can now easily pick exactly only what you need.

If we take a look at the dependency tree of the camel-core JAR you can see that it's been split up into many modules as shown below:

[INFO] +- org.apache.camel:camel-core:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-api:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-base:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-jaxp:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-management-api:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-support:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-util:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-util-json:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-bean:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-browse:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-caffeine-lrucache:jar:3.0.0-SNAPSHOT:compile
[INFO] |  |  \- com.github.ben-manes.caffeine:caffeine:jar:2.7.0:compile
[INFO] |  |     +- org.checkerframework:checker-qual:jar:2.6.0:compile
[INFO] |  |     \- com.google.errorprone:error_prone_annotations:jar:2.3.3:compile
[INFO] |  +- org.apache.camel:camel-controlbus:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-dataformat:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-dataset:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-direct:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-directvm:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-file:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-language:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-log:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-mock:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-properties:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-ref:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-rest:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-saga:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-scheduler:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-seda:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-stub:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-timer:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-validator:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-vm:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-xpath:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-xslt:jar:3.0.0-SNAPSHOT:compile
[INFO] |  \- org.slf4j:slf4j-api:jar:1.7.25:compile

In Camel 2.x all those modules were included in the same camel-core JAR.

To make the migration from Camel 2.x to 3.0 and also the ease of use, we will keep camel-core as the same set of JAR dependencies as in Camel 2.x. So the output above is from just declaring a dependency on camel-core, in your Maven pom.xml file or gradle build file.

However what is coming in Camel 3 milestone 4, is to make it easy as well to just pick what you need. For example the following shows a the dependency tree where we are only using as little as possible (using the new camel-core-engine JAR as dependency):

+- org.apache.camel:camel-core-engine:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-api:jar:3.0.0-SNAPSHOT:compile
[INFO] |  |  \- org.apache.camel:camel-util:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-management-api:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-support:jar:3.0.0-SNAPSHOT:compile
[INFO] |  \- org.apache.camel:camel-core:jar:3.0.0-SNAPSHOT:compile
[INFO] |     +- org.apache.camel:camel-base:jar:3.0.0-SNAPSHOT:compile
[INFO] |     \- org.apache.camel:camel-properties:jar:3.0.0-SNAPSHOT:compile

To illustrate this we have provided 2 set of examples

The former uses the camel-core as dependency which includes all the core components etc. The latter is the tiny dependency set, which only includes what is needed. The example only uses the bean and quartz2 component and therefore you would need to add them as dependency:

The dependency tree is as follows (mind that JAXB is excluded as test scope)

[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ camel-example-main-tiny ---
[INFO] org.apache.camel.example:camel-example-main-tiny:jar:3.0.0-SNAPSHOT
[INFO] +- org.apache.camel:camel-core-engine:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-api:jar:3.0.0-SNAPSHOT:compile
[INFO] |  |  \- org.apache.camel:camel-util:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-management-api:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.apache.camel:camel-support:jar:3.0.0-SNAPSHOT:compile
[INFO] |  \- org.apache.camel:camel-core:jar:3.0.0-SNAPSHOT:compile
[INFO] |     +- org.apache.camel:camel-base:jar:3.0.0-SNAPSHOT:compile
[INFO] |     \- org.apache.camel:camel-properties:jar:3.0.0-SNAPSHOT:compile
[INFO] +- org.apache.camel:camel-main:jar:3.0.0-SNAPSHOT:compile
[INFO] +- org.apache.camel:camel-bean:jar:3.0.0-SNAPSHOT:compile
[INFO] +- org.apache.camel:camel-quartz2:jar:3.0.0-SNAPSHOT:compile
[INFO] |  +- org.quartz-scheduler:quartz:jar:2.3.1:compile
[INFO] |  |  +- com.mchange:mchange-commons-java:jar:0.2.15:compile
[INFO] |  |  \- com.zaxxer:HikariCP-java7:jar:2.4.13:compile
[INFO] |  \- com.mchange:c3p0:jar:
[INFO] +- com.sun.xml.bind:jaxb-core:jar:2.3.0:test
[INFO] +- com.sun.xml.bind:jaxb-impl:jar:2.3.0:test
[INFO] +- org.apache.logging.log4j:log4j-api:jar:2.11.2:runtime
[INFO] +- ch.qos.logback:logback-core:jar:1.2.3:compile
[INFO] \- ch.qos.logback:logback-classic:jar:1.2.3:compile
[INFO]    \- org.slf4j:slf4j-api:jar:1.7.25:compile

If we compare the size of the JARs between the two examples (incl logger etc), then we have:

  • camel-example-main (camel-core): 48 JAR files with total of 9.3 MB
  • camel-example-main-tiny (camel-core-engine): 21 JAR files with total of 7.4 MB

If we run the two examples then you can also see the number of classes loaded into the JVM is a bit smaller when using camel-core-engine:
  • camel-example-main (camel-core): 3666 classes loaded
  • camel-example-main-tiny (camel-core-engine): 3430 classes loaded
Okay the reduction here is not as significant, but it is expected as Camel will lazy load components in use, and the example only uses the bean and quartz component. However the classpath scanning would be a bit faster as there are roughly 25% less disk-size and 50% less JAR files.

There are many other great things coming in Apache Camel 3 which I will start writing blogs about during this summer leading up to the release (expected in September/October 2019).