2015-07-28

Apache Camel 2.16 - Show me all my incoming and outgoing endpoints in real time

In the upcoming Apache Camel 2.16 release we expose even more information about your running Camel applications.

We recently added specialized mbeans for all the different EIPs to expose EIP specific information. Before we had a few mbeans for some selected EIPs. But in Camel 2.16 we have for all of them. For example the content based router now keep track of which of its predicates that matches how many messages at runtime. A screenshot below.

The Content Based Router can now show information about each predicate and how many times it has matched during routing


The throttler EIP will expose how many messages are currently held back, and how long time before they are released. The circuit break exposes the state of the breaker whether its open, half open or closed. And the failover load balancer keeps taps of how many times each different exception has triggered a failover, and so on.

Also each EIP is provides details about how its configured, and you can also dump that information as XML (as you may know you can do for the entire route), but this is for the EIP alone.

Another new feature is that we also keep track of the utilization of all the endpoints. There is a global mbean that provides a list of all top 1000 recently used endpoints (the size is configurable), and how many times those endpoints have been used. What is cool is that we also track whether the endpoint is an incoming or outgoing endpoint.

To demonstrate this in action, I recoded a little shot video showing this, together with the new endpoint-stats command we have in the Camel commands.

Apache Camel 2.16 video to show all incoming and outgoing endpoints in real time, as Camel applications get deployed to a running Karaf container

We also introduced an Extended JMX statistics level, which you can turn on. When running in this mode, each EIP will also track their personal usage of each endpoints. So what the video does is on a global level. But with the extended mode, you can drill down on each EIP and see how many different endpoints they use, and which ones are the most often used. For example if using the Recipient List or Dynamic Router EIP to route to many different endpoints.

We all this new information it allows Camel tooling to go even further. We are working on adding more coolness to hawtio, so it can show a wobble diagram of all the inter relationship with your Camel routes within a Camel context, but also at a higher level, the communication between different Camel applications.

And of course as well in table like view, as the endpoint-stats command does in the video.


2015-07-14

Camel in Action II - We will be back with an even more awesome book

So its time again - we are back and announcing Camel in Action 2nd edition.

Camel in Action 2nd edition - Nah the Camel is still lazy and sitting down ;)

The first book has done well and its time we get back into the saddle and ride hard to deliver another excellent book.

Jonathan and I are thrilled to be back after 5 years and writing about Apache Camel again. Readers of the 1st edition will get the same level of dedication and hard work we put into the first book. We have taken all the best chapters from the first edition and updated them with all the latest functionality that Apache Camel offers together with all the excellent feedback we have received over the years about the book. We have also not been afraid of dropping content from those chapters which are less attractive today to add in more content.

We are adding 6 new chapters, and are expecting the book to be 1/3 longer - yeah more work for us, but we want to capture all the details of what is possible with Camel today. As for new content you will find chapters that extensively cover how to do microservice style development with Camel. How you can quickly build and run Camel standalone in fat JAR style, or with popular containers such as Spring Boot, DropWizard or with Jetty or CDI containers.

You will also find chapters that takes Camel even further how to scale and cluster Camel so you run many of them. And of course a lot has changed since the 1st book, such as the cloud, docker, and kubernetes, fabric8 and hawtio. And for the reactive movement we will talk about RX Java and vertx. So we will go even further this time and take you through all this exciting new shiny toys you can do with Camel.

Users of JBoss Fuse, or ServiceMix / Karaf will not be forgotten. We are biting the bullet and cover using Camel on OSGi much more in depth than in the 1st edition - back then OSGi was not really that well mature as it is today.

Oh btw we would love to do a chapter on iot, so let us know if you feel the same way. Then we will reach out to our friends from Camel Labs and have them help out as guest author to write about all that. They have done some great blogs to showcase what Camel already can do on embedded devices and micro controllers.

Manning have told us that if you purchase the 2nd edition, you will get the 1st edition for free (an electronic copy, not the hardcover).

And to celebrate this announcement the Camel in Action 2nd edition is MEAP of today, and you can get 50% off if you order today from Manning website using the coupon code: dotd071415au


2015-07-02

JBoss Fuse 6.2 released - guide how to quickly try it

At the Red Hat summit last week, it was announced that JBoss Fuse 6.2 was released.

I want to congratulate the fuse team for getting this release out. I know that they have been working very hard all this year to QA and ensure its passes the high barrier of quality expected from an enterprise product.

So with my Camel hat, its great to see a product that includes the very last Apache Camel 2.15.2 release and subscribers have enterprise support and maintenance of Camel for a very long time.

Jonathan Anstey, the great co-author of Camel in Action, wrote a blog post about the highlights of this release. You can find more details from his blog. The biggest highlight (I am biased) is of course the upgrade from Camel 2.12 to 2.15  ;)

The JBoss Fuse website has been updated with details about this new release, where you can also find the download and how to get started.

Now unfortunately as a personal taste of mine, they made the getting started guide longer and ... well to much JBoss tasty. Users do not need to start with a "complicated" install procedure of installing JBoss Eclipse tooling and then starting the Fuse server from an editor. 

Instead I suggest jump straight to install JBoss Fuse, which is a simple download and unzip, and run the script.

Okay so here is my alternative and simpler install procedure

1. Download JBoss Fuse 6.2.0.GA
2. Unzip the download, which you can do from the shell from linux/osx and using the file explorer for windows users etc.

    unzip jboss-fuse-full-6.2.0.redhat-133.zip

3. From the shell change directory to where you unzipped the download

    cd jboss-fuse-6.2.0.redhat-133/

4. Start JBoss Fuse

      bin/fuse

     and for windows

    bin/fuse.bat

5. Wait for Fuse to start and show the welcome screen

JBoss Fuse 6.2 - Welcome screen
6. Create the administrator user (only need once)

The first time the welcome screen is presented, it shows a message about an admin user has not been created.

No user found in etc/users.properties. Please use the 'esb:create-admin-user'
command to create one.

So to create such an user, you execute that command to type in the username and password of choice. For development and personal use, it may be a good idea to just use admin/admin as that is easy to remember.

JBossFuse:karaf@root> esb:create-admin-user
Please specify a user...
New user name: admin
Password for admin:
Verify password for admin:

7. Open the web console and login

Now that we have an administrator user which has permission for the web console, we can try that. So open a web browser with http://localhost:8181 and login with the user you just created.

The web console is based on the excellent hawtio, which offers a rich set of capabilities. On the screenshot we can access the server logs, or even the shell terminal, and whatnot.

Fuse web console showing the server logs

8. Lets build and install a Camel quickstart

Okay we love Camel so lets build an example and deploy it to JBoss Fuse. To do so open a shell and in the quickstarts directory a number of examples is shipped.

We will use camel-log as its a simple example that logs a message every 5th second. To build the example

     cd quickstarts/beginner/camel-log

And then run

    mvn clean install

Notice each quickstart has a readme file with full instructions how to build and install (install are 2 options as standalone or with using fuse fabric). Here we use standalone mode.

You will see something along the lines of this when running

davsclaus:/opt/jboss-fuse-6.2.0.redhat-133/quickstarts/beginner/camel-log/$ mvn clean install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building JBoss Fuse Quickstart: camel-log 6.2.0.redhat-133
[INFO] ------------------------------------------------------------------------
...
[INFO] --- maven-bundle-plugin:2.3.7:install (default-install) @ beginner-camel-log ---
[INFO] Local OBR update disabled (enable with -DobrRepository)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------


If the build is a success, then we are ready to install it. We can do this from either the web console, or from the JBoss Fuse shell.

From the fuse shell type this a bit long command to install and start the example we just build

JBossFuse:karaf@root> osgi:install -s mvn:org.jboss.quickstarts.fuse/beginner-camel-log/6.2.0.redhat-133
Bundle ID: 265
JBossFuse:karaf@root>

And in the web console a Camel button will appear in the top, which allows you to see all the running Camel applications inside Fuse. 

Fuse web console showing all the Camel applications running inside with real time metrics

And if you click on the Logs button then you will see the camel application is logging a message every 5th second.

You can also see the logs from the fuse shell, if you type

      log:display

2015-07-02 10:07:15,441 | INFO  | #0 - timer://foo | log-route                        | ?                                   ? | 198 - org.apache.camel.camel-core - 2.15.1.redhat-620133 | >>> Hello from Fabric based Camel route! : root
2015-07-02 10:07:20,441 | INFO  | #0 - timer://foo | log-route                        | ?                                   ? | 198 - org.apache.camel.camel-core - 2.15.1.redhat-620133 | >>> Hello from Fabric based Camel route! : root

You can also tail the log from the shell (use ctrl + c to stop tail)

         log:tail


9. Where to go more

Okay that was just an alternative getting started guide to quickly try out the new JBoss Fuse 6.2 release. 

With this release there is also the JBoss Eclipse Tooling you can try. It has the impressive visual Camel debugger, and also the graphical Camel route editor. The Fuse Eclipse team has improve the tooling a lot, and continues to do so. I have previously blogged a little bit about that. To install this tooling then follow the step 1 on the official JBoss Fuse getting started guide.



2015-06-28

JBCNConf - A great new conference in Barcelona

I just returned from Barcelona, after having a great time at the JBCNConf 2015 conference. Its the first big Java conference in Barcelona, and there was more than 220 people attending coming from 40 different countries.

JBCNConf 2015 - A great new conference has born in Barcelona

I was there with my fellow colleagues from the xPaaS team. We arrived on thursday and the speakers dinner was down at the beach so we had a nice walk towards the place, where we then enjoyed some beers and tapas.

We are walking at the beach towards the speakers dinner at the beach

Speakers dinner at the beach

The conference started on friday morning and James Strachan kicked off the event with the keynote where he presented Kubernetes for Java developers. As usually James does a brilliant job at presenting.

James Strachan giving the keynote about Kubernetes for Java developers

I then watched a few talks such as Bartek presenting vertx, and Jean Babptiste talked about Apache Karaf. vertx 3 is really looking impressive, and I would like to see how Apache Camel can become a plugin in the vertx family. We do have a camel-vertx component today but I feel its more for Camel needed to call into an external vertx event bus. The idea for the Camel vertx plugin would be to flip this around, so vertx developers can easily use Camel and it would feel all natural in the vertx family.

Apache Karaf 4 is on the way, and its great to see some of the ideas from fabric8 1.x has found its way into the core of Karaf. And hopefully they make Karaf easier and better as a immutable container that fits modern times with docker/linux containers.

Then it was lunch time which is tapas in the square outside which is really nice.

Lunch time outside in the square with nice tapas


The xPaaS team chilling out in Barcelona, where Rob is making final touches on his slides before his talk
After lunch it was Rob Davies to present ActiveMQ in the cloud and how the fabric8 MQ module makes deploying and scaling messaging in the cloud much easier. Rob demonstrated an autoscaler functionality where MQ detects that if the number of brokers are under more pressure to handle the load, then it can automatic scale up and down new brokers, and re-shard the queues across the brokers.

Rob Davies talking about scaling ActiveMQ with fabric8

After Rob it was myself to present. And of course it is about Apache Camel, and it was a similar talk as I did 2 weeks ago in London. However this time I had time to polish and improve a few areas. For example we have managed to get the centralized logging to work, so I could show the logs from the pods in real time in the nice kibana console. There room was almost full and about 10% of the audience have heard about or used Camel. So its great to be able to present Camel to new users.

Claus presenting Camel. Yes Camel is awesome as micoservice in docker containers ;)

At the end there were a few questions from the audience, and one from James himself. He hasn't seen the Camel forge addons working in IDEA. So he was curious to see more what it could do. So we spend 5 minutes extra demonstrating how the tool easily can edit or add endpoints, where you get a UI wizard that has a form to fill out the options.

The slides for my talk has been posted on slideshare.

James Rawling was also with us, but this time he 'got lucky' and didn't have to do a talk. A bit of shame as his knowledge and experience with CI/CD is really interesting. Every customer and organization we talk to want such a solution. And with fabric8 and kubernetes we have built that using the most popular open source software, such as jenkins, gogs, nexus, sonarcube, taiga, gerrit and so on.

Then it was late in the afternoon and we went for a 'business meeting' in a cafeteria nearby. It is surely lovely here in Spain that all those cafeterias often have beer on draft. After the 'meeting' we want back to the conference to join the networking and party event on the square in the campus. It was great fun to speak to more people and hear about what they do. And that they are eager to learn more about the new cool stuff such as kubernetes, docker and fabric8.

Later in the evening we went out for dinner and then grabbed a jin and tonic at the local tavern that we had discovered yesterday which was located nearby our hotel. Yes we never leave a good opportunity wasted - some of the cool stuff that we have implemented over the many years has initially started as an idea on our many road trips.

The next day I went back to the conference for the 2nd day. I attended a few talks in the morning.

Manning has sponsored 5 Camel in Action books. So before I had to leave I signed those books, that later will be used in a raffle as a give away to the lucky winners.

Book signing with my impressions of the conference

Because the Camp Nou football stadium was only 1.5km from the event, I walked down there, and did the stadium tour. As I had to leave to the airport in about 1 hour, I had to rush a bit the tour. There were a lot of people and kids where they all the time had to stop and take photos and selfies.

Claus at the Camp Nou football stadium, home of FC Barcelona

Here at the end I want to congratulate the BarcelonaJUG for an amazing well organized conference. I really enjoyed it. It was laid back and great atmosphere. Hope to be there again in 2016.

On twitter I noticed they posted a photo of the top-5 talks. Its great to hear that our talks was well received. Also I was placed one spot higher than my boss ;)





2015-06-24

Apache Camel in space ... erh in Docker and Kubernetes and fancy fabric8 web console

I have just recorded a 5 minute video that demonstrates running an out of stock example from Apache Camel release, the camel-example-servlet packaged as a docker container and running on a kubernetes platform, such as openshift 3.

camel-servlet-example scaled up to 3 running containers (pods) which is easy with kubernetes and fabric8
In this video I have already deployed the example and then demonstrates how we can use the fabric8 web console to manage our application. And also connect to the running container and see inside, such as the Camel routes visually as shown above.

Then I run a simple bash script from my laptop that sends a HTTP GET to the Camel example and prints the response. The script runs in a endless loop and demonstrates how kubernetes can easily scale up and down multiple Camel containers and load balance across the running containers. And at the end its even self healing when I force killing docker containers.

So I suggest to grab a fresh cup of tea or coffee and sit back and play the 5 minutes video.

The video is hosted on vimeo and can be seen from this link.

2015-06-13

Returning from Microservices Event in London

I just returned from our recent event - Microservices Architecture Developer Day - which happened in London.

It was a great day and I enjoyed hearing the stories about Apache Camel making a difference and being used in all sorts of industries today. It was great to see that almost all registered turned up so that people would have to stand up in the back of the room. Though we had the RH employees to stand up so you all had a chair.

Packed room at the event.
The event started with James Strachan giving the keynote about docker, kubernetes and microservices is a great combination and has such a huge potential for the way we develop and think about services, deployments and applications.

James Strachan talking giving the keynote

I took over and talked about Apache Camel, and how did live demo how in practice you can developer Apache Camel micro services from scratch, and dockerize your projects and run them on a kubernetes platform with openshift 3 (you can use vanilla kubernetes platform). As preparation for this talk I recorded two videos as backup in case there was trouble doing the live coding at the event.

Claus Ibsen talking about Apache Camel and microservices

A talk that gathered great interest was Marc Savy and Kurt Stam's talk about api management. First Marc gave an overview of the apiman project, and Kurt did the live demonstration showing this in action.

Kurt and Marc presenting the apiman project. Notice Kurt is wearing the apiman tie from the project logo ;)

James Rawlings did a fantastic talk about a sometimes underestimated area, but that is becoming more and more important for business. How to get their software quicker into production. That topic is sometimes captured as DevOps and the continues integration / deployment movement. And couple that with social aspects and event notifications delivered easily to the people. And then the culture change and people problems that come along when organizations has the change to embrace this. I really enjoyed hearing about James war stories and demonstrating the CI/CD pipeline that fabric8 offers a single one click installer.

James Rawlings talking about DevOps, CI/CD, microservices and culture and people problems.
After lunch there was break-out sessions where we hosted a BOF talking about integration, microservices, fabric8, Apache Camel, JBoss Fuse, et all. It was a great interest and we had great and honest discussions.

At the same time Jimmy Dyson took the stage and presented logging and metrics with fabric8. When you see this for the first time you are a bit amazed. That you can get centralized logging and metrics collected from all your pods. The web ui is using the very popular Kibana for logging and Grafana for metrics.

Jimmi Dyson talking passionally about logging and metrics
This was a talk I would have loved to attend. Jimmi is a very knowledge and passionate engineer.

After the BOF there was a few talks focusing on classic JBoss projects such as Wildfly and developing microservices. Marc Little presented the Wildfly Swarm project - which looks really promising. Just hope they also get around making it appealing for the non JEE developers, and make Apache Camel integrated out of the box. Arun Gupta then talked about microservices, patterns and developing that with JBoss. I didn't catch this talk as I was engaged in hallway conversations with Camel users and people keen interested in fabric8.

The last talk I watched was Roland's talk about Maven and Docker. Roland just joined our xPaaS team and its great to hear about what the maven-docker-plugin can do, and how it makes dockerizing Java projects much easier.

Roland Huss talking about Docker and Maven about the docker-maven-plugin

And we also had time to do a few selfies with fellow Red Hatters.





The last talk of the day was about mobile, but as it was beer-o-clock a bunch of us skipped and went for the bar ;)

Of course there is beer after the event.


For interested parties I suggest the following reading order:

As all events was recorded and slides is handed over to the organizers. Then I assume they will process this and later publish the material. However it may be behind a registration page. The above content is 100% FOSS ;)

The next day as my flight was at 5pm, we had a sight seeing day. Myself and Ioannis walked the streets of London. All the way from Paddington, through Hyde Park. Passing the Buckingham (the queen was not home to see us) and then as a 2nd choice we went to 10 Downing Street, but the guards didn't accept our RH credentials so we went pass Big Ben, and to the London Eye. But at that point we was tired of walking so we headed back to Embarkment on the foot bridge, to catch the tube to Paddington for the express to Heathrow.

Buckingham Palace

On a closing note I really enjoyed the day. It felt like the good old FuseSource events where all the talks was from engaged developers talking passionally about the upstream projects and communities. There was no sales pitch or mandatory slides about the RH or JBoss Middleware products. Thank you for that. As the event was such a success I have been told that they want to arrange this again, but sooner than having to wait a full year.





2015-06-03

Apache Camel doing IoT to help measure air quality in the streets of France

Today I read a very interesting blog post from Abdellatif Bouchama, about a system he developed in less than a day, that gathers air quality metrics from IoT devices from the cities of France, and then using MQTT protocol gathers the data using ActiveMQ message broker, and then process the data using Apache Camel, inserting the data into an Elasticsearch database.

IoT devises sending air quality data over MQTT to A-MQ broker for further processing by Apache Camel - Using JBoss Fuse product.
When data are in Elasticsearch there is lovely web consoles such as Kibana that can present the data in geo graphical displays.

Geo graphical display using Kibana to highligh the cities with air quality data

I encourage you to continue reading this blog from the original posting at bushhorn. The post is here.
The source code of this is posted on github as well.

PS: Yes the Camel route is only 3 lines of code (4 if you add the log statement) ;)

Well done Abdellatif.


PS: If you are interested in more about IoT and Apache Camel, then there is the new community project called camel-labs that provides a number of Camel components and tooling around this very topic.