2015-02-02

Your Apache Camel applications now includes out of the box documentation

A couple of months back I blogged about the upcoming 2.15 release includes functionality to obtain fine grained information about each property configured on your endpoints in the blog titled - Apache Camel please explain me what these endpoint options mean.

We have continued on this path, and today we have extended this from endpoints to cover EIPs, data formats, and languages as well. In a nutshell almost all what you use and configure in your Camel applications now has the capability of including documentation out of the box at runtime. In addition all of those are associated with label(s) which we use as a way of categorizing them. So you can list all components for database, cloud, http, rest, messaging, etc etc.

As the EIPs is provided in the camel-core those all their options is fully documented, including details about their default values, if the option is required, whether an option is deprecated, and of course type information etc. The EIP itself is although currently only documented with a brief summary.

As Camel includes about 175 components then it will take some time to include documentation about each one of them. But we will setup in the build process of Apache Camel a report that outputs which components has missing documentation.

Okay a picture is worth a thousand words, so let me include a bit of eye candy of this in action using hawtio - the web console.

First an example of the EIP such as the aggregate EIP which has many options.

Showing properties of the Aggregate EIP in our Camel application,
including documentation out of the box 
Notice in the top there is checkboxes to:

  • show/hide documentation (the documentation is always provided as a tooltip)
  • show/hide default values (not in current screenshot, as just added)
  • show/hide unused values

And another screenshot showing the properties of the endpoints:

Showing properties of an endpoint in our Camel application,
including documentation out of the box
This page is similar to the EIP page. It will also include the same checkboxes to filter.
Also notice those grey badges, they represent the label(s) this endpoint has been associated with. As we can see this is an endpoint from the core, and its scheduling.

All this information can be accessed at both runtime and design time. At runtime there is Java and JMX API from camel-core. And at design time the information is included in a new camel-catalog JAR that has json schema files for each EIP, component and so on. Likewise these json schema files can be retrieve at runtime using the Java and JMX API from camel-core as well.

So in other words this allows both end users and tooling to access this information.

For more information, read my previous blog, which I refer to in the top of this blog post.

What's Next

We have yet to implement a maven plugin as part of the build process of Apache Camel, that injects the EIP documentation in the XSD schema for the Spring and Blueprint XML DSL. When this is in place, then any IDE tooling should be able to leverage this, so while you edit those XML files, then the tooling can assist you and present the documentation etc.


2015-01-20

Webinar - Integrating microservices with Apache Camel

Tomorrow on wednesday 21st of january, our fellow Camel rider, Christian Posta is giving a webinar about microservices and Apache Camel.
Microservices architectures are all the rage these days, but we have to balance hype with reality. Microservices make it harder to manage your deployments and makes for complex inter-service communications patterns. How do you balance the tradeoffs and focus on getting the most out of your investment in highly scalable, decoupled systems?
Join this webinar to learn how to use patterns from SOA to build out intelligent routing systems with Apache Camel, and centralized management, service discovery, versioning, and tooling support from Red Hat JBoss Fuse for managing complex integrations using a microservice approach.
Date: Wednesday, January 21, 2015
Time: 16:00 UTC | 11:00 am (New York) | 5:00 pm (Paris) | 9:30 pm (Mumbai)
Duration: 60 minutes

You can find more details and register for the webinar using the following link.

2015-01-08

IDC Report on Business Value of using JBoss Fuse (with Apache Camel)

This is just a blog post that has more commercial nature, but you can't have one without the other. In fact this is what keeps Apache Camel alive and doing so well, due also to its commercial success.

This report may be of interest to people who are looking for hard evidence of the value add of using Apache Camel commercially, from products such as JBoss Fuse.
IDC interviewed 6 organizations that report achieving significant business value by using Red Hat® JBoss® Fuse, in particular, and making their application integration and development efforts more efficient and productive. These 6 organizations are achieving a 3 year average return on investment (ROI) of 488% and earning back their investments in JBoss Fuse in 8.2 months.
The report is downloadable as-is (no registration) from Red Hat website at: http://www.redhat.com/en/resources/value-red-hat-integration-products

Some of the highlights I see in relationship to Apache Camel are:


  • JBoss Fuse allows developers to program in Java, which becomes more of an extension to the application rather than a separate development experience. This resulted in making JBoss Fuse easier to adopt. As one customer explained, "One of the biggest advantages that we see with this product is that the integration tool is provided as a domain language. So it's like a natural extension of the Java language. You write integrations as if you're writing Java code. That means that all of the skills you need to debug in Java are the same skills required when you write integrations in this language. You don't have to go to another IDE or another toolset to understand how to write it. It's just Java code."
  • Another customer cited the team's desire to actually know and understand what the code is as a reason for adoption: "Before, we had a problem with the software, and actually being able to prove to the vendor that there was a problem before they would get around to fixing it is a lot harder if you can't actually see any of the code. And this is not a small issue … In order to lodge a request, we would often have to prove to them by writing a sample program and prove that they are the cause of the issue — in a way that it's reproducible in their environment."
  • A shipment and logistics company's application integration and development efforts benefit from the flexibility of JBoss Fuse's use of Apache Camel: "JBoss gives us the opportunity to help the developers in ways that wouldn't necessarily be possible with a graphical user interface tool."
  • Customers also said that they benefit from Red Hat's support and the ability to have access to and understand the code used for their applications. One customer praised Red Hat's support: "The customer service support from Red Hat has been exceptional. They gave us access directly to the developers who are writing the code, whereas it can be hard to actually get access to developers with other solutions."

  • Just a note to the last bullet. Our support program allowed the customer direct access to myself and other Camel core committers, so we were able to help them quickly, and also fix and improve Apache Camel based on their issues and findings.

    The report has more details, and hard numbers stating the "before" vs "after" and their gains (development time, performance, and many others) and cost benefits, such as pretty graphs as shown below:



    2014-12-31

    Some great Hawtio videos by Robin Huiser

    Recently Robin Huiser recorded three videos of demonstrating hawtio in action. I just wanted to help spread the word of his videos, as I enjoyed watching them.

    hawtio videos by Robin Huiser

    hawtio is a true agnostic web console that supports anything running in a Java JVM. Robin uses Talend ESB as his container for hosting Camel applications. You can of course run Camel anywhere as well, so you can also use Apache Karaf / ServiceMix / JBoss Fuse / Wildfly or any other platform of choice, and hawtio "just works".

    The three videos is about 10 minutes give or take, so grab yourself a cup of coffee and sit back and enjoy.

    Links to the videos on youtube.

    Have a great new years even and see you in 2015 for a great year of Camel and hawtio. There is a ton of new stuff coming in Camel 2.15 that makes hawtio and Camel even more awesome.

    2014-11-07

    Apache Camel please explain me what these endpoint options mean

    In the upcoming Apache Camel 2.15, we have made Camel smarter. It is now able to act as a teacher and explain to you how its configured and what those options mean.

    The first lesson Camel can do is to tell you how all the endpoints have been configured and what these option mean.

    Lessons we are working on next is to let Camel explain the options for the EIPs are.

    Okay a picture is worth a thousand words, so let me show a screenshot from Apache Karaf, where you can use the new endpoint-explain command to explain how the endpoints have been configured.


    The screenshot from Apache is from the SQL example which I have installed in Karaf. This example uses a number of endpoints, and among those a timer to trigger every 5 seconds. As you can see from above, the command list the endpoint uri: timer://foo?period=5s and then explain the option(s) below. As the uri only has 1 option, there is only one listed. We can see that the option is named period. Its java type is a long. The json schema type is integer. We can see the value is 5s, and below the description which explains what the value does.
    So why is there two types listed? The idea is that there is a type that is suitable for tooling etc, as it has a simpler category of types accordingly to the JSonSchema specification. The actual type in Java is listed as well.
    The timer endpoint has many more options, so we can use the --verbose option to list all the options, as shown below:


    The explain endpoint functionality is also available as JMX or as Java API on the CamelContext. For JMX each endpoint mbean has an explain operation that returns a tabular data with the data as above. This is illustrated in the screenshot below from jconsole:


    In addition there is a generic explainEndpointJson operation on the CamelContext MBean, this allows to explain any arbitrary uri that is provided. So you can explain endpoints that are not in use by Camel.

    So how does this works?

    During the built of the Apache Camel release, for each component we generate a HTML and JSon schema where each endpoint option is documented with their name, type, and description. And for enums we list the possible values.

    Here is an example of such a json schema for the camel-sql component:



    Now for this to work, the component must support the uri options, which requires to annotation the endpoint with the @UriEndpoint. Though the Camel team has not migrated all the 160+ components in the Camel release yet. But we plan to migrate the components over time.

    And certainly now where we have this new functionality, it encourages us to migrate all the components.

    So where do we get the documentation? Well its just java code, so all you have to do is to have getter/setter for an endpoint option. Add the @UriParam annotation, and for the setter you just add javadoc. Yes we grab the javadoc as the documentation. So its just documented in one place and its in the source code, as standard javadoc.
    I hope we in the future can auto generate the Camel website documentation for the components, so we do not have to maintain that separately in its wiki system. But that would take hard work to implement. But eventually we should get there, so every component is documented in the source code. For example we could have a readme.md for each component that has all the component documentation, and then the endpoint options is injected from the Camel built system into that readme.md file automatic. Having readme.md files also allow github users to browse the Camel component documentation nicely using github style ;o

    So what is next?

    The hawtio web console will integrate this as well, so users with Camel 2.15 onwards have that information in the web console out of the box.

    And then its onwards to include documentation about the EIP in the XML schemas for Spring/Blueprint users. And improve the javadoc for the EIPs, as that then becomes the single source of documentation as well. This then allows tooling such as Eclipse / IDEA / Netbeans and whatnot to show the documentation when people develop their Camel routes in the XML editor, as the documentation is provided in the XSD as xsd:documentation tags.

    We have captured some thoughts what else to do in the CAMEL-7999 ticket. If you have any ideas what else to improve or whatnot, then we love feedback from the community.


    2014-09-19

    66th Apache Camel release is out - its the 2.14 release

    Today Apache Camel 2.14.0 hit the streets. Its our 66th release (include all patch releases) since the project was created over 7 years ago.

    There is a bunch of great stuff in this release, so let me try to distill a list of 10 highlights (in no particular order)


    1) Java 8 support
    The 2.14.x branch marks the 1st release where we officially support Java 8. We made sure the code compiles and all tests passes on Java 8 also. We also support Java 7.

    Now you may ask about all the coolness from Java 8 about lambda's and whatnot. Yeah we are looking into that for the next release to provide examples and a first cut of a Java 8 based DSL.


    2) Spring 4 support
    The same goes here. We now support Spring 4.x also. There is one migration effort, as Spring 3.x and 4.x is not compatible when using spring testing. So we had to introduce a camel-test-spring3 module for Spring 3.x users, and camel-test-spring is for 4.x users. Also there is a known issue in camel-test-spring about Spring 4.1.x support which we will fix in the next patch release.


    3) Rest DSL
    This is likely one of the most exciting new features in this release. We now offer a REST styled DSL which can be used with Java or XML. The intention is to allow end users to define REST services using a REST style with verbs such as get, post, delete etc.

    Here is a small taste of it in action:

    rest("/customers/")
        .get("/{id}").to("direct:customerDetail")
        .get("/{id}/orders").to("direct:customerOrders")
        .post("/neworder").to("direct:customerNewOrder");

    The DSL can be used in both Java and XML. As this is the 1st release with the Rest DSL we will of course continue to improve on this, and certainly based on the feedback from the community.


    4) Netty 4.x support
    We now have support for both Netty 3.x and 4.x. As we take backwards compatibility seriously we decided to leave camel-netty as-is for Netty 3.x users, and port the code to a new camel-netty4 for Netty 4.x users. The same goes for the camel-netty-http and camel-netty-http4 modules.


    5) API component
    Dhiraj Bodke worked on a great new API component support in camel-core, that makes creating new Camel components for APIs much easier and faster. We use this to integrated with new components such as box, linkedin, google-drive, and salesforce and what else comes in the future. He wrote a great blog about this component and how to use it. So expect many more of these API components in upcoming releases.


    6) More JMX MBeans
    We now also enlist producers in JMX, and as well have a registry to capture runtime usage of endpoints, which allows you to "see" which endpoints are in use, and as well how endpoints may be shared across routes, e.g. sending to queue A in route 1 and consume from queue A in route 2. All based on dynamic usage such as dynamic patterns like the recipient list.

    The aim is to provide better understand how your Camel routes are being used at runtime. We will continue to improve tooling in this area such as hawtio, and the Camel Karaf commands, etc.


    7) Codahale metrics
    And continued from #6 we also have a new camel-metrics component to integrate with the excellent codahale metrics library. On top of that we also allow to use this to capture Camel routing statistics using codehale. This can then be used with existing codahale tooling and also with hawtio. I wrote a blog about this earlier.

    The screenshot below illustrates routing information captured by camel-metrics and displayed by hawtio in the new route metrics tab which supports codahale.

    camel-metrics used to capture routing statics which can be displayed in hawtio using the new route metrics tab.


    8) Circuit breaker EIP
    We introduce a new EIP using the circuit breaker pattern.


    9) Swagger integration
    We have a new camel-swagger module that in combination with the Rest DSL allows to expose an API documentation of your REST services. This can then be used with existing swagger tooling and web consoles. For example hawtio has a swagger console out of the box, such as shown below

    Camel example using the new Rest DSL which offers swagger API documentation, which can be viewed using a swagger console such as offered by hawtio


    10) And the usual stuff
    We added 15 new components, 1 new EIP, 1 new data format, and 1 new language. And as usual a bunch of bug fixes, improvements and hardening.  And we dropped support for Java 6.


    If you want to see some of the new stuff in action, then you can try the new examples. We have the camel-example-servlet-rest-tomcat. This is a plain WAR file which you can run in Apache Tomcat, Jetty, Wildfly, fabric8 or any other web container. Then you can see both the new Rest DSL, Swagger and codehale metrics in action, all in the same application. And for hawtness then try deployed hawtio as well and see what it can do as well.

    The same example is also available for Karaf users by the camel-example-servlet-rest-blueprint.



    You can find more details about this release in the 2.14 release notes. And if you are upgrading from a previous release, then make sure to read about the changes that may effect you.

    You can get this release from Maven Central and download from the Apache Camel website.

    2014-09-10

    More metrics in Apache Camel 2.14

    Apache Camel 2.14 is being released later this month. There is a slight holdup due some Apache infrastructure issue which is being worked on.

    This blog post is to talk about one of the new functions we have added to this release. Thanks to Lauri Kimmel who donated a camel-metrics component, we integrated with the excellent codehale metrics library.

    So I took this component one step further and integrated it with the Camel routes so we have additional metrics about the route performances using codehale metrics. This allows end users to seamless feed Camel routing information together with existing data they are gathering using codehale metrics.

    Also take note we have a lot of existing metrics from camel-core which of course is still around.

    What codehale brings to the table is that they have additional statistical data which we do not have in camel-core.

    To use the codehale metics all you need to do is:

    1. add camel-metrics component
    2. enable route metrics in XML or Java code

    To enable in XML you declare a as shown below:

      &;t;bean id="metricsRoutePolicyFactory"
            class="org.apache.camel.component.metrics.
                   routepolicy.MetricsRoutePolicyFactory"/>


    And doing so in Java code is easy as well by calling this method on your CamelContext

      context.addRoutePolicyFactory(new MetricsRoutePolicyFactory());

    Now performance metrics is only useable if you have a way of displaying them, and for that you can use hawtio. Notice you can use any kind of monitoring tooling which can integrate with JMX, as the metrics is available over JMX. The actual data is 100% codehale json format, where a piece of the data is shown in the figure below.

    Sample of the route metrics JSON data
    The next release of hawtio supports Camel 2.14 and automatic detects if you have enabled route metrics and if so, then shows a sub, where the information can be seen in real time in a graphical charts.
    hawtio have detected that we have route metrics enabled, and shows a sub tab where we can see the data in real time

    The screenshot above is from the new camel-example-servlet-rest-tomcat which we ship out of the box. This example demonstrates another new functionality in Camel 2.14 which is the Rest DSL (I will do a blog about that later). This example enables the route metrics out of the box, so what I did was to deploy this example together with hawtio (the hawtio-default WAR) in Apache Tomcat 8.

    With hawtio you can also build custom dashboards, so here at the end I have put together a dashboard with various screens from hawtio to have a custom view of a Camel application.

    hawtio dashboard with Camel route and metrics as well control panel to control the route(s), and the logs in the bottom.