2011-05-10

Faster unit testing Apache Camel source

Over the course of time, we have added many unit tests and components to Apache Camel. So as of today we have more than 7300 unit tests and 100+ artifacts to test.

As a consequence doing a full test on the Camel source code would therefore take time. On a standard developer laptop, it could take 3 hours or more to run all the tests.

Of course we have an arsenal of CI servers that run full tests overnight at Apache, and also FuseSource. So there are plenty of opportunities to check tests results. But nevertheless it would be better if testing was faster, so when we do development on the Camel source code, we can more quickly run unit tests to ensure the changes dont break stuff.

So in recent time I have looked into improving this. I managed to optimize the Camel core so it boot up faster. For example the core will now hard-load the necessary core type converters, instead of doing a classpath scanning. Likewise we have changed the Camel test kit, to by default lazy load additional type converters on-demand. That cut down the need for doing a classpath scan for those additional type converters. This change alone is a great improvement when you have a lot of unit tests, as it all adds up.

There were some other improvements here and there that all helped as well. For example cutting down on the needed dependencies using when testing components.

So all together we have made the testing roughly 20% faster. That is a good improvement. For example we cut down 34m on testing time on the Apache CI servers.

These changes will make it into the next Apache Camel 2.8 release.

5 comments:

Oleg Iavorskyi said...

Claus, this is excellent news! I was also looking into the ways to optimize tests based on Camel Test Kit but without much success, unfortunately.
Do you have any plans to enhance TK further to allow one Camel context per test class so it's not recreated before each test method? Currently setup and teardown in TK base classes are just creating/starting/stoping context which adds significant overhead to cases where similar test setup is required. For example, I'm testing simple route and want first test method to check "sunny day scenario" and another one to check failure handling. Setup of mocks is pretty much same with a bit of tweaking on the method level, however I'm getting Camel context created twice making tests 1-2 seconds slower.
I don't see an issue in reseting mocks manually in setup method if it will provide increased speed but currently it's not possible as far as I know. This mean that I tend to test more in one method to eliminate overhead.

Claus Ibsen said...

Oleg that is a good idea. Could you create a JIRA ticket for that?

Claus Ibsen said...

I created a ticket to track this
https://issues.apache.org/jira/browse/CAMEL-3960

Oleg Iavorskyi said...

Claus, thank you so much! I was just about to create ticket today after work but see you not only created it but also provided implementation already, fabulous!

Claus Ibsen said...

Yeah we want it to go faster :)

You are more than welcome to give it a try with the latest source code.

There is information how to here:
http://camel.apache.org/camel-test.html