Moved to Wordpress

The last post was about the last thing left over from my job at Pramari.
With my switch to Senacor I also decided to change my blog.
The new one is here:
http://codepitbull.wordpress.com/

WCI 2009 talk about CEP

I am giving a talk at the WCI 2009 titled "Taming the Tsunami - Tackling the exponential growth of sensor data using ESP/CEP". As one might have guessed it's about my (current) favorite topic: CEP and its applications. It's going to happen on Friday October 16th, 2009 and I am pretty excited as this is my first conference submission.
For those who can't attend or want to get a little information on CEP I put the paper up on my dropbox.

Faster developing in OSGi with '-console' and 'update'

Just a little hint for those who work with OSGi on a daily basis.
Starting and stopping an OSGi-framework takes time, even for small projects. Working on a bundle that requires a lot of playing around can become quite painful when you have to wait those precious 5 seconds or more until the system is restarted.
Thankfully the OSGi-spec already provides an update facility.
Each bundle can be told to update itself from the location it was loaded from.
That last part is the important part and made my life a lot easier.
Most, if not all, OSGi developers already use the console to get handy commands like 'ss' or 'services'. Another nice command is 'update <bundleid>'.
The cool thing about 'update' is that even bundles you included from your workspace can be used.
Just fire up the framework with the '-console' parameter and whenever you changed something in your workspace use update to get those changes into your running framework.

Embracing Groovy

Actually it's not that much about Groovy than it is about scripting.
After a yearlong battle I finally got my way and we added scripting capabilities to the Rifidi Emulator.
Scripting became an important issue as the scenarios we wanted to create grew more and more complicated.
Now that we also have to support RSSI (Received Signal Strength Indication) we need to be able to simulate tons of different reads with varying tag properties. Something that simply wasn't possible in TagStreamer.
So finally the management agreed that we had to include scripting support.
I already had quite clear picture of what I wanted to achieve and how the thing should look in the end.
So here is a quick outline how Groovy got hooked up to the emulator.

1. At first we had to get rid of the PlugIn-Registry from Eclipse. We were using extension points to register the different reader implementations but decided to be OSGi compliant a while back. We ripped out the registry and replaced it with OSGi-Services and Spring.
2. The nice guys at Groovy were kind enough to already ship an OSGi-enabled jar. So I just downloaded the current binary distribution and put groovy-all.jar into our target platform.
3. We took our RMI interface that we had added to allow TagStreamer to use Emulator instances remotely and refactored it a little.
   We basically made it more scriptable by exposing a couple of helper functions for creating tags and managing readers.
4. Integrating groovy with the new interface was the easiest part:
   Using the executor pattern we start a new Groovy Shell for each script. Each script gets a reference to the reader management service injected and can now create/destroy/start/stop... readers and tags.
5. Now to the part that I like the most:
   We have been using the Equinox-OSGi-console for quite some time now and we love how extensible this little bugger is. We added a couple of new shell commands to manage the scripts.

That's it. 5 days of work and we got full scripting into the emulator. It's not perfect but it will be improved while we use it.
Things we need to work on:

• Integrate with Eclipse (We want to make use of the great tooling Groovy has there)
• Integrate with Edge Server UI.
  
• Start distributing scenarios.

But for now I am quite happy with what we got :)

Exporting an OSGi application from Eclipse

Sometimes working with Eclipse feels like arcane science.
Creating a pure OSGi (Equinox) application is as easy as it can get.
Running and tuning are dead simple. Manipulating the runlevels works like a charm (needed to get load time weaving integrated).
But then try exporting it.
After a little painful research and some nice hints from the newgroup I got the following process:

1. Create a product from your launch config.
2. Open the product and go the configuration-tab. You will only see the bundles that were designated to be auto-started at a certain level. Add all other bundles and set them to auto-start at level 4. The funny thing here is that these are the default settings for every OSGi-Framework application, but in a product the default is to not auto start anything.
3. Now make sure that all fields on the overview-tab are blank (Application has always a preselection, at least on my install)
4.  If you haven't done so before add org.eclipse.simpleconfigurator to be started in level 1.
5. Start the Export Wizard.
   
6. Make sure that "Synchronize before exporting" is UNSELECTED, otherwise the export will fail.
7. Export.
8. If you don't have the delta in your target platform you will now have to copy the native fragments into the export.

There you go.

Living without friends sucks

And again I stumble over a scenario where it would be so absolutely helpful to have C++-like friends in Java.
Again I need to jump through hoops to get the desired behavior.
I am working on an integration of our SAL (Sensor Abstraction Layer) with Esper.
But the messages from SAL can go to other receivers like JMS, too.
The problem comes form the fact that the messages coming from the SAL have to be readonly to not mess Esper up.
As I like my code easy readable I prefer setters over big constructors. But in this scenario setters would be very problematic as they would allow the manipulation of data at any point in the future. Something that must not happen from the moment on the message went into Esper.
So public setters are a no go. Protected setters don't work as they are only accessible from the same package. So I end up having to write a big constructor, collect all the data from the different sources and then create the object.
Exceptions might be another way of solving the problem. But all the boilerplate code ...
Wouldn't it be nice to declare it protected and define a list of friends who can access it?
Then I would be able to create the object, pass it to the friends who are allowed to access the setters and then put the object into Esper without having to worry that somebody messes up my messages.
So sad.
Note:
I put together a little idea using CGLib and a set of different interfaces to built a proxy based solution. As soon as I get some time I will post it here.

K.I.S.S. and the 5 rules of laziness

This article took me a lot longer than I had expected.

I could try to bring up the usual excuses like "my apartment exploded", "my dog (that I don't have) ate my laptop" or "my boss locked me up in my office until my Trac issues were all closed".

Nope, the problem is that simplicity is the frickin' most complicated thing to explain.

I tried to keep it simple and summarize things that I feel are important aspects of simplicity. The resulting article is not even close to what I had intended and I think I have to live with not being satisfied with the result. But at least i lived up to my promise to deliver this.

How often did somebody use K.I.S.S. ("Keep It Simple, Stupid") to tell you how to improve your code?

After using K.I.S.S. on a lot of fellow coders over the years I thought it should be a quite simple task to explain.
Let's cut this short: I was wrong.
There is so much more to this issue than you think in the first moment. Explaining K.I.S.S. would easily fill a ton of books without really getting to the point.
After a lot of discarded posts I remembered a quote from a friend:
"The most important attribute of a good developer is laziness."
Good coding skills in combination with laziness leads to effective simplicity.

So here they are:

My 5 Rules Of Laziness.

1. Be lazy: Look, the world is full of developers, you are most certainly not the first one to run into that particular problem. Search for an open source project or some sample code that fixes your problem at hand (Never ever forget to give credit to the source of your solution!).
2. If 1. failed remember being lazy: Ok, so your google-fu is weak. Most likely you will have several possible ways to get the job done. Always pick the easier route.
3. If you are working on 2., be more lazy: This is a lesson learned from doing projects in an agile way:
   Solve your problems at hand (the ones defined for the current iteration), not the ones that MIGHT occur. Don't forget about the MIGHTs, put them as issues in your tracking system. Discuss them BRIEFLY with your project lead to see if you can replace the MIGHT with a WILL see if it is urgent enough to put in the current iteration. If not, leave it.
4. If you are impressed by the beauty and complexity of your code:
   - Print out a copy of it.
   - Put it on the wall next to the picture of Picard.
   - Start refactoring until you feel like your cat could understand the code.
5. Document it!

1 is quite self explanatory. If you can have somebody else do your work: Have him do it.

2 is a little more interesting. It deals with the arch enemy of of software developers:

Complexity

It's a mythical beast more vicious than Grendel and sneakier than Loki. It is secretly plotting to turn your beautiful source code into a pile of gibberish.

Every software development project has the tendency to grow bigger and more complex. The accomplices of complexity (customers, management, team colleagues) are lurking everywhere. Through feature requests and performance improvements they will make your code more complex. But you don't have to make it worse.

If you can pick and choose always go for the solution that is easier to implement.

If  you need to do sorting and you can choose between different algorithms: pick the simplest one. If that proves to be a problem swap it out later on (you should be using something like the the strategy pattern anyway).

Note (let's not talk about why I have to point this out):

Don't take shortcuts (singletons, declaring things static, breaking the separation of concerns principle....).

My 3rd rule is actually an extension of 2. A lot of times you will discover new problems while you are working on your items. If you find something that breaks the software RIGHT NOW:

• open an issue
• tell the project lead
• fix it.

If it MIGHT be a problem there is a 80% chance that it will never actually happen:

• report it
• discuss it
• If you can still put a MIGHT in the sentence describing the problem, leave it.

An example from my personal experience:

A developer was working on a fairly simple piece of model code. After seeing that the issue wasn't resolved after 4 days of work I asked the developer what was going on. He pointed out that the code for adding and removing items to the model wasn't threadsafe, so he started making it threadsafe. He had turned the whole thing into an intricated (an adjective that should make all your warning lights light up at the same time) piece of threadsafe beauty. At least that was what he thought.

The model class was used exclusively inside an Eclipse RCP application. Eclipse uses the reactor pattern, therefore everything is executed in the same thread.

He not only added tons of code that were simply not needed he also broke some things in the process.

My 4th rule is quite simple:

Nobody will be impressed by a complicated piece of code with the exception of the author. And he will lose the love for that code when he has to go back and understand it a few weeks after he wrote it.

Just remember that somebody will have to maintain it. And it will most likely be yourself.

Number 5 I just threw in because without documenting your code (and I mean in your code and not some fancy user docs) rules 1 - 4 are worthless.

New distinct operator in Esper 3.1.0

REJOICE: Esper 3.1.0 got released yesterday and I am currently moving our code to use it.
A couple of leaks got fixed and some behavior have changed. But that's not what made do this post.
After reading through the changelog I discovered this little gem:
 - A new "every-distinct" operator has been introduced to the pattern language. Very similar to the "every" operator, this operator suppresses duplicate results received from sub-expressions based on a set of expressions returning distinct-values.

Before the this new keyword got introduced I had to create a separate data window and use the following query to fill it with unique events:
insert into collectwin select * from reader as tagevent where not exists (select * from collectwin as collected where collected.key=tagevent.key)
Thanks to the new keyword I can now define a pattern for that job:
pattern[every-distinct(res.value) res = ParsedEvents]
As I am doing ALE I need the parsed events to be unique within a certain time frame. To reset the set of unique events do the following:
pattern[every-distinct(res.value) (res = ParsedEvents where timer:within(5 sec))]

This makes the whole thing a lot easier and better looking :)

Recent Activities

The last few months were quite busy and once again it was my blog that had to suffer. We rewrote our whole middleware, I got into Esper, we had a big show in Orlando and I had to travel back and forth between Germany and the US.
But I didn't stop writing.
I finally pulled myself together to try write articles for software magazins. To my personal amazement it looks like as if people actually like the things I write ;)
The German Java Magazin published a 2 part series of articles on RFID and Open Source RFID tools in issues 4.09 and 5.09.
I mentioned before that I am writing on an article for the German Eclipse Magazin. The article was published in December 2008.
Yes, I know you won't be able to buy them as there are already new issues of both magazins out. I hope to be able to post links to the articles as both magazins tend to republish them online.

Software development basics

Software development is my passion. I love creating code, brooding over algorithms, learning new languages or frameworks and just about everything that is related to it.

Having started to code on the C64 more than 20 years ago I can savely assume that I know a lot about the process and the required mindset for coding.

And it's exactly that mindset that I miss in a lot of coders that come out of university.

It's not there fault. Universities tend to just teach the tools, like a programming language or UML, skipping the essential basics.

I don't want to get started on how much I think it is wrong to start coding with a language like java.

What I want to get started on is the basic set of principles and rules that should be the mantras of every developer and most engineers.

So this article marks the beginning of a small series about what I think are essential rules to follow to become a good coder.

In the series I will cover the following things:

K.I.S.S.

Divide and Conquer

Premature optimization is the death of software

Separation of concerns

Buildr + Scala on Ubuntu

So I got started on my Scala adventure.
Scala is a functional language and I wanted to get started with it since forever.
After playing around with it for some time I had to accept the sad truth:
The Eclipse plugin is a piece of crap.
Tons of crashes, inconsistent behavior, faulty auto-completion ....
So I had to go back to my old friend vim. The only problem was that without eclipse I needed something to do the building. I could have written some shell scripts but as we are looking for a better build system for our projects at Pramari I thought this might be the right time to do so.
Ant was no option, I already knew it and I wanted something new :)
Maven was also not an option. This tool is an abomination and if there is any tool that is able to singlehandedly make agile software development impossible then it is this monster (more rants to come, for now: If you need to dedicate a person to learn the build system you know that there is trouble coming).
After reading a ton of blog-posts I decided to give buildr a shot.
The tool looked very promising, getting scripting and building together looked very intriguing to me.
But before building the wise creators of Ubuntu and Buildr teamed up to give you a thrilling installation experience.
I have been working with Ruby on Rails for some time (Buildr is built on top of Ruby and Rake) so I expected a smooth installation using gem.
Boy, I was wrong.
It took me two days to figure out how to get it up and running.
First of all: I thought gem was a system comparable to apt-get. it's not. I tried doing gem install buildr which failed with a ton of error reports. After manually installing the required versions of rake and several other tools I ended up with a version of buildr.
To my surprise it wasn't added to a directory that in the shell path. After finding the file and calling it manually it failed with a couple of error reports.
To cut this short:
I just figured out how to get it running.
First of all: Don't try to do it from the debian gem repositories, you need to specify the version from rubyforge.
So here is what you need to do:

gem install net-ssh -version 2.0.4
gem install net-ssh -v 2.0.4
gem install net-sftp -v 2.0.1
gem install highline -v 1.4.0
gem install rubyforge -v 1.0.0
gem install hoe -v 1.7.0
gem install rjb -v 1.1.6
gem install rspec -v 1.1.5
gem install xml-simple -v 1.0.11

gem install --source http://gems.tron.name/gems.rubyforge.org/  buildr

This will give you a working version of Buildr 1.3.3.
To get builder to run with scala and java you need to specify SCALA_HOME and JAVA_HOME.
Uh, before I forget it:
Your buildfile needs to contain require 'buildr/scala' to be able to build a scala project. For some reason this line is not added if automatically creating a buildfile inside a directory hierarschy containing scala files.

Apache Commons Logging. IT'S A TRAP!

Apache Commons Logging (ACL) is a nice idea when you first stumble upon it:
Develop your app using commons logging, use whatever logging backend you want to use and let the guy deploying decide what he wants to use for logging.
So far so good. I liked the idea. We are using commons logging all over the place.
Now reality hits.
Two weeks ago we started a new project. This time we wanted to do everything right and wanted to get all logging centralized. Piece of cake when everybody is using it ... which is simply not the case.
Having tons of open source libraries in there we finally had to realize that not all coders bought into the idea of ACL. Which is perfectly understandable as it adds another layer to your logging, another jar to the classpath and it takes away the logging interface you came to love over the years.
People use ACL, log4J and *shudders* Java-Logging. How do you handle that?
A while ago I read a post about a new logging wrapper. The post was incredibly misleading and bad researched so I almost dismissed slf4j. Thankfully I looked again.
slf4j was actually what we needed.
So what does it do?
It's quite simple:
It's a log syndication tool. It just collects logging output from all kinds of different sources and redirects it to your logge rof choice.
Awesome, finally someone got it right.
Using it in OSGi makes my life a lot easier. Using OSGi I could sneak in the replacements for org.apache.commons into all libraries and could also catch logging output from all other sources.
Give it a shot and please, remove ACL from your project, it's a waste of time and space.