Grids and comments

Because of time constraints I am spending a bit less time on my blog lately. Unfortunately, the spam filter on this blog requires some manual attention, and I also had trouble with “normal” people who used bad language and were clearly not looking for any kind of informative debate. That’s why I shut down the ability to comment for now. If you’d like to respond to a blog post, you can still use the contact button, and I’ll update my blog with any useful additions you might have.

On a more positive note, through a blog contact, I am now reading a book called “IBM WebSphere eXtreme Scale 6“. WebSphere XS is formerly known as Object Grid, and is the Grid solution from IBM (can also run outside WebSphere). I took an interest a while back in Grid computing, and on my current project we are doing some CPU intensive calculations on a grid to speed them up quite a bit. Grid computing is often seen as something that “only happens at Google-like companies” but it is actually quite useful and easy to do in smaller applications as well.

I come to think of a grid as more of a “smart cluster”. In a normal cluster, nodes are typically mirrored and scale out because more of the same small tasks are spread out, and when there is a need to fail over, the opposite happens. In a grid, nodes are typically not mirrored, but data and cpu capacity is spread out with more intelligence and flexibility, which means that it can work for large tasks as well. That means coding for a grid requires some extra work and up-front thought. From a systems maintenance and availability perspective though, grids are very similar to clusters (and if you have a grid, you don’t need to cluster it as well).

There are several grid frameworks out there that help a lot with building your own grid system. Most of them are either processing-oriented (typically map/reduce, for instance GridGain), or data-oriented (distributed hashmap, like Coherence or Infinispan). Websphere XS does both, is a relatively mature product and integrates with several frameworks like JPA and Spring. The book is hands-on and easy to read for such a technical topic. A sample chapter is available here.