BioTeam first became interested in Univa UD’s software efforts after hearing Univa CTO Steve Tueckespeak in Regensburg, Germany at the 2007 Grid Engine workshop. Shortly after that event Univa formally became Univa UD after merging with United Devices. At the time, Steve’s company seemed to be one of the few companies positioning themselves to offer full support and professional services encompassing commonly used open source products such as Sun Grid Engine that BioTeam often works with in the field. Individually these popular open source resources are relatively easy to acquire but Univa UD seemed to be making an interesting effort to become the one stop shop for a fully supported and integrated set of commonly required tools and technology.

One outcome of the new Univa UD is the UniCluster Express effort. UniCluster Express is a cluster tool-stack capable of deploying fully configured and integrated set of technologies including Grid Engine 6.1, Grid Engine Analysis & Reporting Console (“ARCo”), Ganglia, central monitoring console and (incredibly) a fully operational Globus grid computing infrastructure. It itself is fully open-sourced.

All of these technologies, of course, are independently available and can be deployed by anyone with sufficient motivation and time. The ease of deployment for these technologies runs the full range: easy (Ganglia) — moderate (Grid Engine & ARCo) — very complex (Globus stack) so one very real value of UniCluster is the ability to deploy all of these technologies, simultaneously, in a fully integrated and supported manner.

Of significant note is that Univa UD plans to offer free support via Grid.org to all users of UniCluster Express. Given the capability and complexity of some of the components (Grid Engine and Globus in particular) this is a significant ‘selling point’ for potential users and Univa UD customers.

UniCluster also ships with a capable internally-developed Java-based central monitoring console (also fully open source) that uses the secure Globus web services framework to simultaneously collect, distill and present data collected from multiple grid sources including Grid Engine ‘qstat’, the ARCo job reporting database and the rrd-based system status monitoring databases created by Ganglia. Data from all of these sources is consolidated into a single console for monitoring and reporting.

We are largely a professional services company. To maintain our own technical skills we encourage our consultants to experiment in the lab with new products and technologies whenever there is downtime available. UniCluster express has been on our “to-do” list since late 2007 but evaluation efforts kept getting pushed back by client projects and consulting engagements. We had also spent most of early 2008 getting deeply familiar with Amazon Web Services, including the EC2 compute cloud and S3 storage services.

BioTeam has talked publicly about some of our EC2 cloud computing projects including a demonstration shown at the 2008 Open Source Grid & Cluster Conference showcasing the automatic deployment and self-organization of full Grid Engine clusters within the EC2 cloud. Knowing our work with and interest in EC2 and related technologies, Univa UD made us an offer that we could not refuse …

Challenge
Univa came to us with an interesting project proposal — a challenge to deploy their UniCluster software within the Amazon Web Services framework and then document the experience. To facilitate, Univa UD would fund four days of BioTeam professional services effort so that we could treat this as a official company project and add it to our busy summer consulting schedule

Download pdf Howto: UniCluster & Amazon EC2