[Bioclusters] Biocluster Build Diary #1: installing and configuring GridEngine on a 64CPU life science linux compute farm

[chris dagdigian]

[ warning: I just posted a similar message to the gridengine users 
mailing list. I expect to be flamed and/or corrected by the experts so 
my "diary" may updated or corrected quite a bit over the next few days ]

Hi folks,

After snneaking over from the Platform LSF world I've just 
built/configured my  first SGE-based life science compute farm.

Overall my reaction is hugely positive. Despite being a beginner I was
able to install and configure SGE to do what the cluster owners wanted.
The system is still going through its shakedown period but at first
blush it is delivering everything that was promised.

My one negative experience was reading in the quickstart documentation
that in order to install execution hosts I was expected to physically
login to each machine as root and manually run an interactive shell script.

That was just too horrific for words for someone who is used to a
systemimager-like world where I can deal with 1 "golden client" and have
my changes pushed out to dozens or even hundreds of other cluster members.

I ended up doing it the recommended way just to punish myself and then I
sat down and wrote some Perl and shell scripts so that I'd never, ever
have to do it again. The scripts essentially recreate the configured SGE
environment using only 'qconf' commands run from the master host.

Now all I need to do to rebuild the system from scratch is run the
'install_qmaster' script interactively. Once that is done three perl
scripts take care of everything else. No need to touch the cluster
compute elements at all (as long as their NFS mounts and startup scripts
are cool...)

The scripts and a "diary" from the build and configure process have been
placed online at http://bioteam.net/dag/biocluster-diary-1/

Feedback, comments, corrections, etc. are welcome.


Regards,
Chris


-- 
Chris Dagdigian, <dag@sonsorol.org>
Life Science IT & Research Computing Geek- http://BioTeam.net
Office: 617-666-6454, Mobile: 617-877-5498, Fax: 425-699-0193
Yahoo IM: craffi