[Bioclusters] parallel blast???

[David Thomas]

I just came from IEEE Cluster 2002 where there was a talk from the PVFS -- 
parallel virtual file system.

I understand that the performance of this software is outstanding.
It's not stable enough and fault-tolerant enough for the enterprise but it 
might be interesting to test it out for ourselves on BLAST.  Has anyone
played with it?

Dave

On Sat, Sep 21, 2002 at 07:01:34PM -0400, Chris Dagdigian (dag@sonsorol.org) wrote:
Subject: Re: [Bioclusters] parallel blast???
Message ID: <13526FA8-CDB6-11D6-8DCD-00039397F4C8@sonsorol.org>

> 
> Last time I looked at them solid state disks were amazingly expensive. 
> I was thinking about trying them out as swap devices on a big 
> alphaserver but ended  up deciding to spend the $$ on more physical 
> memory for the system.
> 
> In a blast or blast-farm context I'd probably just skip the solid state 
> disks and instead put the databases into a ramdisk. That would be a 
> cheaper approach since you don't really need the 
> data-is-kept-when-power-goes-away or the backup hard disk that solid 
> state systems give you. You also are limited by whatever pipe connects 
> the SSD to the system (SCSI?).
> 
> Even ramdisks are of limited utility given the size and growth rate of 
> some of the more common sequence databases -- you'd fall behind 
> eventually.
> 
> Although -- if you put 1 or 2 GB ramdisks in each of your cluster nodes 
> and then set up a system for chunking blast databases into 
> ramdisk-friendly sizes you could build a really fast blast farm. In 
> that context the performance bottleneck would then become the time and 
> resources needed to merge the XML output from N queries against split 
> databases into a single result file. I've seen such systems in the past 
> and merging the results could in some cases take longer than the actual 
> search did.
> 
> -Chris
> 
> 
> 
> On Tuesday, September 17, 2002, at 08:38  AM, Steve Gaudet wrote:
> 
> > Hello Chris,
> >
> >> <snip>
> >>
> >> You can have the fastest server on earth but if you searching with
> >> blast against an NFS mounted database and your network or
> >> fileserver is
> >> slow then your blast searching speeds will be horrible. Give
> >> me a small
> >> number of speedy linux boxes and I can bring a $300,000
> >> NFS/NAS system
> >> to its knees. Storage does matter.
> >
> > Anyone ever look or try solid state disks?
> >
> >
> >> <snip>
> 
> _______________________________________________
> Bioclusters maillist  -  Bioclusters@bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters