[Bioclusters] mpiBLAST

[Aaron Darling]

Hi David,
We've generally experienced the best performance using 2-4 times the number of fragments as worker processes. If you have 34 CPUs you would have 33 workers, so you want more than 66 fragments.  I would try N=75, depending on the size of your database. N does not need to be a multiple of the number of workers because different fragments take different amounts of time to search depending on size and sequence content. 
We haven't benchmarked the difference between using ncbi's threads:
mpirun -np 16 mpiblast -a 2 
and using 1 mpiBLAST process per cpu:
mpirun -np 32 mpiblast
but I would guess that the former would give better performance as long as you can ensure that only one mpiBLAST process is started per node. 

We have posted a paper describing mpiBLAST online at:
http://www.cs.wisc.edu/~darling/mpiblast-cwce2003.pdf
the paper contains some timing data with regard to variable number of fragments and worker nodes.

-Aaron 

On Thu, Jun 12, 2003 at 04:23:12PM -0400, Lapointe, David wrote:
> I am still trying to figure out the numerology behind mpiblast. I
> haven't applied the patches to mpiformatdb yet.
> 
> 16 nodes dual cpu 2.4G XEON
> 1 node dual cpu 2.4G XEON
> 
> That makes 34 cpus total.
> 
> So is the N the number of cpu's ( minus 1 or 2) in the mpiformatdb
> command or the number of dual nodes? Jeremy seemed to do well with the
> number of cpus, but I'm not getting results that way.
> 
> On the other hand, using 30 as N for mpiformatdb (31 sections of nr) and
> mpirun -np 16 mpiblast -p blastp -d nr -a 2 -i in.file -o out.file
> 
> gives me about 4-5 sec per query, using the -a 2 (use 2 cpus) flag.
> 
> 
> Also I am curious about Eric Engelhard's statement last year (
> Mar01,2002) "but my
> work cluster (18 nodes 2GB RAM/node) cuts through >1500 queries/minute
> against nr."  That's 40 msec per query. He didn't indicate running
> special hardware. If that is true how is that achieved?  It's >100 times
> what people ( Jeremy, John) have been reporting recently.
> 
> David
> --------
> David Lapointe
> Information Services
> UMass Medical School
> Worcester MA 01655