[Bioclusters] FPGA in bioinformatics clusters (again?) Ref: Cray and SGI

George Magklaras georgios at ulrik.uio.no
Mon Jan 23 09:22:08 EST 2006

Tim Cutts wrote:
> On 19 Jan 2006, at 6:48 pm, George Magklaras wrote:
>> Tim Cutts wrote:

> The processor isn't what gives SGI a niche, as you say; you can buy  
> less outrageously expensive Itanics from elsewhere, if you wish.   What 
> SGI have that is, at present, more or less unique, is the  ability to 
> build machines where you can scale CPUs, memory and I/O  independently, 
> and which you can scale essentially as far as you like  until your money 
> runs out.  This can be very important for some niche  applications.
> You want a machine with only 4 CPUs but a terabyte of RAM?  You can  buy 
> an Altix in that configuration.  I don't know of anyone else  evenly 
> vaguely mainstream that offers machines with as much physical  memory in 
> a single system image, although please enlighten me if  there is one.  
> We have an Altix here with 4 CPUs and 192GB of RAM -  not nearly as big 
> as it could be, but still a configuration  impossible to buy from anyone 
> else, at the time (and still is, as far  as I know).  I'd rather it 
> wasn't IA64, but it's the memory that's  important to us, not the CPU 
> technology.  And it's the memory that  makes them expensive anyway.  48 
> x 4GB DIMMs are going to be  expensive regardless of whether you buy 
> them from SGI or anyone  else.  That RAM alone is around $50,000 at 
> current prices (http:// 
> osnews.pricegrabber.com/search_getprod.php/masterid=634351)
I too do not know of a system config that would give you 4 CPUs and 1 
Tbyte of RAM and I do not understand why one would like to stretch the 
CPU, memory and I/O dependency to certain limits (if you have a huge 
data set that you would like to place in RAM, why is it optimum to place 
it in 2 Tbytes of RAM and have only 4 CPUs to crunch and not 8...?), 
especially if you are a business and you make money. Well balanced 
systems do not dictate 1 CPU accessing 4 Tbytes of RAM. Although I do 
know of machines that could give you 4 CPUs and around half a Tbyte of 
RAM a couple of years ago, which is, well scalable by many folks 
standards (based on the p5-570 memory controller that IBM's POWER 5 570 
boxes wear) and does fit the current profile of your Altix.

But my point is that SGI boasts its kernel scalability features by 
quoting the X and the Y tests and not the independent scaling as you 
present it above.  Their SMP scalability is of course a result of tweaks 
they have performed, based AND on NUMA/SMP scalability techniques 
pioneered by many other earlier OS platforms and Linux itself.

As for the memory scalability. 64 bit processor addressing on the makes 
the theoretical memory addressing limit to 16 exabytes. Which means that 
what limits the addressable RAM is the chipset limits, which comes down 
to the memory controller design (bus arbitration, number of slots, etc). 
A major manufacturer has the technical means to scale that on the 
"motherboard" backplane, provided that they are going to make money 
selling them. You can then say that they offer indeed a unique config, 
but that is a result of eyeing a market, not that of technological 
innovation. The ability to offer a unique config based on existing 
technologies (what you are saying) differs from the ability to offer a 
unique technology(what I am talking about).

> The NSA and their friends do have the big budgets you mention, and  they 
> want to search large databases as fast as possible.  That means  holding 
> the whole database in memory.  You also need to load that  database into 
> memory initially very quickly; the Altix can win there  too - you just 
> keep adding HBAs and spindles until the aggregate  bandwidth reaches 
> your requirements.  We've heard of examples where  SGI have built 
> systems to load terabyte databases into RAM from  storage in a couple of 
> seconds.
> This is a market no-one else is really playing in, and because the  
> likes of the NSA and GCHQ like this sort of machine, they're not  going 
> to let the one company go under, that can provide them.
I can tell you of a branded manufacturer who did provide large RDBMS 
NUMA boxes, and does (AFYIK) provide the same service to people like the 
folks you mention off list (I do not mean to promote companies). Now, 
apart from RDBMS, the organizations you mention do work that requires 
trivial parallelization and hence they need smaller memory chunks but 
massive throughput. I am not sure that huge single image RAM chunks and 
a small number of CPUs fit that sort of workload :-).

> PS.  I realise this is bioclusters, not comp.sys.sgi.advocacy, so I  
> apologise for the off topic post - if anyone wants to continue this  
> particular discussion we should take it off list.
Quite right. I think it should be clear by now that I am not an SGI 
advocate :-).

George B. Magklaras

Senior Computer Systems Engineer/UNIX Systems Administrator
The Biotechnology Centre of Oslo,
University of Oslo

EMBnet Norway: http://www.biotek.uio.no/EMBNET/

More information about the Bioclusters mailing list