On 14 Jul 2005, at 11:24 pm, David Kramer wrote: > Respectfully, and at the risk of sounding, ridiculously naive -- > why not > consider upgrading the I/O switching technology to Myrinet or > Infiniband for > higher-bandwidth and ultra-low latency, before buying more servers? Depends on the application. Gigabit ethernet is reasonable bandwidth, it's just not very good latency. So if you do a lot of properly parallel MPI stuff where fast job turnaround is important, then the low latency interconnect might be a winner. But for those of us (and Sanger is an example) where throughput is paramount, and individual job turnaround less so, the embarrassingly parallel single- threaded job approach is king; in such scenarios low latency interconnects cost a lot of money for relatively little gain, and increasing node count gives a better return. I've even heard this said in fields where traditionally low latency interconnects were de rigeur; at ClusterWorld a couple of years ago a speaker from Chrysler said that he'd abandoned low latency interconnects in favour of gigabit ethernet. He lost about 10% in application performance, but reduced his hardware cost by 50%, so he bought twice as many nodes (since his application didn't scale beyond 16 or so processors, he got better throughput from just having two small clusters running jobs independently, rather than one small cluster with low latency interconnect or one large cluster with gigabit) Tim -- Dr Tim Cutts Informatics Systems Group, Wellcome Trust Sanger Institute GPG: 1024D/E3134233 FE3D 6C73 BBD6 726A A3F5 860B 3CDD 3F56 E313 4233