[Bioclusters] gigabit ethernet performance

[george wm turner]

> scp cyphers

Usually, the default encryption is Triple DES where the
DES algorithm is applied three times.  Just about any
of the other cyphers will give you a 3X performance
boost.  I measured it once or twice and that's about
what I saw.  Blowfish was designed to encrypt streaming
data so it usage with scp should be obvious.  I think
DES is a block cypher where the data is encrypted in
chunks but I admit I don't remember for sure.

> cluster/GigE rates

As for cluster IO throughput, what we noticed when
we moved our fileservers from 100 Mb/s to GigE
was that the IO server (disk and/or PCI bus) became the
bottleneck; painfully slow response to file commands (ls, etc.)
My hypothesis is that before the network was acting as
a flow mediator; i.e. a throttle or governor.  Now
the IO requests just slams into the fileserver faster than it
can deal with it, I/O requests backup, load shoots up
(under Linux, one outstanding IO request will induce a
load of one) and the humand get annoyed.   A common occurrence
for this is when one user doing the same thing (IO wise) gets
scheduled onto several of the compute nodes at once.  That's what
the parallel file system or local scratch is for but users
don't seem to want to think outsde their home file space
for some reason.

george wm turner
uits/rats @ indiana university
812 855 5156

On Sun, 11 Jul 2004, Joe Landman wrote:

> Hi Chen:
>
>   This is interesting.  Someone once had mentioned to me that the
> "blowfish" encryption was the best for performance (e.g. least
> expensive), though I have not ascertained whether or not this is true by
> rigorous measurement.
>
>   While SCP is secure in that all information is encrypted, it is costly
> in that the encryption algorithms are not lightweight.
>
>   If you want to try alternative encryption schemes, try using the
>
> 	-c algorithm
>
> on the scp command line.  Note also that this depends strongly upon
> whether or not you are using version 2 of the ssh protocol, and how your
> OpenSSH has been compiled.
>
> You might get "faster" performance using a "kerberized" rcp, which will
> encrypt the login portion but not the data.  You might also look at
> rsync and other methods as well, though if security is the primary
> concern on the nodes, you probably want to stick with ssh.
>
> Joe
>
>
>
> On Sun, 2004-07-11 at 11:34, Chen Peng wrote:
> > Hi all,
> >
> > Thank all of you for valuable suggestions and being involved in
> > thediscussion.
> >
> > We have finally figured out where is the bottleneck for our
> > gigabitethernet. To put it simple, the FTP is limited to 12MB-13MB/s
> > isbecause of disk IO. Our main FTP server got some file system
> > problemand disk is getting slower than it should be. After fixing the
> > diskproblem, the FTP speed restores to be around 20MB/s.
> >
> > Surprisingly, the SCP performance is heavily bound to CPU power. Inthe
> > following table we compared performance among four differentmachines.
> > Note that this is NOT benchmark, we use this table only tounderstand
> > where is the bottleneck.
> >
> > All the operation is against a 100MB file.
> >
> > Model           Powerbook       Xserve G4       IBM X440
> > SunFire 280R
> > CPU             1GHZ G4         2x1.25G G4      4x2.0G Xeon
> > 1.2GHZSPARC
> > Disk            IDE 4200rps     ATA-133 IDE     SISC-RAID 1     SCSI
> >
> > Copy            5.710 sec       2.523 sec       4.390 sec
> > 3.187sec
> > Speed           17.51 MB/s      39.63 MB/s      22.78 MB/s
> > 31.38MB/s
> >
> > SCP             17.575 sec      10.135 sec      8.787 sec
> > 26.680sec
> > Speed           5.690 MB/s      9.868 MB/s      11.37 MB/s
> > 3.748MB/s
> >
> > FTP             6.586 sec       5.288 sec       ---             ---
> > Speed           15.18 MB/s      18.91 MB/s
> >
> > Copy            cp ./dummy.100M ./dummy.100M.2
> > SCP             scp *******:/tmp/dummy.100M ./dummy.100M.2
> > FTP             ncftpget -u*** -p*** ftp://******/dummy.100M
> >
> > It isclear that disk IO is not the bottleneck for all the models. The
> > morepowerful is the CPU, the better is the scp performance. In
> > addition,we compared FTP performance. As FTP involves much less CPU
> > workload,the speed instantly boosts to 19MB/s, while SCP is only
> > 10MB/s for thesame Xserve G4.
> >
> > Therefore, to tune the network performance, we should focus on
> > eachconnection point instead of only looking at the switch or NIC. CPU
> > andHD speed is also important. Traffic from machine A to B involves:
> >
> >       (1)        (2)        (3)        (4)
> > A(HD) --> A(NIC) --> SWITCH --> B(NIC) --> B(HD)
> >
> > Allconnection from 1 to 4 need to be checked and benchmarked
> > carefully.
> >
> > Cheers,
> > --
> > Chen Peng <chenpeng@tll.org.sg>
> > Senior System Engineer
> > Temasek Life Sciences Laboratory
> --
> Joseph Landman, Ph.D
> Scalable Informatics LLC,
> email: landman@scalableinformatics.com
> web  : http://scalableinformatics.com
> phone: +1 734 612 4615
>
> _______________________________________________
> Bioclusters maillist  -  Bioclusters@bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters
>