[Bioclusters] gigabit ethernet performance

[Joe Landman]

Hi Chen:

  This is interesting.  Someone once had mentioned to me that the
"blowfish" encryption was the best for performance (e.g. least
expensive), though I have not ascertained whether or not this is true by
rigorous measurement.

  While SCP is secure in that all information is encrypted, it is costly
in that the encryption algorithms are not lightweight.  

  If you want to try alternative encryption schemes, try using the

	-c algorithm

on the scp command line.  Note also that this depends strongly upon
whether or not you are using version 2 of the ssh protocol, and how your
OpenSSH has been compiled.

You might get "faster" performance using a "kerberized" rcp, which will
encrypt the login portion but not the data.  You might also look at
rsync and other methods as well, though if security is the primary
concern on the nodes, you probably want to stick with ssh.

Joe



On Sun, 2004-07-11 at 11:34, Chen Peng wrote:
> Hi all,
> 
> Thank all of you for valuable suggestions and being involved in
> thediscussion.
> 
> We have finally figured out where is the bottleneck for our
> gigabitethernet. To put it simple, the FTP is limited to 12MB-13MB/s
> isbecause of disk IO. Our main FTP server got some file system
> problemand disk is getting slower than it should be. After fixing the
> diskproblem, the FTP speed restores to be around 20MB/s.
> 
> Surprisingly, the SCP performance is heavily bound to CPU power. Inthe
> following table we compared performance among four differentmachines.
> Note that this is NOT benchmark, we use this table only tounderstand
> where is the bottleneck.
> 
> All the operation is against a 100MB file.
> 
> Model           Powerbook       Xserve G4       IBM X440       
> SunFire 280R
> CPU             1GHZ G4         2x1.25G G4      4x2.0G Xeon    
> 1.2GHZSPARC
> Disk            IDE 4200rps     ATA-133 IDE     SISC-RAID 1     SCSI
> 
> Copy            5.710 sec       2.523 sec       4.390 sec      
> 3.187sec
> Speed           17.51 MB/s      39.63 MB/s      22.78 MB/s     
> 31.38MB/s
> 
> SCP             17.575 sec      10.135 sec      8.787 sec      
> 26.680sec
> Speed           5.690 MB/s      9.868 MB/s      11.37 MB/s     
> 3.748MB/s
> 
> FTP             6.586 sec       5.288 sec       ---             ---
> Speed           15.18 MB/s      18.91 MB/s
> 
> Copy            cp ./dummy.100M ./dummy.100M.2
> SCP             scp *******:/tmp/dummy.100M ./dummy.100M.2
> FTP             ncftpget -u*** -p*** ftp://******/dummy.100M
> 
> It isclear that disk IO is not the bottleneck for all the models. The
> morepowerful is the CPU, the better is the scp performance. In
> addition,we compared FTP performance. As FTP involves much less CPU
> workload,the speed instantly boosts to 19MB/s, while SCP is only
> 10MB/s for thesame Xserve G4.
> 
> Therefore, to tune the network performance, we should focus on
> eachconnection point instead of only looking at the switch or NIC. CPU
> andHD speed is also important. Traffic from machine A to B involves:
> 
>       (1)        (2)        (3)        (4)
> A(HD) --> A(NIC) --> SWITCH --> B(NIC) --> B(HD)
> 
> Allconnection from 1 to 4 need to be checked and benchmarked
> carefully. 
> 
> Cheers,
> --
> Chen Peng <chenpeng@tll.org.sg> 
> Senior System Engineer
> Temasek Life Sciences Laboratory
-- 
Joseph Landman, Ph.D
Scalable Informatics LLC,
email: landman@scalableinformatics.com
web  : http://scalableinformatics.com
phone: +1 734 612 4615