[BiO BB] How to FTP 34,000 protein files? Is it possible?

Theodore H. Smith delete at elfdata.com
Fri Dec 9 14:09:54 EST 2005


I'm trying to download the files at:

ftp://ftp.rcsb.org/pub/pdb/data/structures/all/pdb

There are 34,142 files in this directory as of today.

I have at my disposal, a Mac, a Linux box, and a Windows PC.

OSX's Finder tells me "server disconnected us" when it tries to list  
the contents of this directory, because it can't handle that many  
files over FTP.

I tried wget on Linux. It also told me "invalid server response".

Fetch on Mac handled better in that it could list the whole contents  
of this directory. But trying to download either results in one  
README file being downloaded, or the entire Mac crashing if I attempt  
to drag and drop 34,142 files to the Finder!

Any suggestions for downloading? Like I said, I have a Mac, a Unix,  
and a Windows PC.

Honestly I am surprised at wget. I've never seen it fail before.



More information about the BBB mailing list