[Bioclusters] OS X and NFS

[Glen Otero]

Hi Bill-

I'd like to try YDL.

Glen

On Jul 14, 2005, at 7:18 PM, William Harman wrote:

> If there are interested parties that would like to try Yellow Dog  
> Linux, and
> the corresponding cluster distro, Y-HPC, let me know off line, and  
> I will
> arrange for a 30-45 days, no cost evaluation, will also provide  
> technical
> support if required.  Supported are ext3, XFS, PVFS2.
> http://www.terrasoftsolutions.com/products/y-hpc/applications.shtml
>
> I would also suggest that you might want to check our web site in  
> early
> August for an announcement concerning a Bio package which Terra  
> Soft will
> release for a PowerPC - Linux platform, as well as some new PowerPC
> products.  www.terrasoftsolutions.com
>
>
> Bill Harman,
> Director of Sales
> Terra Soft Solutions
> Salt Lake City office
> P - (801) 572-9252  F - (801) 571-4927
> wharman at terrasoftsolutions.com
> wharman at prism.net
> billharman at comcast.net
> skype: harman8015729252
>
>
> -----Original Message-----
> From: bioclusters-bounces+wharman=prism.net at bioinformatics.org
> [mailto:bioclusters-bounces+wharman=prism.net at bioinformatics.org]  
> On Behalf
> Of John H. Lee
> Sent: Thursday, July 14, 2005 4:38 PM
> To: Clustering, compute farming & distributed computing in life  
> science
> informatics
> Subject: Re: [Bioclusters] OS X and NFS
>
>
> Since Juan's system is OS X, can anyone offer a suggestion for OS X?
> Cluster filesystems like Lustre and PVFS sound promising for Linux,  
> but what
> options do we have on Macs?  Any experience with Xsan?  With a  
> fibre channel
> limit of 64 endpoints, can Xsan even be considered for systems as  
> large as
> Tim's?
>
> We have a 16-node Xserve cluster with one Xserve RAID.  The RAID is
> connected via FC to one server, which exports the volumes via AFP  
> and NFS
> over gigabit ethernet.  As expected, the AFP/NFS server is a  
> bottleneck.
>
> -John
>
> On Jul 14, 2005, at 1:32 AM, Tim Cutts wrote:
>
>
>> On 13 Jul 2005, at 7:01 pm, M. Michael Barmada wrote:
>>
>>
>>
>>> Hi Carlos,
>>>
>>> If its any help, we also had similar problems with our cluster.
>>> Our solution
>>> was to train the users to include code in their scripts that would
>>> create
>>> local directories (on the compute node - in /tmp) and copy the
>>> files they
>>> needed to those directories, then do their computing locally and
>>> copy back
>>> the results.
>>>
>>>
>>
>> Absolutely.  And preferably do the copying with something other
>> than NFS too - rcp or rsync work well, or the scheduler's built-in
>> mechanism.
>> Most batch schedulers have built in abilities to this - LSF
>> certainly does, in the form of lsrcp and various options to bsub.
>> I don't know about SGE - I'm not familiar with it, but I imagine
>> the same sort of features are available.
>>
>> It really is quite amazing how badly NFS scales.  I remember having
>> serious problems with it on the first Linux cluster I built at
>> Incyte's UK office about 6 years ago, and that was just 7 dual-CPU
>> nodes talking to a Sun E3000 NFS server.  It didn't crash, but it
>> got *really* slow - and that was deliberately caching the data
>> locally (I wrote wrapper scripts around blastall and other
>> applications to cache the databases locally, blowing them away by a
>> least-recently-used method if there wasn't room).
>>
>> Sanger's current 1100 node cluster still has NFS in places, and it
>> regularly causes us grief.  Our medium-term aim is to remove pretty
>> much all NFS from the cluster altogether, with the possible
>> exception of automounted home directories, and use cluster
>> filesystems like Lustre for shared data.
>>
>> Tim
>>
>> -- 
>> Dr Tim Cutts
>> Informatics Systems Group, Wellcome Trust Sanger Institute
>> GPG: 1024D/E3134233 FE3D 6C73 BBD6 726A A3F5  860B 3CDD 3F56 E313  
>> 4233
>>
>> _______________________________________________
>> Bioclusters maillist  -  Bioclusters at bioinformatics.org
>> https://bioinformatics.org/mailman/listinfo/bioclusters
>>
>>
>
> --
> John H. Lee
> Berkeley Phylogenomics Group
> http://phylogenomics.berkeley.edu
>
> _______________________________________________
> Bioclusters maillist  -  Bioclusters at bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters
>
>
> _______________________________________________
> Bioclusters maillist  -  Bioclusters at bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters
>
>