[Bioclusters] (no subject)

Dan Bolser bioclusters@bioinformatics.org
Tue, 30 Mar 2004 19:00:04 +0100 (BST)


On Wed, 31 Mar 2004, LAI Loong Fong wrote:

> Hmm strange that you are getting 2GB per process, my large job normally 
> died around the 3GB on a standard linux kernel. Size of swap got 
> nothing to do with this limit especially when you already has 4GB of 

This was just a rumour I heard about true64. Kernel limits physical memory
to the size of the swap for some reason.


> physical RAM. ulimit is actually dependent on the shell that you use. 
> Maybe more info on your jobs that you are running and the kernel and 
> hardware that you are using before we can offer more help.

The job is my dumb approach to memory allocation ...

#!/usr/bin/perl -w
 
my $x = "xoxhelloxjdadfadsfdqassh aadjo";
 
for(0..25) {
  $x .= $x;
}

warn "OK\n";

my $y = 0;
while(1){
  $y ++;
}

:)

uname -a
Linux bio8 2.4.20-8bigmem #1 SMP Thu Mar 13 17:32:29 EST 2003 i686 i686
i386 GNU/Linux

more /etc/redhat-release
Red Hat Linux release 9 (Shrike)

Is there some way I can find out the make / model of the machines without
going down to the server room and looking at the label?

They are dual cpu (hyperthreaded) xeon ... My colegue told me that often
on dual machines the memory is split, one block per cpu?

Thanks very much again for your help,

Dan.

> LAI Loong-Fong
> 
> On Mar 30, 2004, at 8:21 PM, Dan Bolser wrote:
> 
> > On Tue, 30 Mar 2004, LAI Loong Fong wrote:
> >
> >> The limitation actually lies in the 32 bit architecture of Xeon.
> >> Physical 32 bit limitation is 4GB, PAE gives us up to 64 GB. For 
> >> normal
> >> Linux, you should have about 3GB per process and maybe can tune the
> >> kernel to use 3.5GB per process.
> >
> > Hmmm..... Looks like we are only getting 2GB per process (2GB per 
> > CPU?).
> >
> > We are using standard linux, but with a swap space below the size of 
> > the
> > ram (2GB). Could this be a problem? (i.e. true64 style).
> >
> > ulimit -a
> > core file size        (blocks, -c) 0
> > data seg size         (kbytes, -d) unlimited
> > file size             (blocks, -f) unlimited
> > max locked memory     (kbytes, -l) unlimited
> > max memory size       (kbytes, -m) unlimited
> > open files                    (-n) 1024
> > pipe size          (512 bytes, -p) 8
> > stack size            (kbytes, -s) 8192
> > cpu time             (seconds, -t) unlimited
> > max user processes            (-u) 7168
> > virtual memory        (kbytes, -v) unlimited
> >
> > Mem:  4124724k av,
> > Swap: 2040244k av
> >
> > Thanks very much for your help,
> >
> > Dan.
> >
> >
> >>
> >> LAI Loong-Fong
> >>
> >> On Mar 30, 2004, at 2:46 PM, Dan Bolser wrote:
> >>
> >>>
> >>> I have a question about Xeon and memory... It looks like I have one 
> >>> Gb
> >>> per
> >>> CPU, and not 4 Gb for the 4 CPU without restriction. Is this problem 
> >>> at
> >>> the hardware level?
> >>>
> >>> What is the maximum amount of memory a CPU can use?
> >>>
> >>> I heard talk of a Tb memory machine, but it was part of a 1000 node
> >>> cluster, so I am thinking OK 1 Gb per node.
> >>>
> >>> Can anyone clarify this for me?
> >>>
> >>> Cheers,
> >>> Dan.
> >>>
> >>> On Mon, 29 Mar 2004, Philip MacMenamin wrote:
> >>>
> >>>> On Monday 29 March 2004 05:38 am, you wrote:
> >>>>> hello all,
> >>>>>
> >>>>> i'm interested to the constructiong of a linux cluster of computer
> >>>>> for
> >>>>> bioinformatics purpose. but i dont have a cluu about the performans
> >>>>> it
> >>>>> should have. is there any one who can give any suggestion?
> >>>>>
> >>>>> in  the hope of an answer
> >>>>>
> >>>> Its kind of a nebulous question.
> >>>>
> >>>> Basically... buy Xeon || AMD two way boxes. Be boring, look at Dell 
> >>>> or
> >>>> penguin computing or something. Dont bother with 64 bit. Think about
> >>>> that in
> >>>> about 2 years time. It will irritate you in 2 months after you buy
> >>>> it, *i
> >>>> promise*.
> >>>>
> >>>> Dont buy from some indy very cheap manufacturer, cause, if the nodes
> >>>> will
> >>>> fail, you will be left waiting... or make them give you a decent
> >>>> guarantee.
> >>>> Or a big box of bits.
> >>>>
> >>>> Look at the price of the chip : speed ratio. This graph will have an
> >>>> inflection point, at which more money input gives diminishing 
> >>>> returns
> >>>> speed-wise. Buy at this inflection point. This changes all the
> >>>> time... of
> >>>> course. There is no point in buying the *best* out there right this
> >>>> second.
> >>>> Just buy a little behind it, and buy an extra box or two.
> >>>>
> >>>> Find out what you want to run.
> >>>> Buy as enough memory so it doesn't thrash your disks. (Or just fill
> >>>> it with
> >>>> memory).
> >>>>
> >>>> Spending about 2 grand (USD) on them a piece (I dont know what that
> >>>> is in
> >>>> Lira) should buy you something decent operating at 3 gigs, with a
> >>>> couple of
> >>>> gigs ram.
> >>>>
> >>>> Then have a look at
> >>>> http://bioinformatics.org/biobrew/
> >>>> www.rocksclusters.org/
> >>>>
> >>>> Sorry if this is very uninspiring/boring advice... its just, you 
> >>>> want
> >>>> to keep
> >>>> everything as simple as you can. There will be tricky bits no matter
> >>>> how
> >>>> simple you make everything.
> >>>> Good luck.
> >>>> Philip.
> >>>> _______________________________________________
> >>>> Bioclusters maillist  -  Bioclusters@bioinformatics.org
> >>>> https://bioinformatics.org/mailman/listinfo/bioclusters
> >>>>
> >>>
> >>> _______________________________________________
> >>> Bioclusters maillist  -  Bioclusters@bioinformatics.org
> >>> https://bioinformatics.org/mailman/listinfo/bioclusters
> >>
> >
>