[Bio-Linux] Bio-Linux Digest, Vol 77, Issue 2
Xavier Martínez Serrano
xmarti6 at xtec.cat
Thu Feb 5 09:19:58 EST 2015
Thank you very much to all for all your sharings, they will be really
useful to us to set everything up.
Following your extended comments seems that the best option will probably
be to use Slurm and Uprade
all Biolinux to v8, and installing MATE on top. The network will be
configured manually. We will probably do
all this next week. I'll let you know if we get into big issues; hopefully
not! :)
Best, greetings,
Xavi
2015-02-05 12:18 GMT+01:00 <bio-linux-request at nebclists.nerc.ac.uk>:
> Send Bio-Linux mailing list submissions to
> bio-linux at nebclists.nerc.ac.uk
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux
> or, via email, send a message with subject or body 'help' to
> bio-linux-request at nebclists.nerc.ac.uk
>
> You can reach the person managing the list at
> bio-linux-owner at nebclists.nerc.ac.uk
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bio-Linux digest..."
>
>
> Today's Topics:
>
> 1. Biolinux8 - Best job scheduler for small cluster?
> (Xavier Mart?nez Serrano)
> 2. Biolinux8 - Is there an installation for "server mode"?
> (without X11, just x2go connection) (Xavier Mart?nez Serrano)
> 3. Re: Biolinux8 - Is there an installation for "server mode"?
> (without X11, just x2go connection) (Tony Travis)
> 4. Re: Biolinux8 - Best job scheduler for small cluster?
> (Tony Travis)
> 5. Re: Biolinux8 - Is there an installation for "server mode"?
> (without X11, just x2go connection) (Tim Booth)
> 6. Re: Biolinux8 - Best job scheduler for small cluster? (Tim Booth)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 5 Feb 2015 08:16:38 +0100
> From: Xavier Mart?nez Serrano <xmarti6 at xtec.cat>
> To: bio-linux at nebclists.nerc.ac.uk
> Subject: [Bio-Linux] Biolinux8 - Best job scheduler for small cluster?
> Message-ID:
> <
> CAPYyywinF97V3eiL_CfZEDE5uS9i_Z0o89d3x_9y_cf4QZ54Xw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> We have 4 computers with ~20CPU each in a hospital research group and we
> would like to set a cluster with them. Which is the best suitable job
> scheduler for us if we are running Biolinux7 in three of them and Biolinux8
> in the other?
>
> We have been looking at common job schedulers like: Slurm, SGE, Condor,
> Torque, ... but we wonder which best suits for our small resources, to make
> it as easy as possible.
>
> What can you recommend us?
>
> Many thanks in advance,
> Xavi
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://nebclists.nerc.ac.uk/pipermail/bio-linux/attachments/20150205/048bb640/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Thu, 5 Feb 2015 08:23:08 +0100
> From: Xavier Mart?nez Serrano <xmarti6 at xtec.cat>
> To: bio-linux at nebclists.nerc.ac.uk
> Subject: [Bio-Linux] Biolinux8 - Is there an installation for "server
> mode"? (without X11, just x2go connection)
> Message-ID:
> <
> CAPYyywjGoQ9orbWs5WQq2GJpYvesFY3Adde0z8wRhaWWr+vgzg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> We are a research group working in metagenomics, and we would like to
> install Biolinux8 in a server recently acquired in "server mode", i.e.
> whithout X11 enabled, we just need a ssh connection and might be in some
> case an x2go connection.
>
> Is there an option to install Biolinux8 in such mode, revising the
> documentation online we could not see it.
>
> Many thanks,
> Xavi
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://nebclists.nerc.ac.uk/pipermail/bio-linux/attachments/20150205/8ef9f33e/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Thu, 5 Feb 2015 10:31:22 +0000
> From: Tony Travis <tony.travis at abdn.ac.uk>
> To: <bio-linux at nebclists.nerc.ac.uk>
> Subject: Re: [Bio-Linux] Biolinux8 - Is there an installation for
> "server mode"? (without X11, just x2go connection)
> Message-ID: <54D3467A.8020903 at abdn.ac.uk>
> Content-Type: text/plain; charset="windows-1252"
>
> On 05/02/15 07:23, Xavier Mart?nez Serrano wrote:
> > Hi,
> >
> > We are a research group working in metagenomics, and we would like to
> > install Biolinux8 in a server recently acquired in "server mode", i.e.
> > whithout X11 enabled, we just need a ssh connection and might be in
> > some case an x2go connection.
> >
> > Is there an option to install Biolinux8 in such mode, revising the
> > documentation online we could not see it.
>
> Hi, Xavi.
>
> We run Bio-Linux 8 on servers using Tim's default installation without
> any issues. If you want to use Bio-Linux as an "x2go" terminal server,
> you will need X11 and the MATE desktop installed anyway. The only real
> problem is you have to add the network configuration manually to prevent
> network manager from configuring the NIC's dynamically. You can use
> "network manager" to statically configure NIC's, but it is intended to
> be used to configure NIC's dynamically on laptops or workstations.
>
> HTH,
>
> Tony.
>
> --
> Dr. A.J.Travis, University of Aberdeen, Institute of Biological and
> Environmental Sciences, Cruickshank Building, St. Machar Drive, Aberdeen
> AB24 3UU, Scotland, UK. tel +44(0)1224 272700, fax +44 (0)1224 272 396
> http://www.abdn.ac.uk, mailto:tony.travis at abdn.ac.uk, skype:ajtravis
>
>
> The University of Aberdeen is a charity registered in Scotland, No
> SC013683.
> Tha Oilthigh Obar Dheathain na charthannas cl?raichte ann an Alba, ?ir.
> SC013683.
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 5 Feb 2015 10:46:06 +0000
> From: Tony Travis <tony.travis at abdn.ac.uk>
> To: <bio-linux at nebclists.nerc.ac.uk>
> Cc: Luca Clivio <luca.clivio at marionegri.it>
> Subject: Re: [Bio-Linux] Biolinux8 - Best job scheduler for small
> cluster?
> Message-ID: <54D349EE.5050800 at abdn.ac.uk>
> Content-Type: text/plain; charset="windows-1252"
>
> On 05/02/15 07:16, Xavier Mart?nez Serrano wrote:
> > Hi,
> >
> > We have 4 computers with ~20CPU each in a hospital research group and
> > we would like to set a cluster with them. Which is the best suitable job
> > scheduler for us if we are running Biolinux7 in three of them and
> > Biolinux8 in the other?
> >
> > We have been looking at common job schedulers like: Slurm, SGE, Condor,
> > Torque, ... but we wonder which best suits for our small resources, to
> > make it as easy as possible.
> >
> > What can you recommend us?
>
> Hi, Xavi.
>
> I've run SGE under Bio-Linux 7, but I would not advocate running a
> mixture of Bio-Linux 7 and 8. Tim's Bio-Linux 7-8 upgrade script works
> very well and I would recommend upgrading your Bio-Linux 7 machines to
> Bio-Linux 8. One thing you might consider looking at is Qlustar:
>
> https://qlustar.com/
>
> They have made some commitment to support Debian/Med and I demo'ed a
> version with a Bio-Linux 'Front-end' at last year's Debian-Med Sprint:
>
> https://qlustar.com/news/qlustar-demo-debianmed-sprint
>
> Q-Leap gave a presentation at the recent Debian-Med Sprint, which I
> watched on Skype because I couldn't attend the meeting:
>
> https://wiki.debian.org/Sprints/2015/DebianMed2015
>
> Maybe Tim can tell you more because he was there. It's pay-for software,
> and I have some reservations about their licensing model, but from what
> I've seen of it Qlustar is very good cluster software. Q-Leap support
> Slurm in particular, but do consider Amdahl's Law:
>
> http://en.wikipedia.org/wiki/Amdahl's_law
>
> I'm working with Luca Clivio to install a Qlustar cluster with a
> Bio-Linux Front-end at MNI (Mario Negri Institute) in Milan:
>
> http://www.marionegri.it/mn/it/index.html
>
> Please contact Luca directly if you are interested in this project.
>
> HTH,
>
> Tony.
>
> --
> Dr. A.J.Travis, University of Aberdeen, Institute of Biological and
> Environmental Sciences, Cruickshank Building, St. Machar Drive, Aberdeen
> AB24 3UU, Scotland, UK. tel +44(0)1224 272700, fax +44 (0)1224 272 396
> http://www.abdn.ac.uk, mailto:tony.travis at abdn.ac.uk, skype:ajtravis
>
>
> The University of Aberdeen is a charity registered in Scotland, No
> SC013683.
> Tha Oilthigh Obar Dheathain na charthannas cl?raichte ann an Alba, ?ir.
> SC013683.
>
>
> ------------------------------
>
> Message: 5
> Date: Thu, 5 Feb 2015 11:02:12 +0000
> From: Tim Booth <tbooth at ceh.ac.uk>
> To: Bio-Linux help and discussion <bio-linux at nebclists.nerc.ac.uk>
> Subject: Re: [Bio-Linux] Biolinux8 - Is there an installation for
> "server mode"? (without X11, just x2go connection)
> Message-ID:
> <1423134132.23727.117.camel at wllt1771.nerc-wallingford.ac.uk>
> Content-Type: text/plain; charset="UTF-8"
>
> Hi,
>
> Tony's right - the Bio-Linux ISO is based on the Ubuntu desktop ISO, so
> you basically need an X-session to run the installer even if you
> subsequently remove X and run the thing headless. It's possible to
> strip out the X server packages while still retaining the libraries
> needed by MATE for x2go but for simplicity I'd just take out LightDM so
> X is present but never started.
>
> By "add the network configuration manually" this means that if you set
> your settings for any network card in /etc/network/interfaces then when
> the NM daemon is restarted it will leave that card alone. In my
> experience NM now works very well but on a server with fixed network
> routing it's just another layer of complexity you don't need. Use the
> command "nmcli d" to see what devices NM thinks it is managing.
>
> Cheers,
>
> TIM
>
> On Thu, 2015-02-05 at 10:31 +0000, Tony Travis wrote:
> > On 05/02/15 07:23, Xavier Mart?nez Serrano wrote:
> > > Hi,
> > >
> > > We are a research group working in metagenomics, and we would like to
> > > install Biolinux8 in a server recently acquired in "server mode", i.e.
> > > whithout X11 enabled, we just need a ssh connection and might be in
> > > some case an x2go connection.
> > >
> > > Is there an option to install Biolinux8 in such mode, revising the
> > > documentation online we could not see it.
> >
> > Hi, Xavi.
> >
> > We run Bio-Linux 8 on servers using Tim's default installation without
> > any issues. If you want to use Bio-Linux as an "x2go" terminal server,
> > you will need X11 and the MATE desktop installed anyway. The only real
> > problem is you have to add the network configuration manually to prevent
> > network manager from configuring the NIC's dynamically. You can use
> > "network manager" to statically configure NIC's, but it is intended to
> > be used to configure NIC's dynamically on laptops or workstations.
> >
> > HTH,
> >
> > Tony.
> >
> > --
> > Dr. A.J.Travis, University of Aberdeen, Institute of Biological and
> > Environmental Sciences, Cruickshank Building, St. Machar Drive, Aberdeen
> > AB24 3UU, Scotland, UK. tel +44(0)1224 272700, fax +44 (0)1224 272 396
> > http://www.abdn.ac.uk, mailto:tony.travis at abdn.ac.uk, skype:ajtravis
> >
> >
> > The University of Aberdeen is a charity registered in Scotland, No
> SC013683.
> > Tha Oilthigh Obar Dheathain na charthannas cl?raichte ann an Alba, ?ir.
> SC013683.
> > _______________________________________________
> > Bio-Linux mailing list
> > Bio-Linux at nebclists.nerc.ac.uk
> > http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux
>
> --
> Tim Booth <tbooth at ceh.ac.uk>
> NERC Environmental Bioinformatics Centre
>
> Centre for Ecology and Hydrology
> Maclean Bldg, Benson Lane
> Crowmarsh Gifford
> Wallingford, England
> OX10 8BB
>
> http://environmentalomics.org/bio-linux
> +44 1491 69 2297
>
>
>
> ------------------------------
>
> Message: 6
> Date: Thu, 5 Feb 2015 11:18:22 +0000
> From: Tim Booth <tbooth at ceh.ac.uk>
> To: Bio-Linux help and discussion <bio-linux at nebclists.nerc.ac.uk>
> Cc: Xavier Mart?nez Serrano <xmarti6 at xtec.cat>
> Subject: Re: [Bio-Linux] Biolinux8 - Best job scheduler for small
> cluster?
> Message-ID:
> <1423135102.23727.126.camel at wllt1771.nerc-wallingford.ac.uk>
> Content-Type: text/plain; charset="UTF-8"
>
> Hi Xavi,
>
> To be honest I don't really know. Hopefully someone on the list will
> have some practical experience to share. We use Torqe on a cluster here
> at CEH but it's not actually running Bio-Linux and I had no part in
> setting it up so I don't know how easy it is to configure.
>
> I have experience with Condor and I don't think it is what you need at
> all. SGE used to be good but Oracle have not looked after it well.
>
> My guess would be that Slurm is the best answer for you. The latest
> version of QIIME recommends it, which is a good sign, and I'm told that
> set-up for a small cluster is "very simple". You might well have issues
> with running a cluster of Bio-Linux 7 and 8 machines, though, because
> the packaged versions of Slurm and of the bioinformatics software will
> be different on the different nodes. I'd strongly recommend upgrading
> everything to the same version.
>
> Also, you really should set up a fast shared NFS drive that all the
> machines can see. Running a cluster without a shared filesystem is
> possible but for most bioinformatics tasks it's highly impractical.
>
> Cheers,
>
> TIM
>
>
>
>
> On Thu, 2015-02-05 at 07:16 +0000, Xavier Mart?nez Serrano wrote:
> > Hi,
> >
> > We have 4 computers with ~20CPU each in a hospital research group and
> > we would like to set a cluster with them. Which is the best suitable
> > job scheduler for us if we are running Biolinux7 in three of them and
> > Biolinux8 in the other?
> >
> > We have been looking at common job schedulers like: Slurm, SGE,
> > Condor, Torque, ... but we wonder which best suits for our small
> > resources, to make it as easy as possible.
> >
> > What can you recommend us?
> >
> > Many thanks in advance,
> > Xavi
> >
>
> --
> Tim Booth <tbooth at ceh.ac.uk>
> NERC Environmental Bioinformatics Centre
>
> Centre for Ecology and Hydrology
> Maclean Bldg, Benson Lane
> Crowmarsh Gifford
> Wallingford, England
> OX10 8BB
>
> http://environmentalomics.org/bio-linux
> +44 1491 69 2297
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Bio-Linux mailing list
> Bio-Linux at nebclists.nerc.ac.uk
> http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux
>
>
> ------------------------------
>
> End of Bio-Linux Digest, Vol 77, Issue 2
> ****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.bioinformatics.org/pipermail/bio-linux-list/attachments/20150205/322ef909/attachment.html>
More information about the Bio-linux-list
mailing list