From tbooth at ceh.ac.uk  Thu Nov  3 14:08:09 2011
From: tbooth at ceh.ac.uk (Tim Booth)
Date: Thu, 03 Nov 2011 18:08:09 +0000
Subject: [Bio-linux-dev] Qiime update
Message-ID: <1320343690.6631.55961.camel@barsukas.nwl.ac.uk>

Hi All,

Following on from the previous discussion, I have uploaded a new qiime
and bio-linux-qiime package, along with a fix to chimeraslayer.  This
should address the issues that Mike reported and also clear up some but
not all of the problems reported by "qiime print_qiime_config -t".
Specifically I now get:

FastTree is in path and version is supported ... ok
INFERNAL is in path and version is supported ... ok
AmpliconNoise install looks sane. ... ok
blast is in path and version is supported ... FAIL
blastall_fp is set to a valid path ... ok
blastmat_dir is set to a valid path. ... ok
cdbtools is in path and version is supported ... ok
cd-hit is in path and version is supported ... ok
no obvious problems with ChimeraSlayer install ... ok
clearcut is in path and version is supported ... FAIL
cluster_jobs_fp is set to a valid path and is executable ... ok
denoiser aligner is ready to use ... ok
local qiime_config has no extra params ... ok
maptplotlib version is supported ... FAIL
mothur is in path and version is supported ... ERROR
muscle is in path and version is supported ... FAIL
numpy version is supported ... ok
pynast version is supported ... ok
pynast_template_alignment_blastdb, if set, is set to a valid path ... ok
pynast_template_alignment, if set, is set to a valid path ... ok
python_exe_fp is set to a working python env ... ok
python is in path and version is supported ... ok
qiime_scripts_dir, if set, is set to a valid path ... ok
raxmlHPC is in path and version is supported ... FAIL
temp_dir, if set, is set to a valid path ... ok
template_alignment_lanemask, if set, is set to a valid path ... ok
uclust is in path and version is supported ... ok
working_dir, if set, is set to a valid path ... ok

If you are still seeing other failures, please check you have upgraded
the packages:

% dpkg -s qiime bio-linux-qiime chimeraslayer | grep ^Version
Version: 1.3.0-0ubuntu6
Version: 1:1.3.0bl-6
Version: 20101212-1ubuntu5

If there are still issues after checking this then let me know.

Of the remaining issues, a couple more things I should be able to fix,
but some of the others seem to be due to us having newer versions,
rather than older versions, of dependencies, and I'm not going to make a
downgraded package unless someone tells me it really is needed (ie. a
real error, not just a test warning).

Cheers,

TIM

-- 
Tim Booth <tbooth at ceh.ac.uk>
NERC Environmental Bioinformatics Centre 

Centre for Ecology and Hydrology
Maclean Bldg, Benson Lane
Crowmarsh Gifford
Wallingford, England
OX10 8BB 

http://nebc.nerc.ac.uk
+44 1491 69 2705

-- 
This message (and any attachments) is for the recipient only. NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless
it is exempt from release under the Act. Any material supplied to
NERC may be stored in an electronic records management system.



From a.travis at abdn.ac.uk  Thu Nov  3 16:03:13 2011
From: a.travis at abdn.ac.uk (Tony Travis)
Date: Thu, 03 Nov 2011 20:03:13 +0000
Subject: [Bio-linux-dev] Qiime update
In-Reply-To: <1320343690.6631.55961.camel@barsukas.nwl.ac.uk>
References: <1320343690.6631.55961.camel@barsukas.nwl.ac.uk>
Message-ID: <4EB2F381.8050209@abdn.ac.uk>

On 03/11/11 18:08, Tim Booth wrote:
> Hi All,
>
> Following on from the previous discussion, I have uploaded a new qiime
> and bio-linux-qiime package, along with a fix to chimeraslayer.  This
> should address the issues that Mike reported and also clear up some but
> not all of the problems reported by "qiime print_qiime_config -t".

Hi, Tim.

I've decided to uninstall your qiime package and install QIIME from 
source. I'm using this wrapper to create a custom QIIME environment 
under Bio-Linux 6, without breaking anything else:

--- cut here ---
# @(#)qiime.sh  2011-10-05  A.J.Travis

#
# Qiime environment
#

# Python
export QIIME=/usr/local/qiime
export QIIME_CONFIG_FP=$QIIME/etc/qiime_config
export PATH=$QIIME/bin:$PATH
export PYTHONPATH=$QIIME/lib/python2.6/site-packages:$PYTHONPATH

# NCBI BLAST 2.2.22
export BLAST_HOME=/usr/local/blast-2.2.22
export BLASTMAT=$BLAST_HOME/data
export PATH=$BLAST_HOME/bin:$PATH

# AmpliconNoise
export AMPLICON_NOISE=$QIIME/AmpliconNoiseV1.24
export PATH=$AMPLICON_NOISE/Scripts:$AMPLICON_NOISE/bin:$PATH
export PYRO_LOOKUP_FILE=$AMPLICON_NOISE/Data/LookUp_E123.dat
export SEQ_LOOKUP_FILE=$AMPLICON_NOISE/Data/Tran.dat

# ChimeraSlayer
export PATH=/usr/share/ChimeraSlayer:$PATH

# QIIME shell
export debian_chroot=QIIME
exec /bin/bash
--- cut here ---

> rwt017 at topcat:~$ qiime
> (QIIME)rwt017 at topcat:~$ print_qiime_config.py -t
>
> System information
> ==================
>          Platform:	linux2
>    Python version:	2.6.5 (r265:79063, Apr 16 2010, 13:57:41)  [GCC 4.4.3]
> Python executable:	/usr/bin/python
>
> Dependency versions
> ===================
>                      PyCogent version:	1.5.1
>                         NumPy version:	1.3.0
>                    matplotlib version:	0.98.5.3
>                 QIIME library version:	1.3.0
>                  QIIME script version:	1.3.0
>         PyNAST version (if installed):	1.1
> RDP Classifier version (if installed):	rdp_classifier-2.0.jar
>
> QIIME config values
> ===================
>                      blastmat_dir:	None
>       topiaryexplorer_project_dir:	None
>      pynast_template_alignment_fp:	None
>                   cluster_jobs_fp:	/usr/local/qiime/bin/start_parallel_jobs.py
> pynast_template_alignment_blastdb:	None
>                      torque_queue:	friendlyq
>    template_alignment_lanemask_fp:	None
>                     jobs_to_start:	1
>                 cloud_environment:	False
>                 qiime_scripts_dir:	/usr/local/qiime/bin
>             denoiser_min_per_core:	50
>                       working_dir:	None
>                     python_exe_fp:	python
>                          temp_dir:	None
>                       blastall_fp:	blastall
>                  seconds_to_sleep:	60
>
>
> running checks:
>
> FastTree is in path and version is supported ... ok
> INFERNAL is in path and version is supported ... ok
> AmpliconNoise install looks sane. ... ok
> blast is in path and version is supported ... ok
> blastall_fp is set to a valid path ... ok
> blastmat_dir is set to a valid path. ... ok
> cdbtools is in path and version is supported ... ok
> cd-hit is in path and version is supported ... ok
> no obvious problems with ChimeraSlayer install ... ok
> clearcut is in path and version is supported ... ok
> cluster_jobs_fp is set to a valid path and is executable ... ok
> denoiser aligner is ready to use ... ok
> local qiime_config has no extra params ... ok
> maptplotlib version is supported ... ok
> mothur is in path and version is supported ... ok
> muscle is in path and version is supported ... ok
> numpy version is supported ... ok
> pynast version is supported ... ok
> pynast_template_alignment_blastdb, if set, is set to a valid path ... ok
> pynast_template_alignment, if set, is set to a valid path ... ok
> python_exe_fp is set to a working python env ... ok
> python is in path and version is supported ... ok
> qiime_scripts_dir, if set, is set to a valid path ... ok
> raxmlHPC is in path and version is supported ... ok
> temp_dir, if set, is set to a valid path ... ok
> template_alignment_lanemask, if set, is set to a valid path ... ok
> uclust is in path and version is supported ... ok
> working_dir, if set, is set to a valid path ... ok
>
> ----------------------------------------------------------------------
> Ran 28 tests in 0.621s
>
> OK
> (QIIME)rwt017 at topcat:~$

I could adapt your deb package to work like this, but I want to get my 
source version working properly first. In particular, I want all the 
unit tests from the QIIME source distribution tree to run successfully.

Making progress on that objective, but not quite got there yet ;-)

HTH,

   Tony.
-- 
Dr. A.J.Travis, University of Aberdeen, Rowett Institute of Nutrition
and Health, Greenburn Road, Bucksburn, Aberdeen AB21 9SB, Scotland, UK
tel +44(0)1224 712751, fax +44(0)1224 716687, http://www.rowett.ac.uk
mailto:a.travis at abdn.ac.uk, http://bioinformatics.rri.sari.ac.uk


From tbooth at ceh.ac.uk  Fri Nov  4 08:43:07 2011
From: tbooth at ceh.ac.uk (Tim Booth)
Date: Fri, 04 Nov 2011 12:43:07 +0000
Subject: [Bio-linux-dev] Qiime update
In-Reply-To: <4EB2F381.8050209@abdn.ac.uk>
References: <1320343690.6631.55961.camel@barsukas.nwl.ac.uk>
	<4EB2F381.8050209@abdn.ac.uk>
Message-ID: <1320410587.6631.58196.camel@barsukas.nwl.ac.uk>

Hi Tony,

Good to hear you are still working on Qiime.  Due to some build issues
between Ubuntu LTS and Debian Sid I'm comitting to a fork in SVN just
now, but if you want to do any work on the source package it's all here:

svn://svn.debian.org/svn/debian-med/trunk/packages/qiime/branches/lucid/debian

Or if you just have tested fixes for individual issues just let me know
and I'll fold them in.

For Qiime to really be useful in regular Debian it needs to not use
uClust by default.  Other clustering options are available but I don't
know what the implications of using these are and I've not had time to
investigate yet.

Cheers,

TIM

On Thu, 2011-11-03 at 20:03 +0000, Tony Travis wrote:
> On 03/11/11 18:08, Tim Booth wrote:
> > Hi All,
> >
> > Following on from the previous discussion, I have uploaded a new qiime
> > and bio-linux-qiime package, along with a fix to chimeraslayer.  This
> > should address the issues that Mike reported and also clear up some but
> > not all of the problems reported by "qiime print_qiime_config -t".
> 
> Hi, Tim.
> 
> I've decided to uninstall your qiime package and install QIIME from 
> source. I'm using this wrapper to create a custom QIIME environment 
> under Bio-Linux 6, without breaking anything else:
> 
> --- cut here ---
> # @(#)qiime.sh  2011-10-05  A.J.Travis
> 
> #
> # Qiime environment
> #
> 
> # Python
> export QIIME=/usr/local/qiime
> export QIIME_CONFIG_FP=$QIIME/etc/qiime_config
> export PATH=$QIIME/bin:$PATH
> export PYTHONPATH=$QIIME/lib/python2.6/site-packages:$PYTHONPATH
> 
> # NCBI BLAST 2.2.22
> export BLAST_HOME=/usr/local/blast-2.2.22
> export BLASTMAT=$BLAST_HOME/data
> export PATH=$BLAST_HOME/bin:$PATH
> 
> # AmpliconNoise
> export AMPLICON_NOISE=$QIIME/AmpliconNoiseV1.24
> export PATH=$AMPLICON_NOISE/Scripts:$AMPLICON_NOISE/bin:$PATH
> export PYRO_LOOKUP_FILE=$AMPLICON_NOISE/Data/LookUp_E123.dat
> export SEQ_LOOKUP_FILE=$AMPLICON_NOISE/Data/Tran.dat
> 
> # ChimeraSlayer
> export PATH=/usr/share/ChimeraSlayer:$PATH
> 
> # QIIME shell
> export debian_chroot=QIIME
> exec /bin/bash
> --- cut here ---
> 
> > rwt017 at topcat:~$ qiime
> > (QIIME)rwt017 at topcat:~$ print_qiime_config.py -t
> >
> > System information
> > ==================
> >          Platform:	linux2
> >    Python version:	2.6.5 (r265:79063, Apr 16 2010, 13:57:41)  [GCC 4.4.3]
> > Python executable:	/usr/bin/python
> >
> > Dependency versions
> > ===================
> >                      PyCogent version:	1.5.1
> >                         NumPy version:	1.3.0
> >                    matplotlib version:	0.98.5.3
> >                 QIIME library version:	1.3.0
> >                  QIIME script version:	1.3.0
> >         PyNAST version (if installed):	1.1
> > RDP Classifier version (if installed):	rdp_classifier-2.0.jar
> >
> > QIIME config values
> > ===================
> >                      blastmat_dir:	None
> >       topiaryexplorer_project_dir:	None
> >      pynast_template_alignment_fp:	None
> >                   cluster_jobs_fp:	/usr/local/qiime/bin/start_parallel_jobs.py
> > pynast_template_alignment_blastdb:	None
> >                      torque_queue:	friendlyq
> >    template_alignment_lanemask_fp:	None
> >                     jobs_to_start:	1
> >                 cloud_environment:	False
> >                 qiime_scripts_dir:	/usr/local/qiime/bin
> >             denoiser_min_per_core:	50
> >                       working_dir:	None
> >                     python_exe_fp:	python
> >                          temp_dir:	None
> >                       blastall_fp:	blastall
> >                  seconds_to_sleep:	60
> >
> >
> > running checks:
> >
> > FastTree is in path and version is supported ... ok
> > INFERNAL is in path and version is supported ... ok
> > AmpliconNoise install looks sane. ... ok
> > blast is in path and version is supported ... ok
> > blastall_fp is set to a valid path ... ok
> > blastmat_dir is set to a valid path. ... ok
> > cdbtools is in path and version is supported ... ok
> > cd-hit is in path and version is supported ... ok
> > no obvious problems with ChimeraSlayer install ... ok
> > clearcut is in path and version is supported ... ok
> > cluster_jobs_fp is set to a valid path and is executable ... ok
> > denoiser aligner is ready to use ... ok
> > local qiime_config has no extra params ... ok
> > maptplotlib version is supported ... ok
> > mothur is in path and version is supported ... ok
> > muscle is in path and version is supported ... ok
> > numpy version is supported ... ok
> > pynast version is supported ... ok
> > pynast_template_alignment_blastdb, if set, is set to a valid path ... ok
> > pynast_template_alignment, if set, is set to a valid path ... ok
> > python_exe_fp is set to a working python env ... ok
> > python is in path and version is supported ... ok
> > qiime_scripts_dir, if set, is set to a valid path ... ok
> > raxmlHPC is in path and version is supported ... ok
> > temp_dir, if set, is set to a valid path ... ok
> > template_alignment_lanemask, if set, is set to a valid path ... ok
> > uclust is in path and version is supported ... ok
> > working_dir, if set, is set to a valid path ... ok
> >
> > ----------------------------------------------------------------------
> > Ran 28 tests in 0.621s
> >
> > OK
> > (QIIME)rwt017 at topcat:~$
> 
> I could adapt your deb package to work like this, but I want to get my 
> source version working properly first. In particular, I want all the 
> unit tests from the QIIME source distribution tree to run successfully.
> 
> Making progress on that objective, but not quite got there yet ;-)
> 
> HTH,
> 
>    Tony.

-- 
Tim Booth <tbooth at ceh.ac.uk>
NERC Environmental Bioinformatics Centre 

Centre for Ecology and Hydrology
Maclean Bldg, Benson Lane
Crowmarsh Gifford
Wallingford, England
OX10 8BB 

http://nebc.nerc.ac.uk
+44 1491 69 2705

-- 
This message (and any attachments) is for the recipient only. NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless
it is exempt from release under the Act. Any material supplied to
NERC may be stored in an electronic records management system.



From mikeyj.cox at gmail.com  Wed Nov  9 09:25:40 2011
From: mikeyj.cox at gmail.com (Mike Cox)
Date: Wed, 9 Nov 2011 14:25:40 +0000
Subject: [Bio-linux-dev] Qiime update
In-Reply-To: <1320343690.6631.55961.camel@barsukas.nwl.ac.uk>
References: <1320343690.6631.55961.camel@barsukas.nwl.ac.uk>
Message-ID: <CAKaWfX3_hoNy93xj=BPmAB1zt4eW=fhQsU5CO9AMBL7THipzRw@mail.gmail.com>

Hi Tim,

PyNAST is now behaving and running for me, (I've just checked by running
core_qiime_analyses.py, so anything not run by this I can't comment on).  I
am getting an error message that Matplotlib is an untested version when it
hits the beta diversity section, specifically when calculating the PCoAs,
but the resulting PCoAs seem to look OK, so it doesn't look like the
version in actual use has broken things.

Thanks!

Mike

On Thu, Nov 3, 2011 at 18:08, Tim Booth <tbooth at ceh.ac.uk> wrote:

> Hi All,
>
> Following on from the previous discussion, I have uploaded a new qiime
> and bio-linux-qiime package, along with a fix to chimeraslayer.  This
> should address the issues that Mike reported and also clear up some but
> not all of the problems reported by "qiime print_qiime_config -t".
> Specifically I now get:
>
> FastTree is in path and version is supported ... ok
> INFERNAL is in path and version is supported ... ok
> AmpliconNoise install looks sane. ... ok
> blast is in path and version is supported ... FAIL
> blastall_fp is set to a valid path ... ok
> blastmat_dir is set to a valid path. ... ok
> cdbtools is in path and version is supported ... ok
> cd-hit is in path and version is supported ... ok
> no obvious problems with ChimeraSlayer install ... ok
> clearcut is in path and version is supported ... FAIL
> cluster_jobs_fp is set to a valid path and is executable ... ok
> denoiser aligner is ready to use ... ok
> local qiime_config has no extra params ... ok
> maptplotlib version is supported ... FAIL
> mothur is in path and version is supported ... ERROR
> muscle is in path and version is supported ... FAIL
> numpy version is supported ... ok
> pynast version is supported ... ok
> pynast_template_alignment_blastdb, if set, is set to a valid path ... ok
> pynast_template_alignment, if set, is set to a valid path ... ok
> python_exe_fp is set to a working python env ... ok
> python is in path and version is supported ... ok
> qiime_scripts_dir, if set, is set to a valid path ... ok
> raxmlHPC is in path and version is supported ... FAIL
> temp_dir, if set, is set to a valid path ... ok
> template_alignment_lanemask, if set, is set to a valid path ... ok
> uclust is in path and version is supported ... ok
> working_dir, if set, is set to a valid path ... ok
>
> If you are still seeing other failures, please check you have upgraded
> the packages:
>
> % dpkg -s qiime bio-linux-qiime chimeraslayer | grep ^Version
> Version: 1.3.0-0ubuntu6
> Version: 1:1.3.0bl-6
> Version: 20101212-1ubuntu5
>
> If there are still issues after checking this then let me know.
>
> Of the remaining issues, a couple more things I should be able to fix,
> but some of the others seem to be due to us having newer versions,
> rather than older versions, of dependencies, and I'm not going to make a
> downgraded package unless someone tells me it really is needed (ie. a
> real error, not just a test warning).
>
> Cheers,
>
> TIM
>
> --
> Tim Booth <tbooth at ceh.ac.uk>
> NERC Environmental Bioinformatics Centre
>
> Centre for Ecology and Hydrology
> Maclean Bldg, Benson Lane
> Crowmarsh Gifford
> Wallingford, England
> OX10 8BB
>
> http://nebc.nerc.ac.uk
> +44 1491 69 2705
>
> --
> This message (and any attachments) is for the recipient only. NERC
> is subject to the Freedom of Information Act 2000 and the contents
> of this email and any reply you make may be disclosed by NERC unless
> it is exempt from release under the Act. Any material supplied to
> NERC may be stored in an electronic records management system.
>
> _______________________________________________
> Bio-Linux-dev mailing list
> Bio-Linux-dev at nebclists.nerc.ac.uk
> http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.bioinformatics.org/pipermail/bio-linux-devel/attachments/20111109/24ebd55b/attachment.html>