[Bio-linux-dev] Upgrade to Bio-Linux 7 (at last)
Tim Booth
tbooth at ceh.ac.uk
Thu Nov 15 11:16:14 EST 2012
Hi Mike,
QIIME uses all sorts of internal guesswork to try and find the files it
needs, so this is basically the same problem as with king.jar. In order
to try and simplify things I've reverted my previous fix and added a
different fix that will hopefully sort all these related problems at
once, but then it might break something else. I'll try and get it
tested before I send it out to you.
Cheers,
TIM
On Thu, 2012-11-15 at 15:08 +0000, Cox, Michael J wrote:
> Hi Tim,
>
>
> That's excellent, thanks. A user found another issue - looks like
> there are still some dependencies lurking around that aren't
> available. This time it's with check_id_map.py (which hardly does
> anything - how QIIME can make this complicated I've no idea).
>
>
> The error is:
>
>
> qiime > check_id_map.py -m aran.r2.map.txt -o mapping_output
>
>
>
> Traceback (most recent call last):
> File "/usr/lib/qiime/bin/check_id_map.py", line 149, in <module>
> main()
> File "/usr/lib/qiime/bin/check_id_map.py", line 145, in main
> disable_primer_check, added_demultiplex_field)
> File "/usr/lib/python2.7/dist-packages/qiime/check_id_map.py", line
> 112, in check_mapping_file
> 'js/overlib.js'), join(output_dir,'overlib.js'))
> File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
> with open(src, 'rb') as fsrc:
> IOError: [Errno 2] No such file or directory:
> '/usr/lib/python2.7/dist-packages/qiime/support_files/js/overlib.js'
>
> Thanks
>
>
> Mike
>
>
> PS
>
>
> Locally the bioinformatics support service here have written a wrapper
> that stops ampliconnoise.py from failing (or at least, starts it going
> again from where it left off) when it hits a sample with no good
> reads. I've asked whether they're happy for me to share it with you
> in case it's of use as you're having to do a lot of patching anyway in
> QIIME. Will pass it on if they OK.
>
>
> On 14 Nov 2012, at 15:43, Tim Booth <tbooth at ceh.ac.uk> wrote:
>
> > Hi Mike,
> >
> > Thanks for reporting the problems with QIIME. I've patched the code
> > to
> > make it work with the newer RDP classifier and I've also fixed the
> > issue
> > with KING not working.
> >
> > I've also applied the patch that I saw on the QIIME blog to fix the
> > problem with compare_alpha_diversity reporting the wrong
> > t-statistic.
> >
> > The new packages are now on the package server for BL7.
> >
> > Cheers,
> >
> > TIM
> >
> > On Wed, 2012-11-07 at 16:12 +0000, Michael Cox wrote:
> > > Hi Tim,
> > >
> > >
> > > it looks like something else is going on too - King is now not
> > > working
> > > within the pipeline to generate the kinemage files for PCoA
> > > analysis.
> > > Here's the error:
> > >
> > >
> > > Qiime> beta_diversity_through_plots.py -i otus/otu_table.biom -m
> > > aran.r1.map.txt -o wf_bdiv_even692/ -t otus/rep_set.tre -e 692
> > >
> > >
> > > Traceback (most recent call last):
> > > File "/usr/lib/qiime/bin/beta_diversity_through_plots.py", line
> > > 166,
> > > in <module>
> > > main()
> > > File "/usr/lib/qiime/bin/beta_diversity_through_plots.py", line
> > > 163,
> > > in main
> > > status_update_callback=status_update_callback)
> > > File "/usr/lib/python2.7/dist-packages/qiime/workflow.py", line
> > > 867,
> > > in run_beta_diversity_through_plots
> > > close_logger_on_success=close_logger_on_success)
> > > File "/usr/lib/python2.7/dist-packages/qiime/workflow.py", line
> > > 135,
> > > in call_commands_serially
> > > raise WorkflowError, msg
> > > qiime.workflow.WorkflowError:
> > >
> > > *** ERROR RAISED DURING STEP: Make 3D plots (continuous coloring,
> > > weighted_unifrac)
> > > Command run was:
> > > python /usr/lib/qiime/bin//make_3d_plots.py -p
> > > wf_bdiv_even692//prefs.txt -i
> > > wf_bdiv_even692//weighted_unifrac_pc.txt
> > > -o wf_bdiv_even692//weighted_unifrac_3d_continuous/ -m
> > > aran.r1.map.txt
> > > Command returned exit status: 1
> > > Stdout:
> > >
> > > Stderr
> > > Traceback (most recent call last):
> > > File "/usr/lib/qiime/bin//make_3d_plots.py", line 443, in <module>
> > > main()
> > > File "/usr/lib/qiime/bin//make_3d_plots.py", line 419, in main
> > > shutil.copyfile(os.path.join(jar_path,'king.jar'),
> > > os.path.join(jar_dir_path,'king.jar'))
> > > File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
> > > with open(src, 'rb') as fsrc:
> > > IOError: [Errno 2] No such file or directory:
> > > '/usr/lib/python2.7/dist-packages/qiime/support_files/jar/king.jar'
> > >
> > >
> > > There is no "support_files" directory in that directory. I have
> > > installed King myself independently in /usr/local/bioinf/king/king
> > > and
> > > this available as king from the command line. I guess I could
> > > create
> > > a symlink to this, but it would break on update? For the moment
> > > I'm
> > > going to get this user to load his distance matrices into R as
> > > they
> > > will have been generated and I'm evil ;)
> > >
> > >
> > > Cheers
> > >
> > >
> > > Mike
> > >
> > > On 7 Nov 2012, at 10:27, Tim Booth <tbooth at ceh.ac.uk> wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > Well, if I'd actually tested the thing I guess I'd have found
> > > > this
> > > > out
> > > > for myself. I just couldn't quite believe that Qiime would find
> > > > the
> > > > JAR
> > > > that you've told it to run but then refuse to run it because it
> > > > doesn't
> > > > like the name.
> > > >
> > > > The discussion on the forum makes it quite clear that they added
> > > > this
> > > > check because they wanted to stop people using the older version
> > > > of
> > > > the
> > > > classifier. It is highly likely that the new version works
> > > > fine.
> > > > Maybe
> > > > the 2.2 RDP version added some option that Qiime relies on, in
> > > > which
> > > > case it's unlikely this would be removed in a later version.
> > > >
> > > > My "normalised" comment means that the rdp-classifier package
> > > > now
> > > > meets
> > > > proper packaging standards. It was a bit of a fudge before, and
> > > > I
> > > > couldn't put all the configuration bits in one place. It also
> > > > means
> > > > that the version of the package given by dpkg gives you the
> > > > version
> > > > of
> > > > the classifier - ie. 2.5.
> > > >
> > > > My inclination is to do a bit more testing and then if 2.5 seems
> > > > to
> > > > work
> > > > I'll patch Qiime to accept it. I'll also add a warning when
> > > > Qiime
> > > > is
> > > > started that the classifier being used is not the default. Let
> > > > me
> > > > know
> > > > if you have any problems with the renamed 2.5 classifier or if
> > > > you
> > > > think
> > > > you need to switch back to 2.2 in order to ensure a "standard"
> > > > analysis.
> > > >
> > > > Cheers,
> > > >
> > > > TIM
> > > >
> > > > > I did say below! :P
> > > > >
> > > > > assign_taxonomy.py does a crappy name check
> > > > > https://groups.google.com/d/msg/qiime-forum/o7CaXnfS1J0/DgsFm-Q-s6YJ
> > > > > hence the problem
> > > > >
> > > > > Apparently they only support 2.2 as well, do you know whether
> > > > > Biolinux
> > > > > has the most recent version of RDP classifier installed? I
> > > > > tried
> > > > > to
> > > > > determine the version number, but helpfully it's not included
> > > > > in
> > > > > the
> > > > > help output :)
> > > > >
> > > > > I am fairly sure other versions aren't supported, and after
> > > > > changing
> > > > > the name to rdp_classifier-2.2.jar
> > > > >
> > > > > and the path to
> > > > >
> > > > > # This was previously set in bio-linux-rdp-classifier but can
> > > > > now
> > > > > live
> > > > > here
> > > > > # as the rdp-classifier package has been normalised.
> > > > > export RDP_JAR_PATH=
> > > > > ${RDP_JAR_PATH:-/usr/share/rdp-classifier/rdp_classifier-2.2.jar}
> > > > >
> > > > > It does seem to run and produce a sensible result, but I've
> > > > > not
> > > > > checked within the context of the whole pipeline yet. Will
> > > > > let
> > > > > you
> > > > > know if I hit a problem with it and have to install 2.2. What
> > > > > does
> > > > > the normalised comment refer to?
> > > > >
> > > > > Thanks! QIIME is such fun isn't it? Is Galaxy more or less
> > > > > fun?
> > > > >
> > > >
> > > > --
> > > > This message (and any attachments) is for the recipient only.
> > > > NERC
> > > > is subject to the Freedom of Information Act 2000 and the
> > > > contents
> > > > of this email and any reply you make may be disclosed by NERC
> > > > unless
> > > > it is exempt from release under the Act. Any material supplied
> > > > to
> > > > NERC may be stored in an electronic records management system.
> > >
> >
> > --
> > Tim Booth <tbooth at ceh.ac.uk>
> > NERC Environmental Bioinformatics Centre
> >
> > Centre for Ecology and Hydrology
> > Maclean Bldg, Benson Lane
> > Crowmarsh Gifford
> > Wallingford, England
> > OX10 8BB
> >
> > http://nebc.nerc.ac.uk
> > +44 1491 69 2705
>
--
Tim Booth <tbooth at ceh.ac.uk>
NERC Environmental Bioinformatics Centre
Centre for Ecology and Hydrology
Maclean Bldg, Benson Lane
Crowmarsh Gifford
Wallingford, England
OX10 8BB
http://nebc.nerc.ac.uk
+44 1491 69 2705
More information about the Bio-linux-devel
mailing list