[Bio-linux-dev] Upgrade to Bio-Linux 7 (at last)

Thu Nov 15 10:08:10 EST 2012

Hi Tim,

That's excellent, thanks.  A user found another issue - looks like there are still some dependencies lurking around that aren't available. This time it's with check_id_map.py (which hardly does anything - how QIIME can make this complicated I've no idea).

The error is:

qiime > check_id_map.py -m aran.r2.map.txt -o mapping_output

Traceback (most recent call last):
  File "/usr/lib/qiime/bin/check_id_map.py", line 149, in <module>
    main()
  File "/usr/lib/qiime/bin/check_id_map.py", line 145, in main
    disable_primer_check, added_demultiplex_field)
  File "/usr/lib/python2.7/dist-packages/qiime/check_id_map.py", line 112, in check_mapping_file
    'js/overlib.js'), join(output_dir,'overlib.js'))
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/usr/lib/python2.7/dist-packages/qiime/support_files/js/overlib.js'

Thanks

Mike

PS

Locally the bioinformatics support service here have written a wrapper that stops ampliconnoise.py from failing (or at least, starts it going again from where it left off) when it hits a sample with no good reads.  I've asked whether they're happy for me to share it with you in case it's of use as you're having to do a lot of patching anyway in QIIME.  Will pass it on if they OK.

On 14 Nov 2012, at 15:43, Tim Booth <tbooth at ceh.ac.uk> wrote:

> Hi Mike,
> 
> Thanks for reporting the problems with QIIME.  I've patched the code to
> make it work with the newer RDP classifier and I've also fixed the issue
> with KING not working.
> 
> I've also applied the patch that I saw on the QIIME blog to fix the
> problem with compare_alpha_diversity reporting the wrong t-statistic.
> 
> The new packages are now on the package server for BL7.
> 
> Cheers,
> 
> TIM
> 
> On Wed, 2012-11-07 at 16:12 +0000, Michael Cox wrote:
>> Hi Tim,
>> 
>> 
>> it looks like something else is going on too - King is now not working
>> within the pipeline to generate the kinemage files for PCoA analysis.
>> Here's the error:
>> 
>> 
>> Qiime> beta_diversity_through_plots.py -i otus/otu_table.biom -m
>> aran.r1.map.txt -o wf_bdiv_even692/ -t otus/rep_set.tre -e 692 
>> 
>> 
>> Traceback (most recent call last):
>> File "/usr/lib/qiime/bin/beta_diversity_through_plots.py", line 166,
>> in <module>
>>   main()
>> File "/usr/lib/qiime/bin/beta_diversity_through_plots.py", line 163,
>> in main
>>   status_update_callback=status_update_callback)
>> File "/usr/lib/python2.7/dist-packages/qiime/workflow.py", line 867,
>> in run_beta_diversity_through_plots
>>   close_logger_on_success=close_logger_on_success)
>> File "/usr/lib/python2.7/dist-packages/qiime/workflow.py", line 135,
>> in call_commands_serially
>>   raise WorkflowError, msg
>> qiime.workflow.WorkflowError: 
>> 
>> *** ERROR RAISED DURING STEP: Make 3D plots (continuous coloring,
>> weighted_unifrac)
>> Command run was:
>> python /usr/lib/qiime/bin//make_3d_plots.py -p
>> wf_bdiv_even692//prefs.txt -i wf_bdiv_even692//weighted_unifrac_pc.txt
>> -o wf_bdiv_even692//weighted_unifrac_3d_continuous/ -m
>> aran.r1.map.txt 
>> Command returned exit status: 1
>> Stdout:
>> 
>> Stderr
>> Traceback (most recent call last):
>> File "/usr/lib/qiime/bin//make_3d_plots.py", line 443, in <module>
>>   main()
>> File "/usr/lib/qiime/bin//make_3d_plots.py", line 419, in main
>>   shutil.copyfile(os.path.join(jar_path,'king.jar'),
>> os.path.join(jar_dir_path,'king.jar'))
>> File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
>>   with open(src, 'rb') as fsrc:
>> IOError: [Errno 2] No such file or directory:
>> '/usr/lib/python2.7/dist-packages/qiime/support_files/jar/king.jar'
>> 
>> 
>> There is no "support_files" directory in that directory.  I have
>> installed King myself independently in /usr/local/bioinf/king/king and
>> this available as king from the command line.  I guess I could create
>> a symlink to this, but it would break on update?  For the moment I'm
>> going to get this user to load his distance matrices into R as they
>> will have been generated and I'm evil ;)
>> 
>> 
>> Cheers
>> 
>> 
>> Mike
>> 
>> On 7 Nov 2012, at 10:27, Tim Booth <tbooth at ceh.ac.uk> wrote:
>> 
>>> Hi Mike,
>>> 
>>> Well, if I'd actually tested the thing I guess I'd have found this
>>> out
>>> for myself.  I just couldn't quite believe that Qiime would find the
>>> JAR
>>> that you've told it to run but then refuse to run it because it
>>> doesn't
>>> like the name.
>>> 
>>> The discussion on the forum makes it quite clear that they added
>>> this
>>> check because they wanted to stop people using the older version of
>>> the
>>> classifier.  It is highly likely that the new version works fine.
>>> Maybe
>>> the 2.2 RDP version added some option that Qiime relies on, in which
>>> case it's unlikely this would be removed in a later version.
>>> 
>>> My "normalised" comment means that the rdp-classifier package now
>>> meets
>>> proper packaging standards.  It was a bit of a fudge before, and I
>>> couldn't put all the configuration bits in one place.  It also means
>>> that the version of the package given by dpkg gives you the version
>>> of
>>> the classifier - ie. 2.5.
>>> 
>>> My inclination is to do a bit more testing and then if 2.5 seems to
>>> work
>>> I'll patch Qiime to accept it.  I'll also add a warning when Qiime
>>> is
>>> started that the classifier being used is not the default.  Let me
>>> know
>>> if you have any problems with the renamed 2.5 classifier or if you
>>> think
>>> you need to switch back to 2.2 in order to ensure a "standard"
>>> analysis.
>>> 
>>> Cheers,
>>> 
>>> TIM
>>> 
>>>> I did say below!  :P
>>>> 
>>>> assign_taxonomy.py does a crappy name check
>>>> https://groups.google.com/d/msg/qiime-forum/o7CaXnfS1J0/DgsFm-Q-s6YJ
>>>> hence the problem
>>>> 
>>>> Apparently they only support 2.2 as well, do you know whether
>>>> Biolinux
>>>> has the most recent version of RDP classifier installed?  I tried
>>>> to
>>>> determine the version number, but helpfully it's not included in
>>>> the
>>>> help output :) 
>>>> 
>>>> I am fairly sure other versions aren't supported, and after
>>>> changing
>>>> the name to rdp_classifier-2.2.jar
>>>> 
>>>> and the path to
>>>> 
>>>> # This was previously set in bio-linux-rdp-classifier but can now
>>>> live
>>>> here
>>>> # as the rdp-classifier package has been normalised.
>>>> export RDP_JAR_PATH=
>>>> ${RDP_JAR_PATH:-/usr/share/rdp-classifier/rdp_classifier-2.2.jar}
>>>> 
>>>> It does seem to run and produce a sensible result, but I've not
>>>> checked within the context of the whole pipeline yet.  Will let
>>>> you
>>>> know if I hit a problem with it and have to install 2.2.  What
>>>> does
>>>> the normalised comment refer to?
>>>> 
>>>> Thanks!  QIIME is such fun isn't it?  Is Galaxy more or less fun?
>>>> 
>>> 
>>> -- 
>>> This message (and any attachments) is for the recipient only. NERC
>>> is subject to the Freedom of Information Act 2000 and the contents
>>> of this email and any reply you make may be disclosed by NERC unless
>>> it is exempt from release under the Act. Any material supplied to
>>> NERC may be stored in an electronic records management system.
>> 
> 
> -- 
> Tim Booth <tbooth at ceh.ac.uk>
> NERC Environmental Bioinformatics Centre 
> 
> Centre for Ecology and Hydrology
> Maclean Bldg, Benson Lane
> Crowmarsh Gifford
> Wallingford, England
> OX10 8BB 
> 
> http://nebc.nerc.ac.uk
> +44 1491 69 2705

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.bioinformatics.org/pipermail/bio-linux-devel/attachments/20121115/97ad2d95/attachment.html>