[Bio-Linux] Problem with adding data to GALAXY using FTP upload via transfer directory
Tim Booth
tbooth at ceh.ac.uk
Fri Jan 16 09:24:14 EST 2015
Hi Ian,
It's a bit subtle -
galaxy is the group for regular users who can use Galaxy
eg. manager is in group galaxy
Debian-galaxy is a group just for the GALAXY server process, which runs
under a system user account named Debian-galaxy. It allows me to say
that the manager at localhost directory is visible and writeable for the
manager user and also the GALAXY server but closed to everyone else. No
other users should be added to the Debian-galaxy group and furthermore
the Debian-galaxy user should not be a member of the regular galaxy
group.
Clear as mud?
If you still think there's a typo in my on-line instructions then let me
know which bit you are looking at and I'll try to fix/clarify it.
TIM
On Fri, 2015-01-16 at 11:59 +0000, Ian Donaldson wrote:
> Thanks for your reply Tim.
>
> I deleted the directory and restarted GALAXY, the manager directory was recreated with the permissions you mention:
> drwxrwx--- 2 manager Debian-galaxy 4096 Jan 16 11:51 manager at localhost
>
> I can now input files!
>
> I am a bit confused about the groups though. 'Debian-galaxy' already exists, but 'galaxy' is mentioned in the instructions for setting up the FTP.
>
> Thanks again,
> Ian
>
> ________________________________________
> From: Tim Booth [tbooth at ceh.ac.uk]
> Sent: 16 January 2015 11:12
> To: bio-linux at nebclists.nerc.ac.uk
> Subject: Re: [Bio-Linux] Problem with adding data to GALAXY using FTP upload via transfer directory
>
> Hi Ian,
>
> The correct permission is:
>
> drwxrwx--- 2 user2 Debian-galaxy 4096 Feb 26 2014 user2 at localhost
> (or in your case, manager rather than user2)
>
> And anything else is going to cause problems. Galaxy on Bio-Linux is
> supposed to sort out the permissions when it starts up, but I had a
> couple of bugs in the old version of that startup script that could
> leave directories stuck with the wrong permissions. Would you mind
> trying deleting the manager at localhost directory entirely then restarting
> Galaxy. What permissions does it get recreated with?
>
> Cheers,
>
> TIM
>
> On Fri, 2015-01-16 at 10:45 +0000, Ian Donaldson wrote:
> > Hi,
> >
> > I have followed the advice given in
> > http://environmentalomics.org/bio-linux-galaxy/ and I have
> > successfully set up the galaxy-server-apache-proxy. I have put a sam
> > file (ex1.sam) in the transfer directory for manager at localhost and can
> > see the file in the'Files uploaded via FTP' section. However, when I
> > try to select and upload the file I get errors:
> >
> > From the debug message:
> > Traceback (most recent call last):
> > File "/usr/lib/galaxy-server/tools/data_source/upload.py", line 394,
> > in <module>
> > __main__()
> > File "/usr/lib/galaxy-server/tools/data_source/upload.py", line 383,
> > in __main__
> > add_file( dataset, registry, json_file, output_path )
> > File "/usr/lib/galaxy-server/tools/data_source/upload.py", line 278,
> > in add_file
> > line_count, converted_path = sniff.convert_newlines( dataset.path,
> > in_place=in_place )
> > File "/usr/lib/galaxy-server/lib/galaxy/datatypes/sniff.py", line
> > 116, in convert_newlines
> > shutil.move( temp_name, fname )
> > File "/usr/lib/python2.7/shutil.py", line 302, in move
> > copy2(src, real_dst)
> > File "/usr/lib/python2.7/shutil.py", line 130, in copy2
> > copyfile(src, dst)
> > File "/usr/lib/python2.7/shutil.py", line 83, in copyfile
> > with open(dst, 'wb') as fdst:
> > IOError: [Errno 13] Permission denied:
> > u'/var/lib/galaxy-server/transfer/manager at localhost/ex1.sam'
> >
> >
> > From the server.log:
> > galaxy.tools.actions.upload_common INFO 2015-01-16 10:35:07,359 tool
> > upload1 created job id 2
> > 127.0.0.1 - - [16/Jan/2015:10:35:07 +0000]
> > "POST /galaxy/tool_runner/index HTTP/1.1" 200 -
> > "http://localhost/galaxy/tool_runner?tool_id=upload1" "Mozilla/5.0
> > (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
> > galaxy.jobs DEBUG 2015-01-16 10:35:07,485 (2) Working directory for
> > job is: /usr/lib/galaxy-server/database/job_working_directory/000/2
> > galaxy.jobs.handler DEBUG 2015-01-16 10:35:07,496 (2) Dispatching to
> > local runner
> > 127.0.0.1 - - [16/Jan/2015:10:35:07 +0000]
> > "GET /galaxy/api/histories/1cd8e2f6b131e891/contents HTTP/1.1" 200 -
> > "http://localhost/galaxy/root/index" "Mozilla/5.0 (X11; Ubuntu; Linux
> > x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
> > galaxy.jobs DEBUG 2015-01-16 10:35:07,599 (2) Persisting job
> > destination (destination id: local:///)
> > galaxy.jobs.handler INFO 2015-01-16 10:35:07,622 (2) Job dispatched
> > galaxy.tools.deps DEBUG 2015-01-16 10:35:07,770 Building dependency
> > shell command for dependency 'samtools'
> > galaxy.tools.deps WARNING 2015-01-16 10:35:07,770 Failed to resolve
> > dependency on 'samtools', ignoring
> > galaxy.jobs.runners DEBUG 2015-01-16 10:35:07,833 (2) command is:
> > python /usr/lib/galaxy-server/tools/data_source/upload.py /usr/lib/galaxy-server /usr/lib/galaxy-server/database/tmp/tmp7LWfBs /usr/lib/galaxy-server/database/tmp/tmp9nLfry 2:/usr/lib/galaxy-server/database/job_working_directory/000/2/dataset_2_files:/usr/lib/galaxy-server/database/files/000/dataset_2.dat
> > galaxy.jobs.runners.local DEBUG 2015-01-16 10:35:07,834 (2) executing
> > job
> > script: /usr/lib/galaxy-server/database/job_working_directory/000/2/galaxy_2.sh
> > galaxy.jobs DEBUG 2015-01-16 10:35:07,862 (2) Persisting job
> > destination (destination id: local:///)
> > galaxy.jobs.runners.local DEBUG 2015-01-16 10:35:08,617 execution
> > finished: /usr/lib/galaxy-server/database/job_working_directory/000/2/galaxy_2.sh
> > galaxy.tools.deps DEBUG 2015-01-16 10:35:08,684 Building dependency
> > shell command for dependency 'samtools'
> > galaxy.tools.deps WARNING 2015-01-16 10:35:08,684 Failed to resolve
> > dependency on 'samtools', ignoring
> > galaxy.jobs.runners DEBUG 2015-01-16 10:35:08,684 executing external
> > set_meta script for job
> > 2: /usr/lib/galaxy-server/set_metadata.sh ./database/files /usr/lib/galaxy-server/database/job_working_directory/000/2 . /tmp/universe_wsgi_jymQ.ini /usr/lib/galaxy-server/database/tmp/tmp7LWfBs /usr/lib/galaxy-server/database/job_working_directory/000/2/galaxy.json /usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_in_HistoryDatasetAssociation_2_eqLSN4,/usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_kwds_HistoryDatasetAssociation_2_Y8KWCA,/usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_out_HistoryDatasetAssociation_2_e8sRbx,/usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_results_HistoryDatasetAssociation_2_s439LW,,/usr/lib/galaxy-server/database/job_working_directory/000/2/metadata_override_HistoryDatasetAssociation_2_jlOhII
> > galaxy.jobs.runners DEBUG 2015-01-16 10:35:09,731 execution of
> > external set_meta for job 2 finished
> > galaxy.jobs DEBUG 2015-01-16 10:35:09,789 setting dataset state to
> > ERROR
> > galaxy.jobs DEBUG 2015-01-16 10:35:09,874 job 2 ended
> > galaxy.datatypes.metadata DEBUG 2015-01-16 10:35:09,874 Cleaning up
> > external metadata files
> > 127.0.0.1 - - [16/Jan/2015:10:35:11 +0000]
> > "GET /galaxy/api/histories/1cd8e2f6b131e891/contents HTTP/1.1" 200 -
> > "http://localhost/galaxy/root/index" "Mozilla/5.0 (X11; Ubuntu; Linux
> > x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
> > 127.0.0.1 - - [16/Jan/2015:10:35:11 +0000]
> > "GET /galaxy/api/histories/1cd8e2f6b131e891 HTTP/1.1" 200 -
> > "http://localhost/galaxy/root/index" "Mozilla/5.0 (X11; Ubuntu; Linux
> > x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
> >
> >
> > I noticed permission being mentioned. The status of the transfer
> > directory is:
> > drwxr-xr-x 2 root root 4096 Jan 16 10:12 manager at localhost
> >
> > I change this to:
> > drwxr-xr-x 2 manager manager 4096 Jan 16 10:12 manager at localhost
> >
> > But the same thing happened.
> >
> > Has anyone seen this problem before and have and guidance?
> >
> > Thank you,
> > Ian
> >
> >
>
> --
> Tim Booth <tbooth at ceh.ac.uk>
> NERC Environmental Bioinformatics Centre
>
> Centre for Ecology and Hydrology
> Maclean Bldg, Benson Lane
> Crowmarsh Gifford
> Wallingford, England
> OX10 8BB
>
> http://environmentalomics.org/bio-linux
> +44 1491 69 2297
>
> _______________________________________________
> Bio-Linux mailing list
> Bio-Linux at nebclists.nerc.ac.uk
> http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux
> _______________________________________________
> Bio-Linux mailing list
> Bio-Linux at nebclists.nerc.ac.uk
> http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux
--
Tim Booth <tbooth at ceh.ac.uk>
NERC Environmental Bioinformatics Centre
Centre for Ecology and Hydrology
Maclean Bldg, Benson Lane
Crowmarsh Gifford
Wallingford, England
OX10 8BB
http://environmentalomics.org/bio-linux
+44 1491 69 2297
More information about the Bio-linux-list
mailing list