From tony.travis at ed.ac.uk Mon Jun 3 10:28:30 2013 From: tony.travis at ed.ac.uk (Tony Travis) Date: Mon, 03 Jun 2013 15:28:30 +0100 Subject: [Bio-linux-dev] Bio-Linux upgrade problems Message-ID: <51ACA80E.5060107@ed.ac.uk> Hi, Tim. I've had problems upgrading Bio-Linux servers that use autofs to mount their home directories. In this situation, /home is an automount point, not a folder on which a filesystem is mounted. host1:/export/home -> /home/host1 host2:/export/home -> /home/host2 ... In particular, you can't create or remove a folder in an automounted /home because any entries in the directory representing an automount point are actually keys from the automount map which are themselves mount-points for the automounted filesystems. This is a problem because Bio-Linux updates sometimes try to create or destroy /home/db - As a work-around, I faked an entry for 'host' db: :/export/db -> /home/db This worked fine, until your scripts tried to delete /home/db... I've also encountered a lot of problems recently using proprietary drivers for Ubuntu 12.04 LTS on my laptop and other desktops/servers until I discovered that Canonical have decided to upgrade the kernel from 3.2 to 3.5 in 12.04.2 LTS: https://wiki.ubuntu.com/Kernel/LTSEnablementStack Are you planning to do a point-release based on Ubuntu 12.04.2 LTS? I've been manually upgrading my systems to use the LTSEnablementStack and I would be happy to help with testing a version based on 12.04.2. Bye, Tony. From tbooth at ceh.ac.uk Mon Jun 3 10:59:41 2013 From: tbooth at ceh.ac.uk (Tim Booth) Date: Mon, 3 Jun 2013 15:59:41 +0100 Subject: [Bio-linux-dev] Bio-Linux upgrade problems In-Reply-To: <51ACA80E.5060107@ed.ac.uk> References: <51ACA80E.5060107@ed.ac.uk> Message-ID: <1370271581.6820.286.camel@balisaur> Hi Tony, The good news is that /home/db is dead, or will be once you get the upgrade to work, as can be verified on a BL7 machine by running: tbooth at balisaur[tbooth]dpkg -S /home base-files: /home (I think there may be a script or two that looks for legacy BLAST databases in there, but if the directory is absent these should just silently continue) But if you have already set up automount on /home then you have a problem because APT will try to remove the empty /home/db directory and get eggy when it fails. My suggestion would be to ensure the directory isn't left empty until after the upgrade: # touch /home/db/fakefile # bl_do_upgrade_to7.sh ... # dpkg -S /home base-files: /home # rm /home/db/fakefile # delete_host db ...or whatever to remove your original work-around Does that make sense? I expect this problem only impacts a couple of people, and is a one-off, so I'd prefer to avoid the effort of adding the fix to the script if that's OK. Regarding the 3.5 series kernel, this is already the default in the latest Bio-Linux ISO, but it is not the default if you upgrade from BL6 - you get 3.2 the same as other Ubuntu upgraders. My logic is that the new kernel is most useful for those with new hardware and thus not relevant to most upgraders. (I'm using 3.2 until I see a good reason not to - it never seems to crash on my laptop and everything works) I'm inclined to think that proprietary drivers will cause aggro whatever I do, but if you think there is something obvious I need to change to help out with these issues then let me know. Cheers, TIM On Mon, 2013-06-03 at 15:28 +0100, Tony Travis wrote: > Hi, Tim. > > I've had problems upgrading Bio-Linux servers that use autofs to mount > their home directories. In this situation, /home is an automount point, > not a folder on which a filesystem is mounted. > > host1:/export/home -> /home/host1 > host2:/export/home -> /home/host2 > ... > > In particular, you can't create or remove a folder in an automounted > /home because any entries in the directory representing an automount > point are actually keys from the automount map which are themselves > mount-points for the automounted filesystems. > > This is a problem because Bio-Linux updates sometimes try to create or > destroy /home/db - As a work-around, I faked an entry for 'host' db: > > :/export/db -> /home/db > > This worked fine, until your scripts tried to delete /home/db... > > I've also encountered a lot of problems recently using proprietary > drivers for Ubuntu 12.04 LTS on my laptop and other desktops/servers > until I discovered that Canonical have decided to upgrade the kernel > from 3.2 to 3.5 in 12.04.2 LTS: > > https://wiki.ubuntu.com/Kernel/LTSEnablementStack > > Are you planning to do a point-release based on Ubuntu 12.04.2 LTS? > > I've been manually upgrading my systems to use the LTSEnablementStack > and I would be happy to help with testing a version based on 12.04.2. > > Bye, > > Tony. > > _______________________________________________ > Bio-Linux-dev mailing list > Bio-Linux-dev at nebclists.nerc.ac.uk > http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux-dev -- Tim Booth NERC Environmental Bioinformatics Centre Centre for Ecology and Hydrology Maclean Bldg, Benson Lane Crowmarsh Gifford Wallingford, England OX10 8BB http://nebc.nerc.ac.uk +44 1491 69 2705 From tony.travis at ed.ac.uk Mon Jun 3 11:51:28 2013 From: tony.travis at ed.ac.uk (Tony Travis) Date: Mon, 03 Jun 2013 16:51:28 +0100 Subject: [Bio-linux-dev] Bio-Linux upgrade problems In-Reply-To: <1370271581.6820.286.camel@balisaur> References: <51ACA80E.5060107@ed.ac.uk> <1370271581.6820.286.camel@balisaur> Message-ID: <51ACBB80.6030902@ed.ac.uk> On 03/06/13 15:59, Tim Booth wrote: > Hi Tony, > > The good news is that /home/db is dead, or will be once you get the > upgrade to work, as can be verified on a BL7 machine by running: > [...] > Does that make sense? I expect this problem only impacts a couple of > people, and is a one-off, so I'd prefer to avoid the effort of adding > the fix to the script if that's OK. Hi, Tim. OK, and it's good to know that /home/db is no longer required :-) > Regarding the 3.5 series kernel, this is already the default in the > latest Bio-Linux ISO, but it is not the default if you upgrade from BL6 > - you get 3.2 the same as other Ubuntu upgraders. My logic is that the > new kernel is most useful for those with new hardware and thus not > relevant to most upgraders. (I'm using 3.2 until I see a good reason > not to - it never seems to crash on my laptop and everything works) Ah, I've just run the latest Bio-Linux ISO under VirtualBox to look. Sorry, I didn't realise that it is already based on 12.04.2 LTS. I've been doing 6->7 upgrades, rather than clean installs of Bio-Linux 7. > I'm inclined to think that proprietary drivers will cause aggro whatever > I do, but if you think there is something obvious I need to change to > help out with these issues then let me know. I caused these driver problems by installing the 'quantal' kernel as a side-effect of trying to make the package inventory of my upgraded systems the same as your iso, and I ended up with an unsupported LTS hardware enablement stack package combination. The link I posted is the correct way to update 12.04.1 LTS to the 3.5 kernel and X.org stack. It might be worth mentioning as a comment/printout at the end of the upgrade script otherwise people might not realise that the result of the upgrade is not quite the same as doing a clean install from the Bio-Linux 7 DVD. I didn't, and I should have known about this issue! Bye, Tony. From tony.travis at ed.ac.uk Mon Jun 3 14:41:15 2013 From: tony.travis at ed.ac.uk (Tony Travis) Date: Mon, 03 Jun 2013 19:41:15 +0100 Subject: [Bio-linux-dev] Bio-Linux upgrade problems In-Reply-To: <1370271581.6820.286.camel@balisaur> References: <51ACA80E.5060107@ed.ac.uk> <1370271581.6820.286.camel@balisaur> Message-ID: <51ACE34B.4020403@ed.ac.uk> On 03/06/13 15:59, Tim Booth wrote: > [...] > Regarding the 3.5 series kernel, this is already the default in the > latest Bio-Linux ISO, but it is not the default if you upgrade from BL6 > - you get 3.2 the same as other Ubuntu upgraders. My logic is that the > new kernel is most useful for those with new hardware and thus not > relevant to most upgraders. (I'm using 3.2 until I see a good reason > not to - it never seems to crash on my laptop and everything works) Hi, Tim. I checked again because I was sure that you had based Bio-Linux 7.0.6 on Ubuntu 12.04.1 LTS, from what I read in your 7.0.6 iso's .disk/info: > ajt at beluga:~/Downloads/iso$ sudo mount -o ro,loop bio-linux-7.0.6.iso /mnt > ajt at beluga:~/Downloads/iso$ cd /mnt/.disk > ajt at beluga:/mnt/.disk$ ls > base_installable casper-uuid-generic cd_type info release_notes_url > ajt at beluga:/mnt/.disk$ cat info > Bio-Linux 7 based on Ubuntu 12.04.1 LTS "Precise Pangolin" - Release amd64 (20120823.1) As far as I can see, Bio-Linux 7.0.6 *is* based on 12.04.1 LTS, but with the 3.5 kernel package installed. This is the cause of some of my problems, because the X.org stack from 12.04.1 LTS is not compatible with some of the drivers in the X.org stack from 12.04.2 LTS... I've compared the package inventories of Ubuntu 12.04.1 and 12.04.2 LTS, and the main difference (apart from the quantal kernel) is the X.org stack. However, only the quantal kernel is present in Bio-Linux 7.0.6. The other 12.04.2 LTS quantal packages and dependencies are missing: > ajt at beluga:~/src/dpkg-util/dpkg-dsel$ fgrep quantal bio-linux-7.0.6-filesystem.manifest > linux-image-generic-lts-quantal 3.5.0.26.33 These desktop 'selections' are based on the casper filesystem manifests on the iso, with the packages from filesystem.manifest-removed and should represent the output from "dpkg --get-selections" on a newly installed instance. The attached Makefile and dpkg-audit script are what I'm using to check the package inventories of upgraded systems. I'll audit the package inventory on a running instance of Bio-Linux 7.0.6 under VirtualBox and let you know what I find. BTW, there are some duplicate entries in your Bio-Linux 7.0.6 filesystem.manifest. HTH, Tony. > ajt at beluga:~/src/dpkg-util/dpkg-dsel$ diff ubuntu-12.04.1-desktop.sel ubuntu-12.04.2-desktop.sel > 454a455 >> libdrm-nouveau2 install > 487a489 >> libframe6 install > 512a515 >> libgeis1 install > 518,520c521,523 > < libgl1-mesa-dri install > < libgl1-mesa-glx install > < libglapi-mesa install > --- >> libgl1-mesa-dri-lts-quantal install >> libgl1-mesa-glx-lts-quantal install >> libglapi-mesa-lts-quantal install > 553a557 >> libgrail5 install > 595,596d598 > < libidl0 install > < libidl-common install > 636c638 > < libllvm3.0 install > --- >> libllvm3.1 install > 693d694 > < liborbit2 install > 856,859d856 > < libutouch-evemu1 install > < libutouch-frame1 install > < libutouch-geis1 install > < libutouch-grail1 install > 894c891 > < libxatracker1 install > --- >> libxatracker1-lts-quantal install > 921a919 >> libxrandr-ltsq2 install > 939,944c937,942 > < linux-generic install > < linux-headers-3.2.0-29 install > < linux-headers-3.2.0-29-generic install > < linux-headers-generic install > < linux-image-3.2.0-29-generic install > < linux-image-generic install > --- >> linux-generic-lts-quantal install >> linux-headers-3.5.0-23 install >> linux-headers-3.5.0-23-generic install >> linux-headers-generic-lts-quantal install >> linux-image-3.5.0-23-generic install >> linux-image-generic-lts-quantal install > 1297a1296 >> x11-xserver-utils-lts-quantal install > 1317,1346c1316,1346 > < xserver-xorg install > < xserver-xorg-core install > < xserver-xorg-input-all install > < xserver-xorg-input-evdev install > < xserver-xorg-input-mouse install > < xserver-xorg-input-synaptics install > < xserver-xorg-input-vmmouse install > < xserver-xorg-input-wacom install > < xserver-xorg-video-all install > < xserver-xorg-video-ati install > < xserver-xorg-video-cirrus install > < xserver-xorg-video-fbdev install > < xserver-xorg-video-intel install > < xserver-xorg-video-mach64 install > < xserver-xorg-video-mga install > < xserver-xorg-video-neomagic install > < xserver-xorg-video-nouveau install > < xserver-xorg-video-openchrome install > < xserver-xorg-video-qxl install > < xserver-xorg-video-r128 install > < xserver-xorg-video-radeon install > < xserver-xorg-video-s3 install > < xserver-xorg-video-savage install > < xserver-xorg-video-siliconmotion install > < xserver-xorg-video-sis install > < xserver-xorg-video-sisusb install > < xserver-xorg-video-tdfx install > < xserver-xorg-video-trident install > < xserver-xorg-video-vesa install > < xserver-xorg-video-vmware install > --- >> xserver-common-lts-quantal install >> xserver-xorg-core-lts-quantal install >> xserver-xorg-input-all-lts-quantal install >> xserver-xorg-input-evdev-lts-quantal install >> xserver-xorg-input-mouse-lts-quantal install >> xserver-xorg-input-synaptics-lts-quantal install >> xserver-xorg-input-vmmouse-lts-quantal install >> xserver-xorg-input-wacom-lts-quantal install >> xserver-xorg-lts-quantal install >> xserver-xorg-video-all-lts-quantal install >> xserver-xorg-video-ati-lts-quantal install >> xserver-xorg-video-cirrus-lts-quantal install >> xserver-xorg-video-fbdev-lts-quantal install >> xserver-xorg-video-intel-lts-quantal install >> xserver-xorg-video-mach64-lts-quantal install >> xserver-xorg-video-mga-lts-quantal install >> xserver-xorg-video-modesetting-lts-quantal install >> xserver-xorg-video-neomagic-lts-quantal install >> xserver-xorg-video-nouveau-lts-quantal install >> xserver-xorg-video-openchrome-lts-quantal install >> xserver-xorg-video-r128-lts-quantal install >> xserver-xorg-video-radeon-lts-quantal install >> xserver-xorg-video-s3-lts-quantal install >> xserver-xorg-video-savage-lts-quantal install >> xserver-xorg-video-siliconmotion-lts-quantal install >> xserver-xorg-video-sis-lts-quantal install >> xserver-xorg-video-sisusb-lts-quantal install >> xserver-xorg-video-tdfx-lts-quantal install >> xserver-xorg-video-trident-lts-quantal install >> xserver-xorg-video-vesa-lts-quantal install >> xserver-xorg-video-vmware-lts-quantal install -------------- next part -------------- # @(#)Makefile 2013-06-03 A.J.Travis # # Create a dselect-upgrade 'selections' file from Ubuntu to Bio-Linux # PREFIX=/usr/local SCRIPT=dpkg-dsel SELECT=ubuntu-12.04.1-desktop.sel ubuntu-12.04.2-desktop.sel bio-linux-7.0.6-desktop.sel all: $(SCRIPT) $(SELECT) install: $(SCRIPT) $(SELECT) install -Dp $(SCRIPT) $(PREFIX)/bin/ install -Dp $(SELECT) $(PREFIX)/share/dpkg-dsel/ clean: rm -f $(BIOLINUX)-filesystem.manifest-desktop clobber: clean rm -f $(SELECT) .SECONDARY: %-filesystem.manifest-desktop: %-filesystem.manifest %-filesystem.manifest-remove sort $*-filesystem.manifest | uniq | grep -vf $*-filesystem.manifest-remove > $@ %-desktop.sel: %-filesystem.manifest-desktop awk '{ printf("%-48sinstall\n", $$1) }' $< > $@ -------------- next part -------------- #!/usr/bin/perl -w #@(#)dpkg-audit.pl 2013-05-24 A.J.Travis # # Audit installed packages against distribution manifest # open( LIST, "<$ARGV[0]" ) || die "can't open $ARGV[0]"; open( DPKG, "dpkg --get-selections |" ) || die "can't run dpkg"; open( OUT, "| unexpand -a | sort" ) || die "can't run sort"; # read manifest while () { chomp; ( $name, $sel ) = split; $pkg{$name} = $sel; } close LIST; # read selections while () { chomp; ( $name, $sel ) = split; if ( !defined $pkg{$name} ) { if ( $sel eq "install" ) { $pkg{$name} = "deinstall"; } } else { if ( $pkg{$name} eq $sel ) { delete $pkg{$name}; } } } close DPKG; # write updated selections while ( ( $name, $sel ) = each %pkg ) { printf OUT "%-48s%s\n", $name, $sel; } close OUT; From tbooth at ceh.ac.uk Tue Jun 4 10:53:55 2013 From: tbooth at ceh.ac.uk (Tim Booth) Date: Tue, 4 Jun 2013 15:53:55 +0100 Subject: [Bio-linux-dev] [Bio-Linux] BioLinux install failure In-Reply-To: <1370355874.4199.137.camel@methy1> References: <1369428216.3523.111.camel@methy1> <1369735685.6820.102.camel@balisaur> <1370355874.4199.137.camel@methy1> Message-ID: <1370357635.6820.397.camel@balisaur> Hi Andor, Tony's e-mail earlier today on the developers list suggests that I have managed to make an ISO release that is stuck somewhere between 12.04.1 and 12.04.2 and this is upsetting the drivers for NVidia and probably other things. I will try to get this sorted in the next few days. In the meantime I'm glad you found a solution. Cheers, TIM On Tue, 2013-06-04 at 15:24 +0100, Andor J Kiss wrote: > Hi Tim, > > BioLinux (12.04.2) fails to install the nvidia driver, as the > linux-source package is not there to build against. My solution was > to add the xorg-edgers ppa and install the nvidia module and have it > build against the xorg-edgers ppa linux-source. > > Regards, > > -- Tim Booth NERC Environmental Bioinformatics Centre Centre for Ecology and Hydrology Maclean Bldg, Benson Lane Crowmarsh Gifford Wallingford, England OX10 8BB http://nebc.nerc.ac.uk +44 1491 69 2705 From tony.travis at ed.ac.uk Tue Jun 4 11:12:37 2013 From: tony.travis at ed.ac.uk (Tony Travis) Date: Tue, 04 Jun 2013 16:12:37 +0100 Subject: [Bio-linux-dev] [Bio-Linux] BioLinux install failure In-Reply-To: <1370357635.6820.397.camel@balisaur> References: <1369428216.3523.111.camel@methy1> <1369735685.6820.102.camel@balisaur> <1370355874.4199.137.camel@methy1> <1370357635.6820.397.camel@balisaur> Message-ID: <51AE03E5.1040305@ed.ac.uk> On 04/06/13 15:53, Tim Booth wrote: > Hi Andor, > > Tony's e-mail earlier today on the developers list suggests that I have > managed to make an ISO release that is stuck somewhere between 12.04.1 > and 12.04.2 and this is upsetting the drivers for NVidia and probably > other things. I will try to get this sorted in the next few days. In > the meantime I'm glad you found a solution. > > Cheers, > > TIM > > On Tue, 2013-06-04 at 15:24 +0100, Andor J Kiss wrote: >> Hi Tim, >> >> BioLinux (12.04.2) fails to install the nvidia driver, as the >> linux-source package is not there to build against. My solution was >> to add the xorg-edgers ppa and install the nvidia module and have it >> build against the xorg-edgers ppa linux-source. Hi, Tim and Andor. I also tried using the xorg-edgers ppa, but this makes the situation even more complicated. That's why I backed it out and used the 'official' 12.04.1 -> 12.04.2 upgrade method described at: https://wiki.ubuntu.com/Kernel/LTSEnablementStack My laptop display is working properly again :-) HTH, Tony. From tony.travis at ed.ac.uk Wed Jun 5 15:44:52 2013 From: tony.travis at ed.ac.uk (Tony Travis) Date: Wed, 05 Jun 2013 20:44:52 +0100 Subject: [Bio-linux-dev] [Bio-Linux] BioLinux install failure In-Reply-To: <1370357635.6820.397.camel@balisaur> References: <1369428216.3523.111.camel@methy1> <1369735685.6820.102.camel@balisaur> <1370355874.4199.137.camel@methy1> <1370357635.6820.397.camel@balisaur> Message-ID: <51AF9534.6060202@ed.ac.uk> On 04/06/13 15:53, Tim Booth wrote: > Hi Andor, > > Tony's e-mail earlier today on the developers list suggests that I have > managed to make an ISO release that is stuck somewhere between 12.04.1 > and 12.04.2 and this is upsetting the drivers for NVidia and probably > other things. I will try to get this sorted in the next few days. In > the meantime I'm glad you found a solution. Hi, Tim. This is the list of conflicting packages present in Bio-Linux 7.0.6: > ia32-libs-multiarch > libgl1-mesa-dri > libgl1-mesa-glx > libglapi-mesa > libllvm3.0 > libxatracker1 > python-pynast > xserver-xorg-core > xserver-xorg-input-all > xserver-xorg-input-evdev > xserver-xorg-input-mouse > xserver-xorg-input-synaptics > xserver-xorg-input-vmmouse > xserver-xorg-input-wacom > xserver-xorg > xserver-xorg-video-all > xserver-xorg-video-ati > xserver-xorg-video-cirrus > xserver-xorg-video-fbdev > xserver-xorg-video-intel > xserver-xorg-video-mach64 > xserver-xorg-video-mga > xserver-xorg-video-neomagic > xserver-xorg-video-nouveau > xserver-xorg-video-openchrome > xserver-xorg-video-qxl > xserver-xorg-video-r128 > xserver-xorg-video-radeon > xserver-xorg-video-s3 > xserver-xorg-video-savage > xserver-xorg-video-siliconmotion > xserver-xorg-video-sis > xserver-xorg-video-sisusb > xserver-xorg-video-tdfx > xserver-xorg-video-trident > xserver-xorg-video-vesa > xserver-xorg-video-vmware One are two are harmless virtual packages, but most cause severe compatibility problems when used with the 3.5 kernel. I successfully updated a clean install of Ubuntu 12.04.2 LTS to Bio-Linux 7.0.6 using your script. The list is the difference between package inventories. Bye, Tony.