[Bioclusters] Opteron Perl64 segfault issues
Joseph Landman
bioclusters@bioinformatics.org
16 Aug 2003 10:26:10 -0400
Hi Nathan, Carlos, et al:
I have verified on the test system that a formatdb without the -n
option gives you the (null).[pn]al. Using the -n option provides a
workaround. That is
formatdb -i swissprot -v 2 -o T -p T
does not work properly, though
formatdb -i swissprot -n swissprot -v 2 -o T -p T
does. I have not had the chance to try a real run, I will do that a
little later today.
This formatdb is
file formatdb
formatdb: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
dynamically linked (uses shared libs), not stripped
I rebuilt (as an m64 binary!) using Lawrence's recommended define, and
ran that as well:
test> file formatdb
formatdb: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV),
dynamically linked (uses shared libs), not stripped
test> formatdb -i swissprot -n swissprot -v 2 -o T -p T
test> ls
formatdb.log swissprot.06.pin swissprot.12.psd swissprot.19.phr
swissprot swissprot.06.pnd swissprot.12.psi swissprot.19.pin
swissprot.00.phr swissprot.06.pni swissprot.12.psq swissprot.19.pnd
swissprot.00.pin swissprot.06.psd swissprot.13.phr swissprot.19.pni
swissprot.00.pnd swissprot.06.psi swissprot.13.pin swissprot.19.psd
swissprot.00.pni swissprot.06.psq swissprot.13.pnd swissprot.19.psi
swissprot.00.psd swissprot.07.phr swissprot.13.pni swissprot.19.psq
swissprot.00.psi swissprot.07.pin swissprot.13.psd swissprot.20.phr
swissprot.00.psq swissprot.07.pnd swissprot.13.psi swissprot.20.pin
swissprot.01.phr swissprot.07.pni swissprot.13.psq swissprot.20.pnd
swissprot.01.pin swissprot.07.psd swissprot.14.phr swissprot.20.pni
swissprot.01.pnd swissprot.07.psi swissprot.14.pin swissprot.20.psd
swissprot.01.pni swissprot.07.psq swissprot.14.pnd swissprot.20.psi
swissprot.01.psd swissprot.08.phr swissprot.14.pni swissprot.20.psq
swissprot.01.psi swissprot.08.pin swissprot.14.psd swissprot.21.phr
swissprot.01.psq swissprot.08.pnd swissprot.14.psi swissprot.21.pin
swissprot.02.phr swissprot.08.pni swissprot.14.psq swissprot.21.pnd
swissprot.02.pin swissprot.08.psd swissprot.15.phr swissprot.21.pni
swissprot.02.pnd swissprot.08.psi swissprot.15.pin swissprot.21.psd
swissprot.02.pni swissprot.08.psq swissprot.15.pnd swissprot.21.psi
swissprot.02.psd swissprot.09.phr swissprot.15.pni swissprot.21.psq
swissprot.02.psi swissprot.09.pin swissprot.15.psd swissprot.22.phr
swissprot.02.psq swissprot.09.pnd swissprot.15.psi swissprot.22.pin
swissprot.03.phr swissprot.09.pni swissprot.15.psq swissprot.22.pnd
swissprot.03.pin swissprot.09.psd swissprot.16.phr swissprot.22.pni
swissprot.03.pnd swissprot.09.psi swissprot.16.pin swissprot.22.psd
swissprot.03.pni swissprot.09.psq swissprot.16.pnd swissprot.22.psi
swissprot.03.psd swissprot.10.phr swissprot.16.pni swissprot.22.psq
swissprot.03.psi swissprot.10.pin swissprot.16.psd swissprot.23.phr
swissprot.03.psq swissprot.10.pnd swissprot.16.psi swissprot.23.pin
swissprot.04.phr swissprot.10.pni swissprot.16.psq swissprot.23.pnd
swissprot.04.pin swissprot.10.psd swissprot.17.phr swissprot.23.pni
swissprot.04.pnd swissprot.10.psi swissprot.17.pin swissprot.23.psd
swissprot.04.pni swissprot.10.psq swissprot.17.pnd swissprot.23.psi
swissprot.04.psd swissprot.11.phr swissprot.17.pni swissprot.23.psq
swissprot.04.psi swissprot.11.pin swissprot.17.psd swissprot.24.phr
swissprot.04.psq swissprot.11.pnd swissprot.17.psi swissprot.24.pin
swissprot.05.phr swissprot.11.pni swissprot.17.psq swissprot.24.pnd
swissprot.05.pin swissprot.11.psd swissprot.18.phr swissprot.24.pni
swissprot.05.pnd swissprot.11.psi swissprot.18.pin swissprot.24.psd
swissprot.05.pni swissprot.11.psq swissprot.18.pnd swissprot.24.psi
swissprot.05.psd swissprot.12.phr swissprot.18.pni swissprot.24.psq
swissprot.05.psi swissprot.12.pin swissprot.18.psd swissprot.pal
swissprot.05.psq swissprot.12.pnd swissprot.18.psi
swissprot.06.phr swissprot.12.pni swissprot.18.psq
test> cat swissprot.pal
#
# Alias file created Sat Aug 16 16:03:34 2003
#
#
TITLE swissprot
#
DBLIST swissprot.00 swissprot.01 swissprot.02 swissprot.03 swissprot.04
swissprot.05 swissprot.06 swissprot.07 swissprot.08 swissprot.09
swissprot.10 swissprot.11 swissprot.12 swissprot.13 swissprot.14
swissprot.15 swissprot.16 swissprot.17 swissprot.18 swissprot.19
swissprot.20 swissprot.21 swissprot.22 swissprot.23 swissprot.24
#
#GILIST
#
#OIDLIST
#
Looks like this is working. Is the use of the -n switch on formatdb an
acceptable workaround for the (null) problem?
Note: This machine doesn't have enough disk space to test the binaries
with a big database file. I will post the SUSE 8.2 compiled binaries
and linux.ncbi.mk in a bit. Please let me know if they work for you.
They will be in http://scalableinformatics.com/downloads/opteron/ncbi as
blastall.suse.gz and formatdb.suse.gz as well as the
ncbitoolkit.suse.tar.gz tarball.
Note: I have reorganized the opteron compiled bits to reside under
opteron/...
Joe
On Sat, 2003-08-16 at 07:37, Nathan Siemers wrote:
> Hello Joe,
>
> Folks at IBM are suggesting this mod to compile blastall on opteron 64,
> and for the blastall binary itself it
> seems to work: If the full 64 bit works on redhat, then updating the
> gcc and libs may be the easiest way
> to go, but I would appreciate it if you test formatdb on large (6gig)
> input databases to see what it does...
> see the notes below.
>
>
> Nathan
>
>
>
> 2) BLAST
> Dr. Lawrence Hannon successfully resolved the coredump problem that he
> encountered with the 64-bit BLAST built using the default configuration
> files with gcc as downloaded from NCBI. He modified the NCBI_CFLAGS1 flag
> in ncbi/platform/linux.ncbi.mk to include the definition flag
> -DOS_UNIX_PPCLINUX which effects how the source ncbi/build/tsprintf.c
> treats variable arguments to get the gcc build to work. Both Lawrence
> and I had tested this 64-bit BLAST and so far all tests were successful.
>
>
> However, the resulting binaries haven't passed all my tests. In
> particular, formatdb
> seems to have problems creating intact dbs when it creates multiple volumes:
>
> Tests against small databases seem to be working, but...
>
> Some things are still not functioning perfectly:
>
> Given a large database file that must be split up into volumes, the
> resulting .nal or .pal file produced by formatdb does not have the name
> of the original file, a '(null)' is produced instead:
>
> -rw-r--r-- 1 rioscb root 270 Aug 15 08:25 (null).nal
>
> This was run via:
>
> formatdb -p F -i file
>
> where file was a 6gig database of nucleotide sequences.
>
> The file contents of the .nal file also has (null) where the original
> file name is supposed to be:
>
> #
> # Alias file created Fri Aug 15 08:25:05 2003
> #
> #
> TITLE NCBI_31_Masked_Chromosomes.shredded.100000.50000
> #
> DBLIST (null).00 (null).01 (null).02 (null).03 (null).04 (null).05
> (null).06 (null).07 (null).08 (null).09 (null).10 (null).11 (null).12
> #
> #GILIST
> #
> #OIDLIST
> #
>
> Manually fixing this .nal file and renaming it appropriately lets me run
> blastall against the file, but there are severe errors: possible pointer
> problems, blastall can't retrieve the name of the query, and the
> alignment is garbage:
>
> ________________________
> # blastall -a 2 -p tblastn -v 1 -b 1 -e 0.00001 -i refseq_fly_prot -d
> NCBI_31_Masked_Chromosomes.shredded.100000.50000
> __________________________
>
>
> TBLASTN 2.2.6 [Apr-09-2003]
>
>
> Reference: Altschul, Stephen F., Thomas L. Madden, Alejandro A. Schaffer,
> Jinghui Zhang, Zheng Zhang, Webb Miller, and David J. Lipman (1997),
> "Gapped BLAST and PSI-BLAST: a new generation of protein database search
> programs", Nucleic Acids Res. 25:3389-3402.
>
> Query= BMSPROT:NP_478140 histone H2A; histone [Drosophila
> melanogaster]. [Subset=EMBL/GENBANK/DDBJ,REFSEQ]
> (124 letters)
>
> Database: NCBI_31_Masked_Chromosomes.shredded.100000.50000
> 62,463 sequences; 6,244,139,235 total letters
>
> Searching..................................................done
>
> Score E
> Sequences producing significant alignments: (bits)
> Value
>
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> Unknown 199 5e-50
>
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
> Score = 199 bits (506), Expect = 5e-50
>
> Query: 16 SRSNRAGLQFPVGRIHRLLRKGNYAERVGAGAPVYLAAVMEYLAAEVLELAGNAARDNKK 75
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
> Query: 76 TRIIPRHLQLAIRNDEELNKLLSGVTIAQGGVLPNIQAVLLPKKTE 121
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
>
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
> Score = 196 bits (497), Expect = 5e-49
> Identities = 0/106 (0%), Positives = 0/106 (0%)
> Frame = +2
>
> Query: 16 SRSNRAGLQFPVGRIHRLLRKGNYAERVGAGAPVYLAAVMEYLAAEVLELAGNAARDNKK 75
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
> Query: 76 TRIIPRHLQLAIRNDEELNKLLSGVTIAQGGVLPNIQAVLLPKKTE 121
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
>
>
> All this work was done with:
>
> gcc version:
>
> Reading specs from /usr/lib64/gcc-lib/x86_64-suse-linux/3.2.2/specs
> Configured with: ../configure --enable-threads=posix --prefix=/usr
> --with-local-prefix=/usr/local --infodir=/usr/share/info
> --mandir=/usr/share/man --libdir=/usr/lib64
> --enable-languages=c,c++,f77,objc,java,ada --enable-libgcj
> --with-gxx-include-dir=/usr/include/g++ --with-slibdir=/lib
> --with-system-zlib --enable-shared --enable-__cxa_atexit x86_64-suse-linux
> Thread model: posix
> gcc version 3.2.2 (SuSE Linux)
>
>
> ldd on the executables gives:
>
> opt:/u1/gcgblast # ldd /usr/local/bin/blastall
> libm.so.6 => /lib64/libm.so.6 (0x0000002a9566a000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x0000002a957c2000)
> libc.so.6 => /lib64/libc.so.6 (0x0000002a958e2000)
> /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
> (0x0000002a95556000)
>
>
>
> Looking back at the log of the Make, these messages are disturbing, and
> they appear throughout the make process:
>
> gcc -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -O2 -c
> -DOS_UNIX_PPCLINUX -I../include -I/usr/X11R6/include -L/usr/X11R6/lib
> -DWIN_MOTIF cdnewlib.c
> In file included from ../include/ncbi.h:57,
> from ../include/asn.h:125,
> from ../include/objacces.h:64,
> from ../include/cdromlib.h:71,
> from cdnewlib.c:208:
> ../include/ncbilcl.h:104:2: warning: #warning Unknown processor type.
> Please define something appropriate.
> cdnewlib.c: In function `cd3_CdDocAsnOpen':
> cdnewlib.c:1342: warning: cast to pointer from integer of different size
> cdnewlib.c: In function `UidIndex_ReadPage':
> cdnewlib.c:1937: warning: cast to pointer from integer of different size
> cdnewlib.c:1945: warning: cast to pointer from integer of different size
>
>
>
>
>
>
>
>
> Joseph Landman wrote:
>
> >Hi Nathan:
> >
> > I just ran into this myself. I built the binaries on RedHat GinGin64
> >with the gcc-3.3 compiler. I just tried a SUSE 8.2 platform, and had to
> >recompile the toolkit. If you want to use the binary, I think you need
> >the relevant gcc tree installed. I think I might have that here. I
> >will look.
> >
> > For reasons that are not all that clear to me now, I had to use the
> >-m32 flag to generate a working binary on SUSE. The -m32 forces the use
> >of the IA32 ABI, which means that you cannot use some of the nicer
> >features of the Opteron (large memory, 64 bit mode, etc). I hope to
> >investigate this more.
> >
> > A kind soul just gave me access to their opteron for some testing.
> >Compiling with -m64 generates a binary, though upon completion of a test
> >run, it generates an ABORT. Strace wasn't all that helpful here. I am
> >trying to track this down.
> >
> > I can give you my -m32 binaries if you would like. I would like to
> >get the -m64 binaries working (for SUSE).
> >
> >Joe
> >
> >On Fri, 2003-08-15 at 22:58, Nathan Siemers wrote:
> >
> >
> >>Hello Joe,
> >>
> >>My default SUSE opteron configuration may not have the libs to run your
> >>build:
> >>
> >>
> >>opt:/u1/gcgblast # formatdb -p F -i test &
> >>formatdb: /lib64/libc.so.6: version `GLIBC_2.3' not found (required by
> >>formatdb)
> >>[1] 10941
> >>[1] Exit 1 formatdb -p F -i test
> >>opt:/u1/gcgblast # ldd /usr/local/bin/formatdb
> >>/usr/local/bin/formatdb: /lib64/libc.so.6: version `GLIBC_2.3' not found
> >>(required by /usr/local/bin/formatdb)
> >> libm.so.6 => /lib64/libm.so.6 (0x0000002a9566a000)
> >> libc.so.6 => /lib64/libc.so.6 (0x0000002a957c2000)
> >> /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
> >>(0x0000002a95556000)
> >>
> >>
> >>I assume these are 64 bit executables? Is there a newer libc needed?
> >>
> >>Thanks for your (past and present) help with the opteron platform.
> >>
> >> Nathan
> >>
> >>
> >>
> >>Joseph Landman wrote:
> >>
> >>
> >>
> >>>Hi Doug:
> >>>
> >>> These are 2.2.6. The tree's VERSION (ncbi/VERSION) reads:
> >>>
> >>>[landman@squash.scalableinformatics.com:/big/t/ncbi]
> >>>8 >cat VERSION
> >>>Tue Apr 22 10:41:40 EDT 2003
> >>>
> >>> Please let me know if you have problems with them. This was a first
> >>>pass at getting working systems.
> >>>
> >>>Joe
> >>>
> >>>On Tue, 2003-08-12 at 20:17, Doug Shubert wrote:
> >>>
> >>>
> >>>
> >>>
> >>>>Hello Joe,
> >>>>What version of ncbitools are you working with ?
> >>>>Doug
> >>>>
> >>>>Joe Landman wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Its up. Please let me know if it works. This is a snapshot from a
> >>>>>build done in June, Download is
> >>>>>http://scalableinformatics.com/downloads/opteron.ncbitools.tar.gz . It
> >>>>>is 29MB.
> >>>>>
> >>>>>Joe
> >>>>>
> >>>>>On Tue, 2003-08-12 at 15:46, Joe Landman wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>On Tue, 2003-08-12 at 13:11, Nathan Siemers wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>That's great news, Joe. Has this build passed any regression tests,
> >>>>>>>i.e. rigorous
> >>>>>>>comparison to output from blastall on other architectures?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>I have not had an opteron to run on long enough to run my normal tests.
> >>>>>>It did generate the same results for my short tests.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>--
> >>>>>Joseph Landman, Ph.D
> >>>>>Scalable Informatics LLC,
> >>>>>email: landman@scalableinformatics.com
> >>>>>web : http://scalableinformatics.com
> >>>>>phone: +1 734 612 4615
> >>>>>
> >>>>>_______________________________________________
> >>>>>Bioclusters maillist - Bioclusters@bioinformatics.org
> >>>>>https://bioinformatics.org/mailman/listinfo/bioclusters
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>_______________________________________________
> >>>>Bioclusters maillist - Bioclusters@bioinformatics.org
> >>>>https://bioinformatics.org/mailman/listinfo/bioclusters
> >>>>
> >>>>
> >>>>
> >>>>
> >>_______________________________________________
> >>Bioclusters maillist - Bioclusters@bioinformatics.org
> >>https://bioinformatics.org/mailman/listinfo/bioclusters
> >>
> >>
>
> _______________________________________________
> Bioclusters maillist - Bioclusters@bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters
--
Joseph Landman, Ph.D
Scalable Informatics LLC
email: landman@scalableinformatics.com
web: http://scalableinformatics.com
phone: +1 734 612 4615