[Bioclusters] Opteron Perl64 segfault issues
Nathan O. Siemers
bioclusters@bioinformatics.org
Sat, 16 Aug 2003 11:38:55 -0400
Nope. I feared this would not solve the corruption of the blast
database itself:
(testy database created with formatdb -n testy...)
this is with my compiled version of formatdb using the suggested IBM
config mod...
opt:/u1/gcgblast # blastall -v 1 -b 1 -a 2 -d testy -i refseq_fly_prot
-p tblastn
TBLASTN 2.2.6 [Apr-09-2003]
Reference: Altschul, Stephen F., Thomas L. Madden, Alejandro A. Schaffer,
Jinghui Zhang, Zheng Zhang, Webb Miller, and David J. Lipman (1997),
"Gapped BLAST and PSI-BLAST: a new generation of protein database search
programs", Nucleic Acids Res. 25:3389-3402.
Query= BMSPROT:NP_478140 histone H2A; histone [Drosophila
melanogaster]. [Subset=EMBL/GENBANK/DDBJ,REFSEQ]
(124 letters)
Database: test
62,463 sequences; 6,244,139,235 total letters
Searching..................................................done
Score E
Sequences producing significant alignments: (bits)
Value
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
Unknown
199 5e-50
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
Score = 199 bits (506), Expect = 5e-50
Query: 16 SRSNRAGLQFPVGRIHRLLRKGNYAERVGAGAPVYLAAVMEYLAAEVLELAGNAARDNKK 75
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
Query: 76 TRIIPRHLQLAIRNDEELNKLLSGVTIAQGGVLPNIQAVLLPKKTE 121
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
Score = 196 bits (497), Expect = 5e-49
Identities = 0/106 (0%), Positives = 0/106 (0%)
Frame = +2
Query: 16 SRSNRAGLQFPVGRIHRLLRKGNYAERVGAGAPVYLAAVMEYLAAEVLELAGNAARDNKK 75
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
Query: 76 TRIIPRHLQLAIRNDEELNKLLSGVTIAQGGVLPNIQAVLLPKKTE 121
[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
pointer [0] type [1] not found
Joseph Landman wrote:
> Hi Nathan, Carlos, et al:
>
> I have verified on the test system that a formatdb without the -n
> option gives you the (null).[pn]al. Using the -n option provides a
> workaround. That is
>
> formatdb -i swissprot -v 2 -o T -p T
>
> does not work properly, though
>
> formatdb -i swissprot -n swissprot -v 2 -o T -p T
>
> does. I have not had the chance to try a real run, I will do that a
> little later today.
>
> This formatdb is
>
> file formatdb
> formatdb: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
> dynamically linked (uses shared libs), not stripped
>
> I rebuilt (as an m64 binary!) using Lawrence's recommended define, and
> ran that as well:
>
> test> file formatdb
> formatdb: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV),
> dynamically linked (uses shared libs), not stripped
> test> formatdb -i swissprot -n swissprot -v 2 -o T -p T
> test> ls
> formatdb.log swissprot.06.pin swissprot.12.psd swissprot.19.phr
> swissprot swissprot.06.pnd swissprot.12.psi swissprot.19.pin
> swissprot.00.phr swissprot.06.pni swissprot.12.psq swissprot.19.pnd
> swissprot.00.pin swissprot.06.psd swissprot.13.phr swissprot.19.pni
> swissprot.00.pnd swissprot.06.psi swissprot.13.pin swissprot.19.psd
> swissprot.00.pni swissprot.06.psq swissprot.13.pnd swissprot.19.psi
> swissprot.00.psd swissprot.07.phr swissprot.13.pni swissprot.19.psq
> swissprot.00.psi swissprot.07.pin swissprot.13.psd swissprot.20.phr
> swissprot.00.psq swissprot.07.pnd swissprot.13.psi swissprot.20.pin
> swissprot.01.phr swissprot.07.pni swissprot.13.psq swissprot.20.pnd
> swissprot.01.pin swissprot.07.psd swissprot.14.phr swissprot.20.pni
> swissprot.01.pnd swissprot.07.psi swissprot.14.pin swissprot.20.psd
> swissprot.01.pni swissprot.07.psq swissprot.14.pnd swissprot.20.psi
> swissprot.01.psd swissprot.08.phr swissprot.14.pni swissprot.20.psq
> swissprot.01.psi swissprot.08.pin swissprot.14.psd swissprot.21.phr
> swissprot.01.psq swissprot.08.pnd swissprot.14.psi swissprot.21.pin
> swissprot.02.phr swissprot.08.pni swissprot.14.psq swissprot.21.pnd
> swissprot.02.pin swissprot.08.psd swissprot.15.phr swissprot.21.pni
> swissprot.02.pnd swissprot.08.psi swissprot.15.pin swissprot.21.psd
> swissprot.02.pni swissprot.08.psq swissprot.15.pnd swissprot.21.psi
> swissprot.02.psd swissprot.09.phr swissprot.15.pni swissprot.21.psq
> swissprot.02.psi swissprot.09.pin swissprot.15.psd swissprot.22.phr
> swissprot.02.psq swissprot.09.pnd swissprot.15.psi swissprot.22.pin
> swissprot.03.phr swissprot.09.pni swissprot.15.psq swissprot.22.pnd
> swissprot.03.pin swissprot.09.psd swissprot.16.phr swissprot.22.pni
> swissprot.03.pnd swissprot.09.psi swissprot.16.pin swissprot.22.psd
> swissprot.03.pni swissprot.09.psq swissprot.16.pnd swissprot.22.psi
> swissprot.03.psd swissprot.10.phr swissprot.16.pni swissprot.22.psq
> swissprot.03.psi swissprot.10.pin swissprot.16.psd swissprot.23.phr
> swissprot.03.psq swissprot.10.pnd swissprot.16.psi swissprot.23.pin
> swissprot.04.phr swissprot.10.pni swissprot.16.psq swissprot.23.pnd
> swissprot.04.pin swissprot.10.psd swissprot.17.phr swissprot.23.pni
> swissprot.04.pnd swissprot.10.psi swissprot.17.pin swissprot.23.psd
> swissprot.04.pni swissprot.10.psq swissprot.17.pnd swissprot.23.psi
> swissprot.04.psd swissprot.11.phr swissprot.17.pni swissprot.23.psq
> swissprot.04.psi swissprot.11.pin swissprot.17.psd swissprot.24.phr
> swissprot.04.psq swissprot.11.pnd swissprot.17.psi swissprot.24.pin
> swissprot.05.phr swissprot.11.pni swissprot.17.psq swissprot.24.pnd
> swissprot.05.pin swissprot.11.psd swissprot.18.phr swissprot.24.pni
> swissprot.05.pnd swissprot.11.psi swissprot.18.pin swissprot.24.psd
> swissprot.05.pni swissprot.11.psq swissprot.18.pnd swissprot.24.psi
> swissprot.05.psd swissprot.12.phr swissprot.18.pni swissprot.24.psq
> swissprot.05.psi swissprot.12.pin swissprot.18.psd swissprot.pal
> swissprot.05.psq swissprot.12.pnd swissprot.18.psi
> swissprot.06.phr swissprot.12.pni swissprot.18.psq
> test> cat swissprot.pal
> #
> # Alias file created Sat Aug 16 16:03:34 2003
> #
> #
> TITLE swissprot
> #
> DBLIST swissprot.00 swissprot.01 swissprot.02 swissprot.03 swissprot.04
> swissprot.05 swissprot.06 swissprot.07 swissprot.08 swissprot.09
> swissprot.10 swissprot.11 swissprot.12 swissprot.13 swissprot.14
> swissprot.15 swissprot.16 swissprot.17 swissprot.18 swissprot.19
> swissprot.20 swissprot.21 swissprot.22 swissprot.23 swissprot.24
> #
> #GILIST
> #
> #OIDLIST
> #
>
> Looks like this is working. Is the use of the -n switch on formatdb an
> acceptable workaround for the (null) problem?
>
> Note: This machine doesn't have enough disk space to test the binaries
> with a big database file. I will post the SUSE 8.2 compiled binaries
> and linux.ncbi.mk in a bit. Please let me know if they work for you.
> They will be in http://scalableinformatics.com/downloads/opteron/ncbi as
> blastall.suse.gz and formatdb.suse.gz as well as the
> ncbitoolkit.suse.tar.gz tarball.
>
> Note: I have reorganized the opteron compiled bits to reside under
> opteron/...
>
> Joe
>
>
> On Sat, 2003-08-16 at 07:37, Nathan Siemers wrote:
>
>>Hello Joe,
>>
>>Folks at IBM are suggesting this mod to compile blastall on opteron 64,
>>and for the blastall binary itself it
>>seems to work: If the full 64 bit works on redhat, then updating the
>>gcc and libs may be the easiest way
>>to go, but I would appreciate it if you test formatdb on large (6gig)
>>input databases to see what it does...
>>see the notes below.
>>
>>
>>Nathan
>>
>>
>>
>>2) BLAST
>>Dr. Lawrence Hannon successfully resolved the coredump problem that he
>>encountered with the 64-bit BLAST built using the default configuration
>>files with gcc as downloaded from NCBI. He modified the NCBI_CFLAGS1 flag
>>in ncbi/platform/linux.ncbi.mk to include the definition flag
>>-DOS_UNIX_PPCLINUX which effects how the source ncbi/build/tsprintf.c
>>treats variable arguments to get the gcc build to work. Both Lawrence
>>and I had tested this 64-bit BLAST and so far all tests were successful.
>>
>>
>>However, the resulting binaries haven't passed all my tests. In
>>particular, formatdb
>>seems to have problems creating intact dbs when it creates multiple volumes:
>>
>>Tests against small databases seem to be working, but...
>>
>> Some things are still not functioning perfectly:
>>
>> Given a large database file that must be split up into volumes, the
>>resulting .nal or .pal file produced by formatdb does not have the name
>>of the original file, a '(null)' is produced instead:
>>
>>-rw-r--r-- 1 rioscb root 270 Aug 15 08:25 (null).nal
>>
>> This was run via:
>>
>> formatdb -p F -i file
>>
>> where file was a 6gig database of nucleotide sequences.
>>
>>The file contents of the .nal file also has (null) where the original
>>file name is supposed to be:
>>
>>#
>># Alias file created Fri Aug 15 08:25:05 2003
>>#
>>#
>>TITLE NCBI_31_Masked_Chromosomes.shredded.100000.50000
>>#
>>DBLIST (null).00 (null).01 (null).02 (null).03 (null).04 (null).05
>>(null).06 (null).07 (null).08 (null).09 (null).10 (null).11 (null).12
>>#
>>#GILIST
>>#
>>#OIDLIST
>>#
>>
>>Manually fixing this .nal file and renaming it appropriately lets me run
>>blastall against the file, but there are severe errors: possible pointer
>>problems, blastall can't retrieve the name of the query, and the
>>alignment is garbage:
>>
>>________________________
>># blastall -a 2 -p tblastn -v 1 -b 1 -e 0.00001 -i refseq_fly_prot -d
>>NCBI_31_Masked_Chromosomes.shredded.100000.50000
>>__________________________
>>
>>
>>TBLASTN 2.2.6 [Apr-09-2003]
>>
>>
>>Reference: Altschul, Stephen F., Thomas L. Madden, Alejandro A. Schaffer,
>>Jinghui Zhang, Zheng Zhang, Webb Miller, and David J. Lipman (1997),
>>"Gapped BLAST and PSI-BLAST: a new generation of protein database search
>>programs", Nucleic Acids Res. 25:3389-3402.
>>
>>Query= BMSPROT:NP_478140 histone H2A; histone [Drosophila
>>melanogaster]. [Subset=EMBL/GENBANK/DDBJ,REFSEQ]
>> (124 letters)
>>
>>Database: NCBI_31_Masked_Chromosomes.shredded.100000.50000
>> 62,463 sequences; 6,244,139,235 total letters
>>
>>Searching..................................................done
>>
>> Score E
>>Sequences producing significant alignments: (bits)
>>Value
>>
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>Unknown 199 5e-50
>>
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>
>> Score = 199 bits (506), Expect = 5e-50
>>
>>Query: 16 SRSNRAGLQFPVGRIHRLLRKGNYAERVGAGAPVYLAAVMEYLAAEVLELAGNAARDNKK 75
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>
>>Query: 76 TRIIPRHLQLAIRNDEELNKLLSGVTIAQGGVLPNIQAVLLPKKTE 121
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>
>>
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>
>> Score = 196 bits (497), Expect = 5e-49
>> Identities = 0/106 (0%), Positives = 0/106 (0%)
>> Frame = +2
>>
>>Query: 16 SRSNRAGLQFPVGRIHRLLRKGNYAERVGAGAPVYLAAVMEYLAAEVLELAGNAARDNKK 75
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>
>>Query: 76 TRIIPRHLQLAIRNDEELNKLLSGVTIAQGGVLPNIQAVLLPKKTE 121
>>[blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
>>pointer [0] type [1] not found
>>
>>
>>
>>All this work was done with:
>>
>>gcc version:
>>
>>Reading specs from /usr/lib64/gcc-lib/x86_64-suse-linux/3.2.2/specs
>>Configured with: ../configure --enable-threads=posix --prefix=/usr
>>--with-local-prefix=/usr/local --infodir=/usr/share/info
>>--mandir=/usr/share/man --libdir=/usr/lib64
>>--enable-languages=c,c++,f77,objc,java,ada --enable-libgcj
>>--with-gxx-include-dir=/usr/include/g++ --with-slibdir=/lib
>>--with-system-zlib --enable-shared --enable-__cxa_atexit x86_64-suse-linux
>>Thread model: posix
>>gcc version 3.2.2 (SuSE Linux)
>>
>>
>>ldd on the executables gives:
>>
>>opt:/u1/gcgblast # ldd /usr/local/bin/blastall
>> libm.so.6 => /lib64/libm.so.6 (0x0000002a9566a000)
>> libpthread.so.0 => /lib64/libpthread.so.0 (0x0000002a957c2000)
>> libc.so.6 => /lib64/libc.so.6 (0x0000002a958e2000)
>> /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
>>(0x0000002a95556000)
>>
>>
>>
>>Looking back at the log of the Make, these messages are disturbing, and
>>they appear throughout the make process:
>>
>>gcc -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -O2 -c
>>-DOS_UNIX_PPCLINUX -I../include -I/usr/X11R6/include -L/usr/X11R6/lib
>>-DWIN_MOTIF cdnewlib.c
>>In file included from ../include/ncbi.h:57,
>> from ../include/asn.h:125,
>> from ../include/objacces.h:64,
>> from ../include/cdromlib.h:71,
>> from cdnewlib.c:208:
>>../include/ncbilcl.h:104:2: warning: #warning Unknown processor type.
>>Please define something appropriate.
>>cdnewlib.c: In function `cd3_CdDocAsnOpen':
>>cdnewlib.c:1342: warning: cast to pointer from integer of different size
>>cdnewlib.c: In function `UidIndex_ReadPage':
>>cdnewlib.c:1937: warning: cast to pointer from integer of different size
>>cdnewlib.c:1945: warning: cast to pointer from integer of different size
>>
>>
>>
>>
>>
>>
>>
>>
>>Joseph Landman wrote:
>>
>>
>>>Hi Nathan:
>>>
>>> I just ran into this myself. I built the binaries on RedHat GinGin64
>>>with the gcc-3.3 compiler. I just tried a SUSE 8.2 platform, and had to
>>>recompile the toolkit. If you want to use the binary, I think you need
>>>the relevant gcc tree installed. I think I might have that here. I
>>>will look.
>>>
>>> For reasons that are not all that clear to me now, I had to use the
>>>-m32 flag to generate a working binary on SUSE. The -m32 forces the use
>>>of the IA32 ABI, which means that you cannot use some of the nicer
>>>features of the Opteron (large memory, 64 bit mode, etc). I hope to
>>>investigate this more.
>>>
>>> A kind soul just gave me access to their opteron for some testing.
>>>Compiling with -m64 generates a binary, though upon completion of a test
>>>run, it generates an ABORT. Strace wasn't all that helpful here. I am
>>>trying to track this down.
>>>
>>> I can give you my -m32 binaries if you would like. I would like to
>>>get the -m64 binaries working (for SUSE).
>>>
>>>Joe
>>>
>>>On Fri, 2003-08-15 at 22:58, Nathan Siemers wrote:
>>>
>>>
>>>
>>>>Hello Joe,
>>>>
>>>>My default SUSE opteron configuration may not have the libs to run your
>>>>build:
>>>>
>>>>
>>>>opt:/u1/gcgblast # formatdb -p F -i test &
>>>>formatdb: /lib64/libc.so.6: version `GLIBC_2.3' not found (required by
>>>>formatdb)
>>>>[1] 10941
>>>>[1] Exit 1 formatdb -p F -i test
>>>>opt:/u1/gcgblast # ldd /usr/local/bin/formatdb
>>>>/usr/local/bin/formatdb: /lib64/libc.so.6: version `GLIBC_2.3' not found
>>>>(required by /usr/local/bin/formatdb)
>>>> libm.so.6 => /lib64/libm.so.6 (0x0000002a9566a000)
>>>> libc.so.6 => /lib64/libc.so.6 (0x0000002a957c2000)
>>>> /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
>>>>(0x0000002a95556000)
>>>>
>>>>
>>>>I assume these are 64 bit executables? Is there a newer libc needed?
>>>>
>>>>Thanks for your (past and present) help with the opteron platform.
>>>>
>>>> Nathan
>>>>
>>>>
>>>>
>>>>Joseph Landman wrote:
>>>>
>>>>
>>>>
>>>>
>>>>>Hi Doug:
>>>>>
>>>>>These are 2.2.6. The tree's VERSION (ncbi/VERSION) reads:
>>>>>
>>>>>[landman@squash.scalableinformatics.com:/big/t/ncbi]
>>>>>8 >cat VERSION
>>>>>Tue Apr 22 10:41:40 EDT 2003
>>>>>
>>>>>Please let me know if you have problems with them. This was a first
>>>>>pass at getting working systems.
>>>>>
>>>>>Joe
>>>>>
>>>>>On Tue, 2003-08-12 at 20:17, Doug Shubert wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>Hello Joe,
>>>>>>What version of ncbitools are you working with ?
>>>>>>Doug
>>>>>>
>>>>>>Joe Landman wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Its up. Please let me know if it works. This is a snapshot from a
>>>>>>>build done in June, Download is
>>>>>>>http://scalableinformatics.com/downloads/opteron.ncbitools.tar.gz . It
>>>>>>>is 29MB.
>>>>>>>
>>>>>>>Joe
>>>>>>>
>>>>>>>On Tue, 2003-08-12 at 15:46, Joe Landman wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>On Tue, 2003-08-12 at 13:11, Nathan Siemers wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>That's great news, Joe. Has this build passed any regression tests,
>>>>>>>>>i.e. rigorous
>>>>>>>>>comparison to output from blastall on other architectures?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>I have not had an opteron to run on long enough to run my normal tests.
>>>>>>>>It did generate the same results for my short tests.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>--
>>>>>>>Joseph Landman, Ph.D
>>>>>>>Scalable Informatics LLC,
>>>>>>>email: landman@scalableinformatics.com
>>>>>>>web : http://scalableinformatics.com
>>>>>>>phone: +1 734 612 4615
>>>>>>>
>>>>>>>_______________________________________________
>>>>>>>Bioclusters maillist - Bioclusters@bioinformatics.org
>>>>>>>https://bioinformatics.org/mailman/listinfo/bioclusters
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>_______________________________________________
>>>>>>Bioclusters maillist - Bioclusters@bioinformatics.org
>>>>>>https://bioinformatics.org/mailman/listinfo/bioclusters
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>_______________________________________________
>>>>Bioclusters maillist - Bioclusters@bioinformatics.org
>>>>https://bioinformatics.org/mailman/listinfo/bioclusters
>>>>
>>>>
>>
>>_______________________________________________
>>Bioclusters maillist - Bioclusters@bioinformatics.org
>>https://bioinformatics.org/mailman/listinfo/bioclusters
--
Nathan Siemers|Associate Director|Applied Genomics|Bristol-Myers Squibb
Pharmaceutical Research
Institute|HW3-0.07|P.O. Box 5400|Princeton, NJ
08543-5400|(609)818-6568|nathan.siemers@bms.com