[Bioclusters] Opteron Perl64 segfault issues
Joseph Landman
bioclusters@bioinformatics.org
16 Aug 2003 09:41:22 -0400
Hi Nathan, Carlos, Gregory:
I know Lawrence. I used to ask questions of him and his group when I
was at SGI. I am going to incorporate this flag and see if I can
generate a working m64 SUSE binary.
The (null).[pn]al problem has shown up a number of other places as
well. Specifically in the mpiformatdb code. Have you used the
-n db_name
switch to try to force the name?
I am not sure if the person loaning me the machine time/space will
allow a large file on there for testing. I will inquire.
Joe
On Sat, 2003-08-16 at 07:37, Nathan Siemers wrote:
> Hello Joe,
>
> Folks at IBM are suggesting this mod to compile blastall on opteron 64,
> and for the blastall binary itself it
> seems to work: If the full 64 bit works on redhat, then updating the
> gcc and libs may be the easiest way
> to go, but I would appreciate it if you test formatdb on large (6gig)
> input databases to see what it does...
> see the notes below.
>
>
> Nathan
>
>
>
> 2) BLAST
> Dr. Lawrence Hannon successfully resolved the coredump problem that he
> encountered with the 64-bit BLAST built using the default configuration
> files with gcc as downloaded from NCBI. He modified the NCBI_CFLAGS1 flag
> in ncbi/platform/linux.ncbi.mk to include the definition flag
> -DOS_UNIX_PPCLINUX which effects how the source ncbi/build/tsprintf.c
> treats variable arguments to get the gcc build to work. Both Lawrence
> and I had tested this 64-bit BLAST and so far all tests were successful.
>
>
> However, the resulting binaries haven't passed all my tests. In
> particular, formatdb
> seems to have problems creating intact dbs when it creates multiple volumes:
>
> Tests against small databases seem to be working, but...
>
> Some things are still not functioning perfectly:
>
> Given a large database file that must be split up into volumes, the
> resulting .nal or .pal file produced by formatdb does not have the name
> of the original file, a '(null)' is produced instead:
>
> -rw-r--r-- 1 rioscb root 270 Aug 15 08:25 (null).nal
>
> This was run via:
>
> formatdb -p F -i file
>
> where file was a 6gig database of nucleotide sequences.
>
> The file contents of the .nal file also has (null) where the original
> file name is supposed to be:
>
> #
> # Alias file created Fri Aug 15 08:25:05 2003
> #
> #
> TITLE NCBI_31_Masked_Chromosomes.shredded.100000.50000
> #
> DBLIST (null).00 (null).01 (null).02 (null).03 (null).04 (null).05
> (null).06 (null).07 (null).08 (null).09 (null).10 (null).11 (null).12
> #
> #GILIST
> #
> #OIDLIST
> #
>
> Manually fixing this .nal file and renaming it appropriately lets me run
> blastall against the file, but there are severe errors: possible pointer
> problems, blastall can't retrieve the name of the query, and the
> alignment is garbage:
>
> ________________________
> # blastall -a 2 -p tblastn -v 1 -b 1 -e 0.00001 -i refseq_fly_prot -d
> NCBI_31_Masked_Chromosomes.shredded.100000.50000
> __________________________
>
>
> TBLASTN 2.2.6 [Apr-09-2003]
>
>
> Reference: Altschul, Stephen F., Thomas L. Madden, Alejandro A. Schaffer,
> Jinghui Zhang, Zheng Zhang, Webb Miller, and David J. Lipman (1997),
> "Gapped BLAST and PSI-BLAST: a new generation of protein database search
> programs", Nucleic Acids Res. 25:3389-3402.
>
> Query= BMSPROT:NP_478140 histone H2A; histone [Drosophila
> melanogaster]. [Subset=EMBL/GENBANK/DDBJ,REFSEQ]
> (124 letters)
>
> Database: NCBI_31_Masked_Chromosomes.shredded.100000.50000
> 62,463 sequences; 6,244,139,235 total letters
>
> Searching..................................................done
>
> Score E
> Sequences producing significant alignments: (bits)
> Value
>
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> Unknown 199 5e-50
>
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
> Score = 199 bits (506), Expect = 5e-50
>
> Query: 16 SRSNRAGLQFPVGRIHRLLRKGNYAERVGAGAPVYLAAVMEYLAAEVLELAGNAARDNKK 75
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
> Query: 76 TRIIPRHLQLAIRNDEELNKLLSGVTIAQGGVLPNIQAVLLPKKTE 121
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
>
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
> Score = 196 bits (497), Expect = 5e-49
> Identities = 0/106 (0%), Positives = 0/106 (0%)
> Frame = +2
>
> Query: 16 SRSNRAGLQFPVGRIHRLLRKGNYAERVGAGAPVYLAAVMEYLAAEVLELAGNAARDNKK 75
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
> Query: 76 TRIIPRHLQLAIRNDEELNKLLSGVTIAQGGVLPNIQAVLLPKKTE 121
> [blastall] ERROR: ncbiapi [000.000] BMSPROT:NP_478140: ObjMgrChoice:
> pointer [0] type [1] not found
>
>
>
> All this work was done with:
>
> gcc version:
>
> Reading specs from /usr/lib64/gcc-lib/x86_64-suse-linux/3.2.2/specs
> Configured with: ../configure --enable-threads=posix --prefix=/usr
> --with-local-prefix=/usr/local --infodir=/usr/share/info
> --mandir=/usr/share/man --libdir=/usr/lib64
> --enable-languages=c,c++,f77,objc,java,ada --enable-libgcj
> --with-gxx-include-dir=/usr/include/g++ --with-slibdir=/lib
> --with-system-zlib --enable-shared --enable-__cxa_atexit x86_64-suse-linux
> Thread model: posix
> gcc version 3.2.2 (SuSE Linux)
>
>
> ldd on the executables gives:
>
> opt:/u1/gcgblast # ldd /usr/local/bin/blastall
> libm.so.6 => /lib64/libm.so.6 (0x0000002a9566a000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x0000002a957c2000)
> libc.so.6 => /lib64/libc.so.6 (0x0000002a958e2000)
> /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
> (0x0000002a95556000)
>
>
>
> Looking back at the log of the Make, these messages are disturbing, and
> they appear throughout the make process:
>
> gcc -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -O2 -c
> -DOS_UNIX_PPCLINUX -I../include -I/usr/X11R6/include -L/usr/X11R6/lib
> -DWIN_MOTIF cdnewlib.c
> In file included from ../include/ncbi.h:57,
> from ../include/asn.h:125,
> from ../include/objacces.h:64,
> from ../include/cdromlib.h:71,
> from cdnewlib.c:208:
> ../include/ncbilcl.h:104:2: warning: #warning Unknown processor type.
> Please define something appropriate.
> cdnewlib.c: In function `cd3_CdDocAsnOpen':
> cdnewlib.c:1342: warning: cast to pointer from integer of different size
> cdnewlib.c: In function `UidIndex_ReadPage':
> cdnewlib.c:1937: warning: cast to pointer from integer of different size
> cdnewlib.c:1945: warning: cast to pointer from integer of different size
>
>
>
>
>
>
>
>
> Joseph Landman wrote:
>
> >Hi Nathan:
> >
> > I just ran into this myself. I built the binaries on RedHat GinGin64
> >with the gcc-3.3 compiler. I just tried a SUSE 8.2 platform, and had to
> >recompile the toolkit. If you want to use the binary, I think you need
> >the relevant gcc tree installed. I think I might have that here. I
> >will look.
> >
> > For reasons that are not all that clear to me now, I had to use the
> >-m32 flag to generate a working binary on SUSE. The -m32 forces the use
> >of the IA32 ABI, which means that you cannot use some of the nicer
> >features of the Opteron (large memory, 64 bit mode, etc). I hope to
> >investigate this more.
> >
> > A kind soul just gave me access to their opteron for some testing.
> >Compiling with -m64 generates a binary, though upon completion of a test
> >run, it generates an ABORT. Strace wasn't all that helpful here. I am
> >trying to track this down.
> >
> > I can give you my -m32 binaries if you would like. I would like to
> >get the -m64 binaries working (for SUSE).
> >
> >Joe
> >
> >On Fri, 2003-08-15 at 22:58, Nathan Siemers wrote:
> >
> >
> >>Hello Joe,
> >>
> >>My default SUSE opteron configuration may not have the libs to run your
> >>build:
> >>
> >>
> >>opt:/u1/gcgblast # formatdb -p F -i test &
> >>formatdb: /lib64/libc.so.6: version `GLIBC_2.3' not found (required by
> >>formatdb)
> >>[1] 10941
> >>[1] Exit 1 formatdb -p F -i test
> >>opt:/u1/gcgblast # ldd /usr/local/bin/formatdb
> >>/usr/local/bin/formatdb: /lib64/libc.so.6: version `GLIBC_2.3' not found
> >>(required by /usr/local/bin/formatdb)
> >> libm.so.6 => /lib64/libm.so.6 (0x0000002a9566a000)
> >> libc.so.6 => /lib64/libc.so.6 (0x0000002a957c2000)
> >> /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
> >>(0x0000002a95556000)
> >>
> >>
> >>I assume these are 64 bit executables? Is there a newer libc needed?
> >>
> >>Thanks for your (past and present) help with the opteron platform.
> >>
> >> Nathan
> >>
> >>
> >>
> >>Joseph Landman wrote:
> >>
> >>
> >>
> >>>Hi Doug:
> >>>
> >>> These are 2.2.6. The tree's VERSION (ncbi/VERSION) reads:
> >>>
> >>>[landman@squash.scalableinformatics.com:/big/t/ncbi]
> >>>8 >cat VERSION
> >>>Tue Apr 22 10:41:40 EDT 2003
> >>>
> >>> Please let me know if you have problems with them. This was a first
> >>>pass at getting working systems.
> >>>
> >>>Joe
> >>>
> >>>On Tue, 2003-08-12 at 20:17, Doug Shubert wrote:
> >>>
> >>>
> >>>
> >>>
> >>>>Hello Joe,
> >>>>What version of ncbitools are you working with ?
> >>>>Doug
> >>>>
> >>>>Joe Landman wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Its up. Please let me know if it works. This is a snapshot from a
> >>>>>build done in June, Download is
> >>>>>http://scalableinformatics.com/downloads/opteron.ncbitools.tar.gz . It
> >>>>>is 29MB.
> >>>>>
> >>>>>Joe
> >>>>>
> >>>>>On Tue, 2003-08-12 at 15:46, Joe Landman wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>On Tue, 2003-08-12 at 13:11, Nathan Siemers wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>That's great news, Joe. Has this build passed any regression tests,
> >>>>>>>i.e. rigorous
> >>>>>>>comparison to output from blastall on other architectures?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>I have not had an opteron to run on long enough to run my normal tests.
> >>>>>>It did generate the same results for my short tests.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>--
> >>>>>Joseph Landman, Ph.D
> >>>>>Scalable Informatics LLC,
> >>>>>email: landman@scalableinformatics.com
> >>>>>web : http://scalableinformatics.com
> >>>>>phone: +1 734 612 4615
> >>>>>
> >>>>>_______________________________________________
> >>>>>Bioclusters maillist - Bioclusters@bioinformatics.org
> >>>>>https://bioinformatics.org/mailman/listinfo/bioclusters
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>_______________________________________________
> >>>>Bioclusters maillist - Bioclusters@bioinformatics.org
> >>>>https://bioinformatics.org/mailman/listinfo/bioclusters
> >>>>
> >>>>
> >>>>
> >>>>
> >>_______________________________________________
> >>Bioclusters maillist - Bioclusters@bioinformatics.org
> >>https://bioinformatics.org/mailman/listinfo/bioclusters
> >>
> >>
>
> _______________________________________________
> Bioclusters maillist - Bioclusters@bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters
--
Joseph Landman, Ph.D
Scalable Informatics LLC
email: landman@scalableinformatics.com
web: http://scalableinformatics.com
phone: +1 734 612 4615