Bioinformatics.org
[University of Birmingham]
[Patsnap]
Not logged in
  • Log in
    		•

    Bioinformatics.org
    Membership (44430+) Group hosting [?] Wiki
    Franklin Award
    Sponsorships

    Careers
    About bioinformatics
    Bioinformatics jobs

    Research
    All information groups
    Online databases Online analysis tools Online education tools More tools

    Development
    All software groups
    FTP repository
    SVN & CVS repositories [?]
    Mailing lists

    Forums
    News & Commentary
  • Submit
  • Archives
  • Subscribe

    • Jobs Forum
    (Career Center)
  • Submit
  • Archives
  • Subscribe
    		•

    BIRCH: Comprehensive bioinfo. system - Support tickets

    Submit | Open tickets | Closed tickets

    [ Ticket #1141 ] htmldoc.py - KeyError
    Date:
    04/09/11 14:24
    Submitted by:
    B_Fristensky     		Assigned to:
    B_Fristensky
    Category:
    Unset     		Priority:
    8
    Ticket group:
    Bug     		Resolution:
    Resolved
    Summary:
    htmldoc.py - KeyError
    Original submission:
    htmldoc.py gives the following error on some systems:

    Traceback (most recent call last):
    File "htmldoc.py", line 1063, in <module>
    main()
    File "htmldoc.py", line 1045, in main
    write_category_page(CATFN)
    File "htmldoc.py", line 475, in write_category_page
    writecat(c)
    File "htmldoc.py", line 453, in writecat
    text = pname + ' - ' + PROGDICT[pname].description + 'n'
    KeyError: 'UNINSTALL-birch.sh'

    This will require some work to nail down. Sometimes htmldoc.py completes successfully, other times, we get a message like this. When it fails, the remaining web pages don't get created.

    It does not appear to be a bug in tace. The .ace files dumped seem to be okay.

    I am wondering if this has something to do with the cmp_to_key function in birchlib.py?

    So far, I've only seen this on linux-x86_64 systems with small lbirchdb databases.
    Please log in to add comments and receive followups via email.
    Followups
    Comment Date By
    I think I have solved the problem. The random nature of the appearance of errors had to be caused by something that was not reproducible from run to run. I fixed two things.

    First, in the method DUMPTOACE, tbirchdb.sh was executed using os.system(), which doesn't guarantee that the child process will complete before the rest of the script runs. I changed that to use p=subprocess.Popen, followed by p.wait() to ensure completion.

    This by itself didn't fix the problem so I looked at tbirchdb.sh. It turns out that the script was running tbirchdb.sh in the background. Changing that to run in the foreground guarantees completion before htmldoc.py continues.
    		07/28/14 19:39 B_Fristensky
    The problem can be traced to reading in programs. Normally, after calling ReadProgram, there are 191 programs. But in the error condition, there are only 159 programs. Building PROGDICT depends upon reading the programs, so if there are programs missing, this would mess up the creation of PROGDICT. 07/28/14 18:08 B_Fristensky
    Hint: Just after programs are read, if we print the contents of PROGDICT, normally it's full of programs. But when a KeyError occurs, PROGDICT = {} ie. the empty dictionary.
    So when the error occurs, something either prevents PROGDICT from being created, or zeroes it out.     		07/28/14 17:49 B_Fristensky
    Adding the try/except clause shows that the error isn't only limited to write_category_page or to the three programs mentioned. It's just that those were the first we got to. There seems to be something fundamentally wrong with the way we're constructing the dictionaries.

    cleanup.sh finished
    ***** reading newstr.param
    ***** freshdir
    ***** read OK *****
    ***** writing category pages
    In writecat: KeyError: UNINSTALL-birch.py
    In writecat: KeyError: ssearch
    In writecat: KeyError: tace
    In writecat: KeyError: tblastn
    In writecat: KeyError: tblastx
    In writecat: KeyError: tfastf
    In writecat: KeyError: tfasts
    In writecat: KeyError: tfastx
    In writecat: KeyError: tfasty
    In writecat: KeyError: xace
    In writecat: KeyError: xtract
    In writecat: KeyError: xylem_identify
    In writecat: KeyError: tmev
    In writecat: KeyError: treedist
    In writecat: KeyError: treetool
    In writecat: KeyError: treedist
    In writecat: KeyError: treetool
    In writecat: KeyError: weighbor
    In writecat: KeyError: testcode
    In writecat: KeyError: ssearch
    In writecat: KeyError: tfastf
    In writecat: KeyError: tfasts
    In writecat: KeyError: tfastx
    In writecat: KeyError: tfasty
    In writecat: KeyError: TraceView
    In writecat: KeyError: tcoffee
    In writecat: KeyError: tfastx
    In writecat: KeyError: tfasty
    In writecat: KeyError: wconsensus
    In writecat: KeyError: treedist
    In writecat: KeyError: treetool
    In writecat: KeyError: tacg
    ***** writing program pages
    ***** writing package pages
    Traceback (most recent call last):
    File "./htmldoc.py", line 1101, in <module>
    main()
    File "./htmldoc.py", line 1095, in main
    WritePackagePage(PKGFN, P)
    File "./htmldoc.py", line 737, in WritePackagePage
    WritePkg(PK)
    File "./htmldoc.py", line 708, in WritePkg
    TEXT = PNAME + ' - ' + PROGDICT[PNAME].description + 'n'
    KeyError: 'tace'
    {fafner:/home/birch/BIRCH/install-scripts}
    		07/27/14 19:37 B_Fristensky
    This bug is still alive.

    We see it only in the write_category_page method, and apparently, only for the programs UNINSTALL-birch.py, pirupdate and rmlauncher.py.

    I have added a try/except clause to at least catch this error so that the program will continue properly (we hope)
    if encountering this error.     		07/27/14 19:18 B_Fristensky
    The problem seemed to be in use of the sort function. For example,

    c.program.sort(key=HT.cmp_to_key(lambda x, y: cmp(x.lower(), y.lower())))

    was changed to the simpler

    c.program.sort(key=lambda x: x.lower())

    All sort statements were changed to the new usage.
    		05/26/14 14:56 B_Fristensky
    One clue is that you can run htmldoc.py several times, and sometimes it works, and sometimes it doesn't. The error seems to be randomly determined. One attempted fix was to run htmldoc.py at BIRCHDEV, and make sure it works, prior to running makeframework.csh. However, upon downloading to a new workstation, the run of htmldoc.py that is done by getbirch failed the first time. This might be consistent with an error in htmldoc.py
    		05/25/14 19:50 B_Fristensky
    I am increasing the priority for this bug because if htmldoc.py fails during an update or an install, the program documentation directories will not be created. It's easy to fix that by running htmldoc.py, for reasons I can't fathom. But the user shouldn't have to know how to do that. 08/13/13 21:05 B_Fristensky
    No results for "Dependent on ticket"
    No results for "Dependent on Task"
    No other tickets are dependent on this ticket
    Ticket change history
    Field Old value Date By
    status_id Open 07/28/14 19:39 B_Fristensky
    resolution_id Not Resolved 07/28/14 19:39 B_Fristensky
    close_date 05/26/14 14:56 07/28/14 19:39 B_Fristensky
    status_id Closed 07/27/14 18:37 B_Fristensky
    priority 7 07/27/14 18:37 B_Fristensky
    resolution_id Resolved 07/27/14 18:37 B_Fristensky
    status_id Open 05/26/14 14:56 B_Fristensky
    resolution_id Not Resolved 05/26/14 14:56 B_Fristensky
    close_date 12/31/69 19:00 05/26/14 14:56 B_Fristensky
    priority 4 08/13/13 21:05 B_Fristensky
    priority 5 04/09/11 14:24 B_Fristensky
    assigned_to unset 04/09/11 14:24 B_Fristensky
    resolution_id Unset 04/09/11 14:24 B_Fristensky
    status_id Pending 04/09/11 14:24 B_Fristensky

     
    Copyright © 2024 Scilico, LLC · Privacy Policy