Original submission:
htmldoc.py gives the following error on some systems:
Traceback (most recent call last):
File "htmldoc.py", line 1063, in <module>
main()
File "htmldoc.py", line 1045, in main
write_category_page(CATFN)
File "htmldoc.py", line 475, in write_category_page
writecat(c)
File "htmldoc.py", line 453, in writecat
text = pname + ' - ' + PROGDICT[pname].description + 'n'
KeyError: 'UNINSTALL-birch.sh'
This will require some work to nail down. Sometimes htmldoc.py completes successfully, other times, we get a message like this. When it fails, the remaining web pages don't get created.
It does not appear to be a bug in tace. The .ace files dumped seem to be okay.
I am wondering if this has something to do with the cmp_to_key function in birchlib.py?
So far, I've only seen this on linux-x86_64 systems with small lbirchdb databases.
|
|
Followups
|
Comment
|
Date
|
By
|
I think I have solved the problem. The random nature of the appearance of errors had to be caused by something that was not reproducible from run to run. I fixed two things.
First, in the method DUMPTOACE, tbirchdb.sh was executed using os.system(), which doesn't guarantee that the child process will complete before the rest of the script runs. I changed that to use p=subprocess.Popen, followed by p.wait() to ensure completion.
This by itself didn't fix the problem so I looked at tbirchdb.sh. It turns out that the script was running tbirchdb.sh in the background. Changing that to run in the foreground guarantees completion before htmldoc.py continues.
|
07/28/14 19:39
|
B_Fristensky
|
|
The problem can be traced to reading in programs. Normally, after calling ReadProgram, there are 191 programs. But in the error condition, there are only 159 programs. Building PROGDICT depends upon reading the programs, so if there are programs missing, this would mess up the creation of PROGDICT.
|
07/28/14 18:08
|
B_Fristensky
|
Hint: Just after programs are read, if we print the contents of PROGDICT, normally it's full of programs. But when a KeyError occurs, PROGDICT = {} ie. the empty dictionary.
So when the error occurs, something either prevents PROGDICT from being created, or zeroes it out.
|
07/28/14 17:49
|
B_Fristensky
|
Adding the try/except clause shows that the error isn't only limited to write_category_page or to the three programs mentioned. It's just that those were the first we got to. There seems to be something fundamentally wrong with the way we're constructing the dictionaries.
cleanup.sh finished
***** reading newstr.param
***** freshdir
***** read OK *****
***** writing category pages
In writecat: KeyError: UNINSTALL-birch.py
In writecat: KeyError: ssearch
In writecat: KeyError: tace
In writecat: KeyError: tblastn
In writecat: KeyError: tblastx
In writecat: KeyError: tfastf
In writecat: KeyError: tfasts
In writecat: KeyError: tfastx
In writecat: KeyError: tfasty
In writecat: KeyError: xace
In writecat: KeyError: xtract
In writecat: KeyError: xylem_identify
In writecat: KeyError: tmev
In writecat: KeyError: treedist
In writecat: KeyError: treetool
In writecat: KeyError: treedist
In writecat: KeyError: treetool
In writecat: KeyError: weighbor
In writecat: KeyError: testcode
In writecat: KeyError: ssearch
In writecat: KeyError: tfastf
In writecat: KeyError: tfasts
In writecat: KeyError: tfastx
In writecat: KeyError: tfasty
In writecat: KeyError: TraceView
In writecat: KeyError: tcoffee
In writecat: KeyError: tfastx
In writecat: KeyError: tfasty
In writecat: KeyError: wconsensus
In writecat: KeyError: treedist
In writecat: KeyError: treetool
In writecat: KeyError: tacg
***** writing program pages
***** writing package pages
Traceback (most recent call last):
File "./htmldoc.py", line 1101, in <module>
main()
File "./htmldoc.py", line 1095, in main
WritePackagePage(PKGFN, P)
File "./htmldoc.py", line 737, in WritePackagePage
WritePkg(PK)
File "./htmldoc.py", line 708, in WritePkg
TEXT = PNAME + ' - ' + PROGDICT[PNAME].description + 'n'
KeyError: 'tace'
{fafner:/home/birch/BIRCH/install-scripts}
|
07/27/14 19:37
|
B_Fristensky
|
This bug is still alive.
We see it only in the write_category_page method, and apparently, only for the programs UNINSTALL-birch.py, pirupdate and rmlauncher.py.
I have added a try/except clause to at least catch this error so that the program will continue properly (we hope)
if encountering this error.
|
07/27/14 19:18
|
B_Fristensky
|
The problem seemed to be in use of the sort function. For example,
c.program.sort(key=HT.cmp_to_key(lambda x, y: cmp(x.lower(), y.lower())))
was changed to the simpler
c.program.sort(key=lambda x: x.lower())
All sort statements were changed to the new usage.
|
05/26/14 14:56
|
B_Fristensky
|
One clue is that you can run htmldoc.py several times, and sometimes it works, and sometimes it doesn't. The error seems to be randomly determined. One attempted fix was to run htmldoc.py at BIRCHDEV, and make sure it works, prior to running makeframework.csh. However, upon downloading to a new workstation, the run of htmldoc.py that is done by getbirch failed the first time. This might be consistent with an error in htmldoc.py
|
05/25/14 19:50
|
B_Fristensky
|
|
I am increasing the priority for this bug because if htmldoc.py fails during an update or an install, the program documentation directories will not be created. It's easy to fix that by running htmldoc.py, for reasons I can't fathom. But the user shouldn't have to know how to do that.
|
08/13/13 21:05
|
B_Fristensky
|
|
![[Next-Generation Sequencing Book]](https://www.bioinformatics.org/images/ack_banners/NGS-Book-1-728x90.png)