[Pipet Devel] IRC - Overflow/Piper merging and XML files

[Brad Chapman]

Deanne wrote:
> Why don't we just code our own XML/Python DB storage 'dbm' file? We 
could
> store things in dictionaries and by keys (wink to Brad) instead of
> creating lots of little files. Then we could either load the thing 
into
> memory or just leave it as a file on the filesystem (through 
auto-pickling
> on occasion). I guess it goes by how fast Python can search thru the
> keys, but the dictionary type seems so well suited to this.

I think this is a very good idea, as I thought up the same thing 
myself a little while ago :-) (see the thread 'XML database stuff' 
from the march archives of the Loci list). I was proposing using the 
shelve module, which is basically a dictionary-like interface to 
pickle. I never actually implemented it, but from my reading into it, 
it didn't seem like it would scale well to a lot of files :-<. I 
currently use the shelve module for storing encrypted password 
information, but haven't dared to try and extend it to the xml files 
we store.
    I wrote some preliminary python bindings for XDBM 
(http://www.bowerbird.com.au/XDBM/), which is an xml database manager. 
It might be possible to build a system like you propose based on that, 
but I haven't really gotten into it any more from lack of time/lack 
of necessity, etc. 

    Brad