[Pipet Devel] Data Storage Interfaces

[Alan J. Williams]

On Wed, 23 Jun 1999, Justin Bradford wrote:
 > However, rather than just a SAX interface, I want DOM, too, and possibly
 > others. I think the DOM level 2 spec is shaping to handle what I want.
A well laid out interface to the data is important, but do we really need
to provide multiple interfaces.  For instance, we could standardize on the
SAX interface, but provide function calls and data structures so that
somebody developeing a loci could easily build his/her own tree.

I don't know anything about DOM level 2.  Guess I will have to head over
to the OMG web site.

 > With SAX, a callback function is registered, and then the parser calls
 > this function with each tag of XML it finds. The callback function does
 > what it likes with the data. Instead, we could pass a certain filter, and
 > only those nodes would be returned, but then we're getting into DOM2.
This is exactly what I was thinking (but had no clue that DOM2 addressed
it).  It is essential that we be able to provide filters to speed up data
access across the network.

 > Plus, DOM2 is supposed to eventually describe a query method, which will
 > be even better. DOM2 also describes ranges, which could be used to mark
 > off chunks of the data (data from a specific analysis, for instance).
 > DOM2 even describes notification events
Ok, now I am drooling. (There must be a catch.)

 > We describe a protocol to provide access to a DOM object over a network.
 > GNOME is (or may have already) doing this for CORBA. An IDL is provided
 > which describes the DOM interface. It doesn't matter what the object is, 
 > how big, or how it is stored on the other side of the CORBA interface,
 > and most importantly, the client does not have to transfer the whole
 > object across the ORB to use it. It just calls the functions (via the
 > ORB). We can do the same thing over TCP/IP, if necessary, too.
So could we use the GNOME DOM2 Interface IDL as a starting point for
developing a data access abstraction layer or API for loci.

 > So, we have some kind of object storage, which has our _virtual_ DOM
 > interface over some kind of back-end (be it a big tree in memory,
 > that Lore XML database, AceDB, Oracle, or some kind of caching, 
 > seekable XML parser). These are the central repositories of objects.
 > Other things connect to them and take/put little pieces that
 > they want. A remote object could create it's own object representation
 > of the data, or just rely on the object store to hold it over the 
 > network.
Sounds just like what I was thinking.

 > A really smart one of these things would be obviously integrated 
 > with the Workshop components. Whenever some operation was "committed" 
 > beyond the specific tool in use, it would update this object. There 
 > might have been a notification signal sitting on that specific node, 
 > too, which would then cause some other tool to get a DOM notification 
 > (the details here are still sketchy, as the DOM2 spec is still an 
 > early draft). This would be the central hub of Loci communication.
 > 
 > Now, is this what you mean by DM (data manager, right?)?
Yep.

 > This is kind of like the workflow server (or was it system) I 
 > described back in the early days, but after reading the DOM2 ideas, 
 > it's starting to take shape. Also note, the DOM2 stuff is just an 
 > interface. Consider the concept for now. We could just as easily 
 > do the same thing in a different interface we made up (but why?) 
 > and really, there's no reason why the virtual interface on each 
 > end has to be the same (as long as they speak the low-level
 > network protocol).
 > 
 > This could potentially allow for a little more distributed, roaming
 > path for the object. A client to a store could really be another
 > store itself (where a store is simply a program somewhere which 
 > presents a virtual interface to some data it has), taking bits of 
 > data from other places and presenting a new interface.

************************************************************************  
Alan Williams           
------------------------------------------------------------------------  
University of California, Riverside   "Where observation is concerned,
Dept. of Botany and Plant Sciences     chance favors the prepared mind."  
Alan at TheWilliamsFamily.org                         -- Louis Pasteur
************************************************************************