[Pipet Devel] workflow diagram data model and databases

[Gary Van Domselaar]

Locians,


today Jeff and I agonized over different methods of storing descriptions
of the workspace in a database.  This led us to try and develop a data
model for the workflow diagram, which is no easy task.  the workspace has
elements of tree-based model, in that there exist loci that can contain a
workspace (the composite locus), but also there is a 'relational' web-like
aspect where individual loci are related to other loci via the connections
made between them.  The description of the workspace is in XML.  Finally
Loci paradigm treats every locus as an object (with the exception of
inheritance).  So ultimately we have an XML description of a kind of 'web
model' with a semi-heirachical structure as well.  Because of the hardship
that we've had in finding a database that servers our very specific needs,
and is GPL's we decided to put the issue out on the floor for discussion. 

As we see it, these are some of the issues regarding the storage of the 
Workspace:

0.  The XML description of the WFD should be modular, but easily portable.


1. The WFD should be constructed from a number of smaller XML documents, 
essentially one per locus in the WFD. If the WFD contains a composite 
Locus, then that locus is itself a pointer to the xml documents contained 
within it.

A WFD then should be represented a single database (collection of files).
The DBMS should be able to manage multiple independent databases.

Connectivity between Loci must be preserved:  If you want to extract a 
subset of loci from a WFD you must first disconnect thost loci from any 
'external' loci, or you must extract the entire superset of connected 
loci along with the selected subset. I hope that makes sense.

The DBMS should operate as client/server processes in order to 
accommodate distributed processing requirements.

The DBMS should be able to quickly provide an XML description of 
information stored inside the database.

Others considerations?

Essentially our options, as far as we can see are:

1. Make our own custom database to store our workflow diagram.  This may 
be easier than it sounds because the nature of our data storage needs are 
so unique and specific that trying to write an interface to an existing 
DBMS might be just as hard or harder that writing our own custom loci-db.

2. Use the MySQL database with an XML->SQL->XML interface.  This would 
require some thinking in order to derive a relational data model that can 
accommodate the possibly quite complex Loci WFD.

3.  Use the PostgreSQL Object-Relational database with an XML-SQL-XML 
interface.  I'm not 'up' on how postgres differs from MySQL, but if it 
can more naturally handle objects (loci) and the relationships between 
them  (connections) than this may be a better choice that MySQL.  The 
same considerations exist for creating an intelligent data model for the 
WFD as in option 2.

4. Use an XML database.


We would appreciate any and all comments on this important aspect of 
Loci's design architecture, so fire away!


--gary
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Gary Van Domselaar		gvd at redpoll.pharmacy.ualberta.ca
Faculty of Pharmacy 		Phone: (780) 492-4493
University of Alberta		FAX:   (780) 492-5305
Edmonton, Alberta, Canada