[Pipet Devel] Data Storage Interfaces

[Justin Bradford]

> > would be simpler if the DM just passed the whole xml file (in other words
> > the interface deals with xml entities only and not parts of entities).  On
> > the other hand, if a user just needs one part of the entity, the whole
> > entity must still be transfered over a potentially slow network
> > connection.
> 
> That's a good point.  We haven't considered slow networks or very large files.

I have, and that's my problem with 
1) how Paos passes objects -- it sends the whole thing. I would prefer
   just sending updates. Breaking up the data into linked objects could be
   an adequate compromise.
2) the independently roaming object concept where it's passed directly
   from tool to tool. Without a "home" everything has to be passed, and by
   the end of a complex series, that could be a large object.

> > The alternative is to have the xml parsing take plase on the
> > DM side of the interface.  This would allow for only the requested element
> > to be transfered. In addition, only the requested portion from a non xml
> > data set would have to be wrapped in xml. So despite the additional
> > complexity, it would seem that having the interface be aware of xml parts
> > and not just entities would be highly advantageous.
> 
> Hmmm.  I'd like to hear what Justin, Humberto et al think about that.

I think I was addressing similar issues in my previous mail.
Your second alternative is roughly a description for DOM (including level
2, however), except DOM doesn't make the programs directly write and parse
it's own XML whenever it crosses an interface. At the lowest level, it
might actually be straight XML passed across the network, but the
interface should abstract that.

I'm beginning to think the optimal solution is a virtual interface (or set
of optional interface) across all junctions. It's the most efficient (only
what the receiving end wants is sent [and only the receiving end
really knows what it wants]). It's completely language independent, as
well as "junction" indepedent (each end has a standard interface,
regardless of whether a C, Python, or Perl script is on the other end, or
whether the two are communication via CORBA, TCP/IP, UDP/IP, shared
memory, a pipe, a dynamically-loaded plug-in interface).

This interface method requires a home location where the object resides
throughout its processing life-time. This is what I had envisioned the
work flow system to be (ie. coordinating it's various objects, where
and when they connected, etc). This could be located on the client
machine, and it allows the various other loci to be really dumb (which
means small).

Justin