[Pipet Devel] Loci and data storage

[J.W. Bizzaro]

Gary Van Domselaar wrote:
> 
> Yes, I remember reading this.  My understanding was that the DM was
> designed to extract data from existing databases, then convert the
> information to a format appropriate for further analysis. I wasnt sure
> that the concept had been extended to inputing (and maintaining) data
> from a new or existing database.

Hmmm.  I only assumed DMs would handle both input and output.  Alan will have to
correct me if I'm wrong.  Hello...Alan?

> I think it depends on the nature of the data processing and analysis
> that the end-user wishes to perform. If someone just wants an answer and
> a pretty picture, then there is nothing to gain by storing the result in
> a relational database.

Exactly.

> If, however, the researcher is working on a large
> project, they may very well want to have the results stored in a
> database, so that they can later query the database for interesting
> patterns, irregularities, etc. So I guess it would be an option, just
> one that I think is very important.

Sure.  Again, I think container loci should act as databases.  Anything that
includes query operations and not just storage (like AceDB) should be considered
a processor locus and not a container.

Containers won't include a query mechanism, so they must be connected to one:

    CONTAINER ------> PROCESSOR ------> CONTAINER
                          ^
                  Processor is used to
                  query the container
                  database.  The results
                  are placed in a new
                  container.

More sophisticated databases will expect some data with the query, so they will
be connected like this:

    DATA ------> PROCESSOR ------> CONTAINER
                     ^
             For example, processor
             takes sequence (data) and
             finds multiple alignments,
             which are stored in a
             container.

> So then, for clarity, if Loci were to allow for database storage of
> analysis results, it would be implemented as a DM output option, with
> the various available database interfaces as DM-plug-ins?

Yes, but so that we don't confuse anyone,

    Data Manager == Data Translator (converter)

We are not defining another locus type.  Whether my idea of a Data Translator
becomes more like Alan's idea of a Data Manager or visa versa, I don't know.


Cheers.
Jeff
-- 
                         +----------------------------+
                         |        J.W. Bizzaro        |
                         |  jeff at bioinformatics.org   |
                         |                            |
                         |        THE OPEN LAB        |
                         | Open Source Bioinformatics |
                         |                            |
                         | http://bioinformatics.org/ |
                         +----------------------------+