[Pipet Devel] hierarchy (was: icons)

[J.W. Bizzaro]

Brad Chapman wrote:
> 
> Jeff, I don't know if I am the only dumb one, but I haven't been
> commenting on this thread because I really don't understand what is
> going on :-). I understand what PL networks are because these are the
> networks that Overflow uses right now. The way I have been working
> with things is to make an Overflow network map directly onto the
> Loci "workspace" and dealing with things like this.

There's no real confusion here.

>     What I don't understand is:
> 
> 1. What exactly is a BL network?

A BL network is/was a collection of nodes where each node is an instance of
the BL (or Piper) located on a different, remote computer.  I was recently
thinking that BL networks should have their own workspaces, but now I think
remote nodes can be in the PL workspace (i.e., the BL network is
transparent).  So, we can nix the whole concept of any real "BL network".

> 2. How does a BL network differ from a PL network? Does it have to do
> with network location?

Again, the BL network is transparent, so it doesn't really exist.

> 2a. If it has to do with network location, I am really confused about
> what is going on, because I thought the dl would be mostly responsible
> for remote communication. In the documentation for the definition
> layer (http://bioinformatics.org/piper/documentation/dl_info.pdf) I
> wrote up with diagrams about how I thought remote communication would
> work, and this doesn't involve the bl communicating remotely. I
> thought this was how we were working, but maybe not. Do people not
> like this? Is this thinking incorrect? Please let me know so I can not
> only change it, but modify my thinking as well.

The BL network that I mentioned before is/was a "collection of nodes where
each node is an instance of the BL (or Piper) located on a different, remote
computer".  This means the nodes communicate via BL -> BL connection.  It
seems though, looking again at your whitepaper, that a BL -> BL connection is
supposed to be a BL -> DL -> DL -> BL connection.

Brad, can you  explain why the BL's can't/shouldn't communicate directly, but
instead should communicate back through the Build-Time Subsystem and the DL?

> 3. Shouldn't the whole entire implementation system of Piper be
> invisibile to the user? I thought this was the point of separating out
> the build time and run time layers. Why should the user have to worry
> about whether they are adding a bl or pl network or whatever?

That's the point of my bringing up Option A1.

> 4. What is the point of the PL and BL, exactly, in other people's
> minds? I thought that the PL handled application wrapping and basic
> piping between internal nodes, and that the BL would handle
> "organizing the workflow diagram" into a fast model, and implementing
> cool genetic algorithms to think of ways to make connected nodes run
> faster. Am I totally off? Maybe someone could clarify what is going on
> here if I am :-)

Actually, I never thought the BL would tell the PL how to execute its own
workflow.  It seems Jarl agrees that Overflow's "pull" method is very good.  I
thought that the BL would manage PL network -> remote PL network
communication, directly, with a remote BL.

> 5. Shouldn't dealing with the filesystem (like loading documents) or
> dealing with databases be done by the "build time system" and thus be
> handled by the definition layer? It seems like the run time system
> only cares about having data and manipulating it, and not having to
> deal with stuff like connecting to different databases, etc.

The BTS says where the data is, and the RTS goes and gets it.  In that sense,
they BOTH deal with filesystems and databases.

> 6. What are people's immmediate development goals for Piper? What I
> have been working towards is adding all of the functionality from
> Overflow into Piper, using the Piper framework. So when the system is
> done, we would have something with mostly Overflow functionality, but
> with the added advantage of one user interface, dealing with remote
> connections, having genetic algorithsm to improve running speed in the
> BL, etc.

I'm not sure how genetic algorithms in the BL will improve the execution of
networks, when most of that responsibility lies with the PL.  Remote (BL)
connections will be less common.  In any case, I think the PL and the BL need
to use the same approach for executing networks.  And, it seems like Jarl
likes the pull approach.

> Does this BL/PL network stuff that you are talking about
> affect how I should be coding right now? If so, what should I be
> doing differently?

Perhaps we should take a look at direct BL -> BL communication (keep execution
in the RTS).  From what Jarl has said, it seems the BL is very capable of
doing that.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+