[Pipet Devel] node-to-node communication: you can have it 3 ways!

[J.W. Bizzaro]

jarl van katwijk wrote:
> 
> > Under what circumstances should 'VSH'
> > switch between using Overflow's protocol
> > and GMS's?  Well, if we decide that GMS
> > and Overflow will use DIFFERENT
> > protocols to communicate between themselves
> > (i.e., Overflow-to-Overflow and
> > GMS-to-GMS), we can find these circumstances.
> 
> What does overflow-overflow comm. mean:
> 1) overflow NODE <-> NODE communication
> or 2) overflow SUBNET <-> SUBNET communication?

Definitely 2).  I'm talking about instances of Overflow or Overflow sessions
(same applies to GMS).

> > Perhaps Overflow can use a protocol better
> > for small objects: like CORBA.  And
> > GMS could use a protocol better for moving
> > large files: straight TCP/IP.
> 
> GMS will get a specialised data transport
> protocal, I'm thinking about libgnet.
> (http://www.eecs.umich.edu/~dhelder/misc/gnet/)
> It's a specialised network
> library build on top of glib. So it's portable,
> fast and secure.

I've heard of it, and I think I remember the author proposing it on the
gnome-devel list.

> I think the 'overflow layer' should not use
> corba. That would be too slow and is
> highly vurnerable to exploiting. It should
> have a simple communication mechanism,
> corba would be overkill, also because the
> overflow nodes shouldn't transport data
> into nodes on non-local systems.

Did you hear that, Brad?  ;-)

> I like to
> see the overflow layer as nodes that
> can do anything INSIDE THEIR SUBNET, they
> can not communicate trough the
> overflow layer to another subnet, the
> dataflow must pass the gms layer.

Loci was not going to dictate how applications communicate, if they have their
own protocol.  This has a practical implication for the computational
biologists in our group, because many Internet-based databases are on the Web
(and therefore use HTTP), and many are developing CORBA API's.  If we (the
collaborative) then want to make such a database a node in our system, should
the database's own API be translated?  Why not let it work the way it is
designed to, and let VSH (or whatever) just 'broker' the connection?

Now, regarding Overflow-to-Overflow communication, if we consider an
independently running Overflow system (not used as a library) to be an
application that can communicate with another instance of itself, why not let
it?

The non-broker system you are proposing is still needed for connecting
applications with no Internet API, such as the common UNIX tools.

> I do not care about the numers of nodes, what
> they do, how or why, but they
> should do it all inside their subnet, or
> addressing space. This is critical to me, so
> I want everybody to comment on this.

Okay, as for Overflow, you're saying that node-to-node communication should be
contained within Overflow, like so...

       +------------------------------------+
       |                                    |
       |              Overflow              |
       |                                    |
       |    +--------+         +--------+   |
       |    |        |         |        |   |
       |    |  Node  |________\|  Node  |   |
       |    |        |        /|        |   |
       |    +--------+         +--------+   |
       |                                    |
       +------------------------------------+

But I'm saying that if an Overflow node needs to communicate with another
Overflow node across the Internet, it's not the node that does the
communicating but Overflow (brokering)...

       +-----------------+            +-----------------+
       |                 |            |                 |
       |     Overflow    |            |     Overflow    |
       |                 |            |                 |
       |    +--------+   |            |   +--------+    |
       |    |        |   |___________\|   |        |    |
       |    |  Node  |   |           /|   |  Node  |    |
       |    |        |   |            |   |        |    |
       |    +--------+   |            |   +--------+    |
       |                 |            |                 |
       +-----------------+            +-----------------+

or perhaps GMS (non-brokering)...

    +-----------------------+     +-------------------------+
    |         GMS           |     |           GMS           |
    |  +-----------------+  |     |   +-----------------+   |
    |  |                 |  |     |   |                 |   |
    |  |     Overflow    |  |     |   |     Overflow    |   |
    |  |                 |  |     |   |                 |   |
    |  |    +--------+   |  |     |   |   +--------+    |   |
    |  |    |        |   |  |____\|   |   |        |    |   |
    |  |    |  Node  |   |  |    /|   |   |  Node  |    |   |
    |  |    |        |   |  |     |   |   |        |    |   |
    |  |    +--------+   |  |     |   |   +--------+    |   |
    |  |                 |  |     |   |                 |   |
    |  +-----------------+  |     |   +-----------------+   |
    |                       |     |                         |
    +-----------------------+     +-------------------------+

My point is LITTLE NODES CAN COMMUNICATE ACROSS THE INTERNET TOO! :-)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+