[Pipet Devel] NETPIPES

[J.W. Bizzaro]

Okay, I just wanted to hear from you guys if you thought Netpipes was
applicable or usable.  I guess not.

The reason I wanted some feedback on it was because of some of the
similarities with Piper.  Both Piper and Netpipes extend the Unix paradigm (at
least the standard shell in Netpipes' case) to allow piping across networks,
among other things.  And I thought some ideas or code may be reused from
Netpipes.

I've been trying to keep "The Unix Way" in mind when thinking about the design
of Loci (and now Piper).  For example, "networks" can be thought of as
directories or filesystem volumes (I've mentioned this a number of times). 
And nodes are often networks (and often not), just like Unix files (or what
seems to be a file when you take a listing) are often directories (and often
not).

It's important for user interface design (people will have to use Piper
somehow ;-)) to be able to represent this extended Unix paradigm in a familiar
way.  Those who have been on the Loci list will remember I mentioned a
console-based (textual) "front" (UI) for Loci.  The user would be able to
control Loci/Piper with the UI in a Unix shell-like manner.  How?  That's what
we have to keep in mind.

I'm thinking that the user would always be "inside" of a network, just like a
Unix shell user is inside of a directory.  So, a user can choose to "change
networks" (perhaps with a "cn" command), jumping between parent and child
networks.

Of course, this means the user needs to see what other nodes and networks are
in the "working network".  So, perhaps he/she can take a "listing" ("ls"
command?) of what is in that network.

Do you see what I'm getting at?  We're right in line with "The Unix Way" here
(very smoov), and we should should continue to think like this.  You may also
see now why Netpipes caught my attention (in case you don't, Netpipes ALSO
lets you make CONNECTIONS via command-line :-)).

(I'm not saying to use Netpipes.  Let's not, since it seems obsolete.  But, I
want to borrow some ideas for the console-based UI.)

Cheers.
Jeff

Brad Chapman wrote:
> 
> Okay, after starting at the documentation for a long time (man, this
> thing has a lot of switches!), it seems like Netpipes makes a
> client/server process that runs through TCP/IP, right? Mostly, it just
> seems to make it so that you can deal with sockets at a higher level
> and make simple client/servers. Like in this example:
> 
> server$ faucet 3000 --out tar cf - .
> client$ hose server 3000 --in tar xvf -
> 
> You just set up a simple server that untars everything that comes in
> (it listens on port 3000) and then a client tars up a directory and
> sends it to the server on port 3000. The server untars it and voila,
> you have a transferred directory.
> 
> At least from the python end this type of stuff is already covered in
> the standard library--the uil to dl layer works through these type
> of convenience classes. You also can deal with things using a nice
> scripting language and not shell scripts (bleah!).
> 
> >> It seems to me that we will need to
> >> re-invent much of this in the BL.
> 
> Can you explain exactly how you see netpipes being used? I think I
> understand what it does, I just don't see your vision for how to use
> it.
>     Also have you checked out the dl_info.html doc in the loci module?
> I have a diagram in that of how Jarl and I were last talking about
> handling remote communication. At least, this is what we ended up
> settling on last I heard :-). If stuff works like this then the dl
> shouldn't need to worry about setting up TCP/IP connections, right? Or
> do you disagree with this model?
> 
> > At least in CORBA -- Netpipes uses sockets.
> 
> Didn't we decide that sockets = bad and corba = good? :-)
> 
> > Perhaps the author of Netpipes would like to collaborate with us.
> There are
> > many similarities.  Just my US$0.02.
> 
> I don't know Jeff, the page you sent us was dated 1997, and I
> downloaded the "latest" code and it doesn't look like it's been
> touched since 98. Netpipes is looking pretty fast asleep...

jarl van katwijk wrote:
> 
> Dont want to disencourage, but ... ;)
> Netpipes is like netcat (nc), it's for piping (like the '|'  shell sign) streams
> across a
> network. But only in 2D space. This tool is just too simplistic for our needs.
> It aint
> got the ability to define connection types at all.
> As I said before, I'm intending to use Gnet for high speed connections, and
> corba
> for all the others. Gnet is a direct concurrent to netpipes, but it's a library
> that
> doesn't need any coding to integrate with the BL. Netpipes could be used in a
> 'wrapping' mode, but that's not very tempting to me.
> 
> (Gnet : http://www.eecs.umich.edu/~dhelder/misc/gnet/)


-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+