From chapmanb at arches.uga.edu  Wed Mar  1 04:24:28 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:17 2006
Subject: [Pipet Devel] we're back
Message-ID: <200003010927.EAA63692@archa10.cc.uga.edu>

> Okay, I guess you're talking about different 'sessions' and not about
> multiple
> fronts tapping into one session.  If this is so, perhaps you should 
have...
> 
>  middle/temp/session1/workspace1
>  middle/temp/session2/workspace1
>  middle/temp/session3/workspace1

Okay, sessions then :) The newest cvs does just this--puts the info 
from each front started into a separate session and keeps track of 
them separately. When a front shuts down, the middle will 
automatically delete the xml associated with the session. When all 
fronts shut down, the middle turns itself off. I know that later on 
people might want to leave the middle running all of the time, even 
without any fronts going (ie. if they wanted to be serving specialty 
loci to users at other locations) but for right now I just want to 
make it as clean as possible to start up and close Loci.
 
> My idea for multiple fronts 'listening' or 'tapping' into a single 
session,
> allowed one front to control others.  If you're talking about 
multiple
> sessions, shouldn't each session have its own port?

Why can't all sessions listen on one port? I can distinguish sessions 
over the same port based on the the connection, and properly 
communicate only with a single connection when multiple connections 
are fired up. Is there a performance issue here I should worry about? 
As I mentioned before, I've never tried socket programming, so I don't 
know about this.

> Okay, so the front remains locked up while waiting for a response.

Right--my idea is that the middle can also return errors to the front, 
so this way the front waits until the middle responds positively, 
before it goes ahead and visualizes a change in the GUI. Does this 
make sense?
 
>> So I
>> think it might be worthwhile to put this off a bit until things are
>> smoothed out with this communication method.
> 
> Okay.  But what we're talking about is running the gnome front's GUI 
and
> listening routine as separate threads.  I think it should be done 
this way,
> so that we can have one front manipulated by others.

Okay, if we want to have the front listening on a different thread at 
all times
then we'll need to have some kind of login dialog with the middle like 
the following:

Front asks to login:

<front>
  <login>
    <user name = 'brad'
          password = 'encrypted' />
    <session name = 'my_gnome_session' />
  </login>
</front>

Middle responds:

<middle>
  <success>
    <port number = '12345' />
  </success>
</middle>

Then the front will have to listen on this port for info from the 
middle. The session name gives the session a unique name so that other 
fronts could manipulate the same session. The user name and password 
provide protection for the session. 
    So this could be done, but would require a login type dialog to 
begin a front. I think this is kind of overkill right now since we 
don't even have other fronts to start up :-) I think this is a good, 
and possible, idea, but should wait until things are a little further 
along.

> And I think you're talking about multiple sessions with one middle.  
But
> which
> would be more complex: one middle running multiple sessions or 
multiple
> middles
> with one session each?  Multiple middles would give us the problem I
> mentioned
> of the front (not) knowing which middle to choose.

Multiple middles would be a big pain, because we'd have to have them 
listen on separate ports to start with, and then would have to have 
some file or something where the fronts could find out what middles 
are listening on which ports. I'd rather do things the way I described 
above, with a login dialog. Then the middle can generate all of the 
new ports while running, and pass the info back to the middle. This 
way, all fronts start off by talking to the middle on one standard 
well-known port, and then can subsequently be assigned specific ports 
if necessary. Does this sound snazzy?

Brad





From bizzaro at geoserve.net  Wed Mar  1 10:57:45 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] sessions (was: we're back)
References: <200003010927.EAA63692@archa10.cc.uga.edu>
Message-ID: <38BD3DF9.429D6662@geoserve.net>

Brad Chapman wrote:
> 
> Okay, sessions then :) The newest cvs does just this--puts the info
> from each front started into a separate session and keeps track of
> them separately. When a front shuts down, the middle will
> automatically delete the xml associated with the session. When all
> fronts shut down, the middle turns itself off.

Great.  I like it.

> I know that later on
> people might want to leave the middle running all of the time, even
> without any fronts going (ie. if they wanted to be serving specialty
> loci to users at other locations) but for right now I just want to
> make it as clean as possible to start up and close Loci.

That's right.  We need to think of the middle as an Internet daemon.

> Why can't all sessions listen on one port? I can distinguish sessions
> over the same port based on the the connection, and properly
> communicate only with a single connection when multiple connections
> are fired up. Is there a performance issue here I should worry about?
> As I mentioned before, I've never tried socket programming, so I don't
> know about this.

Okay, I guess we're talking about Internet ports here, so it doesn't make sense
to use more than one.  My bad.  I was just thinking that we certainly want more
than one front-to-middle stream.

> > Okay, so the front remains locked up while waiting for a response.
> 
> Right--my idea is that the middle can also return errors to the front,
> so this way the front waits until the middle responds positively,
> before it goes ahead and visualizes a change in the GUI. Does this
> make sense?

Oh yeah.

> Okay, if we want to have the front listening on a different thread at
> all times then we'll need to have some kind of login dialog with the
> middle like the following:
> 
> Front asks to login:
> 
> <front>
>   <login>
>     <user name = 'brad'
>           password = 'encrypted' />
>     <session name = 'my_gnome_session' />

Session name or number?

>   </login>
> </front>
> 
> Middle responds:
> 
> <middle>
>   <success>
>     <port number = '12345' />
>   </success>
> </middle>

Very nice.

> Then the front will have to listen on this port for info from the
> middle. The session name gives the session a unique name so that other
> fronts could manipulate the same session. The user name and password
> provide protection for the session.

Excellent.  I was thinking that a front could tap into a session by providing
the session number/name on the command line:

    ./loci-gnome --session=3

(note, I want to change the name of front executables to loci-*)

>     So this could be done, but would require a login type dialog to
> begin a front. I think this is kind of overkill right now since we
> don't even have other fronts to start up :-) I think this is a good,
> and possible, idea, but should wait until things are a little further
> along.

Okay, no rush.

> Multiple middles would be a big pain, because we'd have to have them
> listen on separate ports to start with, and then would have to have
> some file or something where the fronts could find out what middles
> are listening on which ports.

We'll go with one middle then.

> I'd rather do things the way I described
> above, with a login dialog. Then the middle can generate all of the
> new ports while running, and pass the info back to the middle. This
> way, all fronts start off by talking to the middle on one standard
> well-known port, and then can subsequently be assigned specific ports
> if necessary. Does this sound snazzy?

Very snazzy.  BTW, what port number will we be using?  It's fixed, right?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar  1 11:47:41 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] config with xml
Message-ID: <38BD49AD.31B0B6A1@geoserve.net>

Locians,

There's an interesting article at arstechnica about Mac OS X using XML for
system configurations (first few pages):

  http://www.arstechnica.com/reviews/1q00/macos-x-dp3/macos-x-dp3-1.html

(BTW, the article says a bit about the GUI and using large icons.)

Of course, OS X is UNIX-based, and so this shows how XML can be used with more
traditional UNIX tools.

XML for configuration/preferences is something I'd like for Loci.  Gary told me
he'd prefer standard rc files.  What do you guys think?

Jeff


From stein at fmnh.org  Wed Mar  1 11:49:47 2000
From: stein at fmnh.org (Jennifer Steinbachs)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] sessions (was: we're back)
In-Reply-To: <38BD3DF9.429D6662@geoserve.net>
References: <200003010927.EAA63692@archa10.cc.uga.edu> <38BD3DF9.429D6662@geoserve.net>
Message-ID: <200003011649.KAA21117@mail.fmnh.org>

<big huge snip>

> > I'd rather do things the way I described
> > above, with a login dialog. Then the middle can 
> > generate all of the new ports while running, and
> > pass the info back to the middle. This way, all
> > fronts start off by talking to the middle on one
> > standard well-known port, and then can subsequently
> > be assigned specific ports if necessary. Does this
> > sound snazzy?
> 
> Very snazzy.  BTW, what port number will we be using? 
> It's fixed, right?

Please keep in mind that not everybody has unrestricted
access (bi-directionally) to the outside world.  How
will this communication work with firewalls?  Not
something you'd necessarily have to deal with in a
typical academic setting, but at my institution we have
serious restrictions on my outside access (e.g. if your
web server is running on port 8080 we cannot access it
without poking a hole in the firewall).

Just stirring the pot a bit :)

-j

------------------------
J. Steinbachs, PhD
Computational Biologist
Dept Botany
The Field Museum
Chicago, IL 60605-2496

312-665-7810
312-665-7158 (fax)
http://cb.fmnh.org
-------------------------


From tim at xoom.com  Wed Mar  1 11:52:45 2000
From: tim at xoom.com (Tim Triche)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] config with xml
In-Reply-To: <38BD49AD.31B0B6A1@geoserve.net>; from bizzaro@geoserve.net on Wed, Mar 01, 2000 at 04:47:41PM +0000
References: <38BD49AD.31B0B6A1@geoserve.net>
Message-ID: <20000301085245.B15154@xoom.com>

I think rc files suck.  The Windows registry was a step in the right direction
but of course marketing concerns caused its implementation to be suboptimal. 

Nonetheless, centralized, standardized configurations are the wave of the future
(and some cynics might add, "and always will be", cf. parallel computing ;-).

-- 

     "Friends help you move.  Real friends help you move bodies."

                                      --anonymous Internet hero


From bizzaro at geoserve.net  Wed Mar  1 12:19:52 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] sessions (was: we're back)
References: <200003010927.EAA63692@archa10.cc.uga.edu> <38BD3DF9.429D6662@geoserve.net> <200003011649.KAA21117@mail.fmnh.org>
Message-ID: <38BD5138.2373DD28@geoserve.net>

Jennifer Steinbachs wrote:
> 
> Please keep in mind that not everybody has unrestricted
> access (bi-directionally) to the outside world.  How
> will this communication work with firewalls?  Not
> something you'd necessarily have to deal with in a
> typical academic setting, but at my institution we have
> serious restrictions on my outside access (e.g. if your
> web server is running on port 8080 we cannot access it
> without poking a hole in the firewall).

That's a good point.  And yeah, we did say that front-to-middle communication
would be _local_.  So, why am I talking about Internet ports suddenly?  Brad,
can you explain how this front-to-middle communication works?  To what extent
does it use sockets and ports?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar  1 12:23:49 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] config with xml
References: <38BD49AD.31B0B6A1@geoserve.net> <20000301085245.B15154@xoom.com>
Message-ID: <38BD5225.8981DDF4@geoserve.net>

Tim Triche wrote:
> 
> I think rc files suck.  The Windows registry was a step in the right direction
> but of course marketing concerns caused its implementation to be suboptimal.
> 
> Nonetheless, centralized, standardized configurations are the wave of the future
> (and some cynics might add, "and always will be", cf. parallel computing ;-).

There is a binary 'registry' being developed for gnome ('gconf' I think it's
called).  But I want to keep gnome dependence with the gnome front if possible
(e.g., I don't want the web front to need gnome left and right).

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar  1 13:15:12 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] locus notes
Message-ID: <38BD5E30.3A1C1861@geoserve.net>

Locians,

The Nautilus file manager allows the user to attach 'notes' to files.  Here is
a screenshot:

  http://nautilus.eazel.com/screenshots/feb-28-2000/notes.jpg

I think this is along the line of what Brad was doing with 'messages'.  Gary
also mentioned to me that the host of a back-end application may want to state
that fact somewhere.

So, how about having a text widget where people can write all sorts of notes,
and have the notes kept as meta-data with the respective locus?  Plus, clicking
in an area of a composited GUI will switch notes to those of a 'sub-locus'
(locus contained within a composite).

Cheers.
Jeff


From bizzaro at geoserve.net  Wed Mar  1 14:23:33 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] config with xml
References: <38BD49AD.31B0B6A1@geoserve.net>
Message-ID: <38BD6E35.B6849733@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> Locians,
> 
> There's an interesting article at arstechnica about Mac OS X using XML for
> system configurations (first few pages):
> 
>   http://www.arstechnica.com/reviews/1q00/macos-x-dp3/macos-x-dp3-1.html

This is an older article, linked to from the above page:

  http://www.arstechnica.com/reviews/4q99/macos-x-dp2/macos-x-dp2-7.html

It discusses 'bundles' and filesystem meta-data in Mac OS X.

It's interesting to me, because I'm wondering about how we should 'plug'
back-ends and widgets into Loci.  We could spread a plug-in throughout the
directory structure (e.g., placing all icons in an icon directory) or keep all
the parts of a plug-in together (e.g., back-end 'foo' would have its own
directory with _all_ of its components kept within).  I'm leaning towards the
latter.  Any thoughts?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar  1 14:46:18 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] temp files
Message-ID: <38BD738A.4D467E91@geoserve.net>

Brad,

I thought you might find this helpful:

----------------------
Temporary (tmp) files

Most programs need to create temporary files, and this can be done perfectly
safely if a little care and attention is taken. The first thing a program
should do is honor the "TMP_DIR" environment variable. This allows users to
create a ~/tmp_dir directory (or similar) which can be much more protected than
the world accessible /tmp directory. Also if you must create temp files give
them truly random names. Using the process PID or time of day, or other
variables which should not "collide" will not prevent malicious users from
creating many tempfiles in the hopes your program will open one and overwrite
it. Also make sure that the file doesn't exist before creating it, but this can
be circumvented if the filename used is predictable. If you are worried about
the temp files a program may be creating you can use L0pht watch to monitor
directories (such as /tmp) and have the output go to syslog (so you can run
this as an administrator to monitor things).

http://securityportal.com/direct.cgi?/closet/closet20000301.html
----------------------

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From chapmanb at arches.uga.edu  Thu Mar  2 14:56:19 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] Fwd: Bioperl: Pise release 5.a
Message-ID: <200003021959.OAA141442@archa12.cc.uga.edu>

Sorry so slow to respond--I just finished exams this week and had a 
chance to check Pise out more.

> Pise looks like a good model for us to study when we imeplement
> our own web-based interface to Loci.

I don't know, I wasn't that excited by the web user interface (I 
scoped it out with the url that Jennifer send). The thing that I 
didn't really like is that:

1. It really hides the command line from the user, so that even if you 
used Pise for a while, you would still never really have an idea how 
to run the same program on the command line. I think a nice thing 
about Loci is that it is kind of a way to teach a user how to work 
with the command line. 

2. The help information for a program is separated into a big chunk at 
the bottom of the page. If you take a look, for instance, at the 
advanced FASTA page, there is a whole bunch of info at the bottom, 
which seems just about as fun to look at as a man page.

It supposedly has x windows/Tcl interfaces as well, but I didn't see 
any screen shots of them (and didn't fire Pise up on my computer). 
They also mention a "coversational" interface, which I assume is like 
the nli interface that Gary and Jeff are proposing, but didn't see any 
more about that either.

> From my cursory rip through
> throught the description, it differs from loci primarily in that the
> data redirection (which is not well defined, so I could be off here),
> seems simplistic.  From what I can tell it is not 
network-distrituted,
> simply piped.  

Right, it seems like they are saying that it is just a simple copy, 
and not even really piped (just analagous to pipes). 

> This implies that the processing applications reside on
> the same machine. The piping appears step-wise.  Consequently the
> communications layer is not abstracted, and thus limited to 
command-line
> programs.  Loci's strength is in its generalized data-connectivity
> brokering capability, 

Of course, implementing that will be not so easy :-0 But it will be 
cool :) It seems like Pise is just meant more for users 
afraid/unwilling to use the command line, and wouldn't be used by more 
advanced users. Hopefully, I think Loci will have enough power that 
advanced users will see an advantage to using it (much as many 
advanced users see an advantage to using an X windows system over 
starting at black and white all day).

> so I don't see Pise as a 'threat' so much as a
> 'resource': Pise is GPL, so we are free to use it and study it, and  
 It
> does appear to have some very good design points (similar to applab 
in
> many respects).

They use XML descriptions of programs to generate the output, which I 
think are really worth taking a look at. They've already got a lot of 
programs running with this, and it might be worthwhile to just use 
their XML to generate our own interfaces (plus better piping), 
provided no one finds their XML system too distasteful. I think this 
would be a good place to start for this, and then we can think about 
how to make Loci a development environment so that you can make these 
XML documents in the context of Loci.

    Of course, the code is hundreds of lines of undocumented perl. 
Fuuuuuuun reading! 

Brad





From chapmanb at arches.uga.edu  Thu Mar  2 15:13:43 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] sessions (was: we're back)
Message-ID: <200003022016.PAA138680@archa12.cc.uga.edu>

> Jennifer Steinbachs wrote:
>> 
>>  How will this communication work with firewalls?

Jeff wrote:

> Brad,
> can you explain how this front-to-middle communication works?  To 
what extent
> does it use sockets and ports?

Thanks for bringing this up, Jennifer! I don't know much about being 
behind firewalls (except for the simple firewall I set up for my 
computer) but from my understanding loopbacks to the localhost are 
normally allowed by the person setting up the firewall. I don't really 
know of any security risks invovled with this. 
    So, right now the only communication is occuring between front and 
middle on the same machine. So, the middle start up and binds to 
'localhost' on a port specified in the configuration file (right now 
it is just a 5 digit port (41877), nice and high). So from my 
understanding, this connection is not available to anyone outside of 
the local machine.
    My very limited understanding of how sockets work come from 
reading the "Socket Programming How-To" on the python web site 
(http://www.python.org/doc/howto/sockets/sockets.html). Basically, the 
middle creates a "server" socket which binds to localhost on the 41877 
port, and then listens for incoming connections. The front creates a 
"client" socket which sends a message to localhost, 41877 port. The 
"server" socket in the middle hears the message, and creates a 
"client" socket to handle it, and this is the connection that 
transfers the message.
    So, the point of all this is that all communication is through a 
single local port, and so shouldn't be affected by firewalls. I'd be 
interested if you tried to fire behind your firewall, Jennifer, and 
see what happens.
    Once we get into middle to middle communication, we'll need to 
worry about connecting between two computers, in which case firewalls 
and security will become an issue. I'm going to argue for using corba 
to do this, and not sockets, so maybe security and firewall 
compatibility can be one of my arguments. Thanks!

Brad
 



From chapmanb at arches.uga.edu  Thu Mar  2 15:19:30 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] locus notes (fwd)
Message-ID: <200003022022.PAA129016@archa11.cc.uga.edu>

> So, how about having a text widget where people can write all sorts 
of notes,
> and have the notes kept as meta-data with the respective locus?  
Plus,
> clicking
> in an area of a composited GUI will switch notes to those of a 
'sub-locus'
> (locus contained within a composite).

This sounds neat, and would definately be along the lines of what I 
was trying to do with the messages--keep the immediate info for a 
locus with a locus, instead of putting all info for all loci into one 
big file. I think in terms of xml representation, I could just store 
the information with a locus inside
<note> tags (or something like this). This could probably happen as 
soon as your ready to start implementing this in the GUI, since we 
could just define a protocol like:

<front>
  <add_note>
   <locus id = 'workspace1/container1.xml'/> <!--The locus to add the 
note to-->
   <info>
     What is in the note. Maybe we could even have <bold>tags</bold> or
     something here.
   </info>
  </add_note>
</front>

I need to work on saving loci so that they can be recalled in another 
session before this will be really useful, but we could start on it 
now, if you want.

Brad
    




From chapmanb at arches.uga.edu  Thu Mar  2 15:27:25 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] temp files
Message-ID: <200003022030.PAA37240@archa12.cc.uga.edu>

Jeff wrote:
> Brad,
> 
> I thought you might find this helpful:
> 
> ----------------------
> Temporary (tmp) files
> 
> Most programs need to create temporary files, and this can be done 
perfectly
> safely if a little care and attention is taken. The first thing a 
program
> should do is honor the "TMP_DIR" environment variable. This allows 
users to
> create a ~/tmp_dir directory (or similar) which can be much more 
protected
> than
> the world accessible /tmp directory. 

Okay, so we should stick our temp directory in a 
protected/non-accessable location. I think this should be defined in 
the config file.

> Also if you must create temp files give
> them truly random names. Using the process PID or time of day, or 
other
> variables which should not "collide" will not prevent malicious 
users from
> creating many tempfiles in the hopes your program will open one and 
overwrite
> it.

Okay, this is something I'm definately not doing. The temp files 
describing a front end are very structured (purposely, to make it 
easier to read and debug and to make the represenation of the 
workspace make sense). I don't think it would be that hard to start 
making them random names, we could just define a dictionary that maps 
random numbers to actual names:

save_file[234A58B67] = 'workspace1/workspace2/viewer3.xml'

or something like this. But I don't really understand the security 
argument here. Why should it matter to have random versus ordered 
names? 

> Also make sure that the file doesn't exist before creating it,

Right now the way it creates directories and files, files will never 
be written over the top of each other.

Do you think this article implies I should change things about the 
current storage to make it safer, or am I doing things right? I'm not 
positive about the random file stuff.

Brad






From bizzaro at geoserve.net  Fri Mar  3 00:12:00 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] temp files
References: <200003022030.PAA37240@archa12.cc.uga.edu>
Message-ID: <38BF49A0.E8ED2B39@geoserve.net>

Brad Chapman wrote:
> 
> Okay, so we should stick our temp directory in a
> protected/non-accessable location. I think this should be defined in
> the config file.

I think the article is saying the user should have a $TMP_DIR environment
variable set that gives the location of a temp directory ACCESSIBLE ONLY TO THE
ONE USER.  But I can't find a $TMP_DIR variable on my system.  We could have
the user define one in their .*shrc file.  But heck, why not have...

    $USER_HOME/loci/temp/

???

That would limit temp directory access to the user alone.

The problem is crackers/script-kiddies often place evil things in temp
directories that can compromise system security.  This has actually been the
cause of many security changes in big apps like apache and sendmail.

> Okay, this is something I'm definately not doing. The temp files
> describing a front end are very structured (purposely, to make it
> easier to read and debug and to make the represenation of the
> workspace make sense). I don't think it would be that hard to start
> making them random names, we could just define a dictionary that maps
> random numbers to actual names:
> 
> save_file[234A58B67] = 'workspace1/workspace2/viewer3.xml'
> 
> or something like this. But I don't really understand the security
> argument here. Why should it matter to have random versus ordered
> names?

I think what happens is a script-kiddie catches on to the fact that an
application _always_ places a file with a set name in /tmp.  For example, if an
application has a file named...

    /tmp/temporary.file

a kiddie can put a symlink in its place...

    /tmp/temporary.file@

that will cause the application to save things to places even less secure.  A
randomly named temp file would prevent this.

(But, I'm not a security guru.)

> > Also make sure that the file doesn't exist before creating it,
> 
> Right now the way it creates directories and files, files will never
> be written over the top of each other.

I think this is for the exact same reason as above.  The program should check
for the existence of a symlink where a file is to be written.

> Do you think this article implies I should change things about the
> current storage to make it safer, or am I doing things right? I'm not
> positive about the random file stuff.

Well, I believe all the points the article makes are good.  But we can worry
about this stuff as Loci gets bigger and more widely used.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar  3 00:22:12 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] sessions (was: we're back)
References: <200003022016.PAA138680@archa12.cc.uga.edu>
Message-ID: <38BF4C04.AB4B6573@geoserve.net>

Brad Chapman wrote:
> 
>     So, the point of all this is that all communication is through a
> single local port, and so shouldn't be affected by firewalls. I'd be
> interested if you tried to fire behind your firewall, Jennifer, and
> see what happens.

For those who have used 'port sniffers', can a loopback like this be detected?

>     Once we get into middle to middle communication, we'll need to
> worry about connecting between two computers, in which case firewalls
> and security will become an issue.

As it is for all applications communicating through the Internet.

> I'm going to argue for using corba
> to do this, and not sockets, so maybe security and firewall
> compatibility can be one of my arguments. Thanks!

What makes CORBA more secure and more 'compatible' with firewalls?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar  3 00:26:46 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] locus notes (fwd)
References: <200003022022.PAA129016@archa11.cc.uga.edu>
Message-ID: <38BF4D16.AD2CB8C7@geoserve.net>

Brad Chapman wrote:
>
> since we could just define a protocol like:
> 
> <front>
>   <add_note>
>    <locus id = 'workspace1/container1.xml'/> <!--The locus to add the
> note to-->
>    <info>
>      What is in the note. Maybe we could even have <bold>tags</bold> or
>      something here.
>    </info>
>   </add_note>
> </front>

This looks nice.  And it reminds me that the middle will have to 'approve' or
be notified of any changes to the notes.  That's going to be a challenge to
implement in the GUI: Should it be done letter-by-letter (nah), or when the
user exits the notes widget, or via a separate dialog box?

> I need to work on saving loci so that they can be recalled in another
> session before this will be really useful, but we could start on it
> now, if you want.

I'll give it a shot!

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar  3 00:32:04 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] locus notes (fwd)
References: <200003022022.PAA129016@archa11.cc.uga.edu>
Message-ID: <38BF4E54.C509A4C1@geoserve.net>

Brad Chapman wrote:
> 
>      What is in the note. Maybe we could even have <bold>tags</bold> or
>      something here.

Or use HTML tags and an HTML widget!

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar  7 02:14:07 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] python cheatsheet
Message-ID: <38C4AC3F.AFA6E4D8@geoserve.net>

For all you guys too timid to dive right into Python, IBM developerWorks has a
little intro:

  http://www-4.ibm.com/software/developer/library/cheatsheet3.html

Jeff


From bizzaro at geoserve.net  Tue Mar  7 02:43:05 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] TAME
Message-ID: <38C4B309.46475BCA@geoserve.net>

I thought this might be of some interest:

  http://www.tameable.com/

It's commercial software, running $15,000 per seat *gag*

Jeff


From bizzaro at geoserve.net  Fri Mar 10 00:51:56 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] barCode
Message-ID: <38C88D7C.E98E53E1@geoserve.net>

Greetings all.

I just got back from a conference/shin-dig in Boston, hosted by IBM. 
It's called 'barCode', and it's a 'developerWorks' conference they hold
at....bars, thus the name.  Well, this was at a very big bar/nightclub
called 'Jillians'.

David Lapointe (from UMass, and on the Loci list) found out about it and
sent me an e-mail.  I then sent an e-mail to Rick Ree (Pyhy/Mavric
coordinator from Harvard, also on the Loci list).  And, we all met
there, and I got to meet Rick for the first time.

So, why am I bringing this up?  Well, the guest speaker at the
conference was Miguel de Icaza.  If you don't know he is, I can start by
saying Loci would not be quite the same without him.  Miguel is the
founder and project leader of Gnome.

Miguel spoke for nearly an hour under colored lights and in front of a
big-screen TV that sits behind a bar, normally showing hockey games. 
Needless to say, it was quite a bit different from the scientific
seminars I am used to.  And his speeking style was right for the
atmosphere: Several times he yelled out that people in the audience were
drunk :-)

He, of course, presented Gnome, using some new presentation app that
runs in Gnome.  From time-to-time he switched to the desktop and demoed
some features.  He got some laughs when Gnumeric (the Gnome spreadsheet
app) crashed.  And they often booed when Miguel chose some tacky Gtk
theme.

Besides the chuckles, I did learn a little bit about Bonobo and thought
a bit about how it could be used in Loci.  I saw a Bonobo component that
drew vector graphics from an SVG file and did very cool things like
antialiasing and alpha-channel semi-transparencies (while being
DnD'd!).  I think this is what we're looking for to do our
bioinformatics viewers, as I reminded David several times during the
talk.

A little while after the presentation (and after some more free drinks,
pizza, and a nifty pen from IBM), David, Rick and I went up to meet
Miguel, who was mobbed by dozens of people.  We all got to meet him, and
I got to talk to him about The Open Lab and Loci.  In fact, I got to say
quite a bit, and I was surprised that he seemed very interested.  One
thing I mentioned was that the name of his company in Boston (Helix
Code) sounds like their involved in bioinformatics.  He said that his
girlfriend is a biologist, thus the use of names such as Bonobo and
Evolution (the PIM).  After several minutes, I gave him my card (which
has no e-mail address on it, duh), and he asked for my e-mail address.

Shortly after meeting Miguel, the three of us went to grab some more
goodies from IBM.  We got a 'squishy alien head', a stainless steel
thermo mug, a coaster, and an even cooler pen.  IBM paid for everything:
all sorts of food AND AN OPEN BAR.  It didn't cost anything to register
either.  They said they'll be holding these conferences once per
quarter.  (Do you see what your missing out on Gary?)  :-) 

Well, it was nice, and it was good to see David and Rick.  And I'd like
to see the rest of you guys some day.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From Kenneth_Marx at uml.edu  Fri Mar 10 08:30:01 2000
From: Kenneth_Marx at uml.edu (Kenneth A. Marx)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] Re: [tol-advisors] barCode
References: <38C88D7C.E98E53E1@geoserve.net>
Message-ID: <38C8F8D9.467972D5@uml.edu>

Hey Jeff, Dave and Rick,

Good for you! I didn't know you could have that much fun at a seminar.
As an aside, was it explained to the audience what IBM's motive is in
hosting such a conference? 
				TGIF,	Ken Marx 



				"J.W. Bizzaro" wrote:
> 
> Greetings all.
> 
> I just got back from a conference/shin-dig in Boston, hosted by IBM.
> It's called 'barCode', and it's a 'developerWorks' conference they hold
> at....bars, thus the name.  Well, this was at a very big bar/nightclub
> called 'Jillians'.
> 
> David Lapointe (from UMass, and on the Loci list) found out about it and
> sent me an e-mail.  I then sent an e-mail to Rick Ree (Pyhy/Mavric
> coordinator from Harvard, also on the Loci list).  And, we all met
> there, and I got to meet Rick for the first time.
> 
> So, why am I bringing this up?  Well, the guest speaker at the
> conference was Miguel de Icaza.  If you don't know he is, I can start by
> saying Loci would not be quite the same without him.  Miguel is the
> founder and project leader of Gnome.
> 
> Miguel spoke for nearly an hour under colored lights and in front of a
> big-screen TV that sits behind a bar, normally showing hockey games.
> Needless to say, it was quite a bit different from the scientific
> seminars I am used to.  And his speeking style was right for the
> atmosphere: Several times he yelled out that people in the audience were
> drunk :-)
> 
> He, of course, presented Gnome, using some new presentation app that
> runs in Gnome.  From time-to-time he switched to the desktop and demoed
> some features.  He got some laughs when Gnumeric (the Gnome spreadsheet
> app) crashed.  And they often booed when Miguel chose some tacky Gtk
> theme.
> 
> Besides the chuckles, I did learn a little bit about Bonobo and thought
> a bit about how it could be used in Loci.  I saw a Bonobo component that
> drew vector graphics from an SVG file and did very cool things like
> antialiasing and alpha-channel semi-transparencies (while being
> DnD'd!).  I think this is what we're looking for to do our
> bioinformatics viewers, as I reminded David several times during the
> talk.
> 
> A little while after the presentation (and after some more free drinks,
> pizza, and a nifty pen from IBM), David, Rick and I went up to meet
> Miguel, who was mobbed by dozens of people.  We all got to meet him, and
> I got to talk to him about The Open Lab and Loci.  In fact, I got to say
> quite a bit, and I was surprised that he seemed very interested.  One
> thing I mentioned was that the name of his company in Boston (Helix
> Code) sounds like their involved in bioinformatics.  He said that his
> girlfriend is a biologist, thus the use of names such as Bonobo and
> Evolution (the PIM).  After several minutes, I gave him my card (which
> has no e-mail address on it, duh), and he asked for my e-mail address.
> 
> Shortly after meeting Miguel, the three of us went to grab some more
> goodies from IBM.  We got a 'squishy alien head', a stainless steel
> thermo mug, a coaster, and an even cooler pen.  IBM paid for everything:
> all sorts of food AND AN OPEN BAR.  It didn't cost anything to register
> either.  They said they'll be holding these conferences once per
> quarter.  (Do you see what your missing out on Gary?)  :-)
> 
> Well, it was nice, and it was good to see David and Rick.  And I'd like
> to see the rest of you guys some day.
> 
> Cheers.
> Jeff
> --
>                       +----------------------------------+
>                       |           J.W. Bizzaro           |
>                       |                                  |
>                       | http://bioinformatics.org/~jeff/ |
>                       |                                  |
>                       |        BIOINFORMATICS.ORG        |
>                       |           The Open Lab           |
>                       |                                  |
>                       |    http://bioinformatics.org/    |
>                       +----------------------------------+
> 
> _______________________________________________
> tol-advisors maillist  -  tol-advisors@bioinformatics.org
> http://bioinformatics.org/mailman/listinfo/tol-advisors


From bizzaro at geoserve.net  Fri Mar 10 10:52:27 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:18 2006
Subject: [Pipet Devel] Re: barCode
References: <38C88D7C.E98E53E1@geoserve.net> <38C8F8D9.467972D5@uml.edu>
Message-ID: <38C91A3B.76855D99@geoserve.net>

"Kenneth A. Marx" wrote:
> 
> As an aside, was it explained to the audience what IBM's motive is in
> hosting such a conference?

Our impression was that IBM is trying to look 'young and hip' these days. 
This was all PR directed at software developers.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 10 20:16:15 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] Overflow
Message-ID: <38C99E5F.3FF866B8@geoserve.net>

It looks like we finally have some competition, or at least somebody who
re-invented the Loci wheel using Gnome:

  http://freespeech.on.openprojects.net/overflow.html

I'm already trying to contact the authors.

Jeff


From bizzaro at geoserve.net  Fri Mar 10 20:47:29 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] Overflow
References: <38C99E5F.3FF866B8@geoserve.net>
Message-ID: <38C9A5B1.4A7D57DB@geoserve.net>

Here you go, Brad.  This is a 'network' in Overflow.  Does it look perhaps
like a Loci WFD?

--------------------------------------
<?xml version="1.0"?>
<Document>
  <Network type="subnet" name="MAIN">
    <Node name="node1" type="Sound" x="-219.000000" y="-36.000000">
      <Parameter name="DEVICE" type="string" value="/dev/dsp"/>
      <Parameter name="RATE" type="int" value="44100"/>
      <Parameter name="STEREO" type="int" value="1"/>
      <Parameter name="BUFFER" type="int" value="40000"/>
    </Node>
    <Node name="node2" type="Constant" x="-372.000000" y="39.000000">
      <Parameter name="VALUE" type="subnet_param" value="IN"/>
    </Node>
    <Node name="node4" type="MAIN_LOOP" x="108.000000" y="26.000000">
      <Parameter name="FRAME_SIZE" type="int" value="256"/>
    </Node>
    <Node name="node3" type="ExecStream" x="-146.000000" y="37.000000">
      <Parameter name="COMMAND" type="string" value="mpg123 --stdout"/>
    </Node>
    <Link from="node1" output="OUTPUT" to="node4" input="OUT_FILE"/>
    <Link from="node2" output="VALUE" to="node3" input="INPUT"/>
    <Link from="node3" output="OUTPUT" to="node4" input="INPUT"/>
    <NetOutput name="OUTPUT" node="node4" terminal="OUTPUT"/>
  </Network>
  <Network type="iterator" name="MAIN_LOOP">
    <Node name="node2" type="FDSaveFrame" x="238.000000" y="213.000000"/>
    <Node name="node3" type="NotDone" x="112.000000" y="52.000000"/>
    <Node name="node1" type="AudioStream" x="-269.000000" y="77.000000">
      <Parameter name="LENGTH" type="subnet_param" value="FRAME_SIZE"/>
      <Parameter name="ENCODING" type="string" value="LIN16"/>
      <Parameter name="STREAM_TYPE" type="string" value="FILE"/>
    </Node>
    <Link from="node1" output="OUTPUT" to="node3" input="INPUT"/>
    <Link from="node1" output="OUTPUT" to="node2" input="OBJECT"/>
    <NetInput name="OUT_FILE" node="node2" terminal="FD"/>
    <NetOutput name="OUTPUT" node="node2" terminal="OUTPUT"/>
    <NetCondition name="CONDITION" node="node3" terminal="OUTPUT"/>
    <NetInput name="INPUT" node="node1" terminal="INPUT"/>
  </Network>
  <Parameter name="IN" type="string" value="/path/to/your.mp3"/>
</Document>
--------------------------------------

Scary.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 10 21:10:54 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] server down time
Message-ID: <38C9AB2E.5065B30A@geoserve.net>

Greetings.

Due to electrical work being done in the building, the servers will be shut
down for a while on Saturday.  We don't know exactly when or for how long,
but it'll probably be sometime during the day for less than an hour.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Sun Mar 12 15:08:40 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] Return of the Servers
Message-ID: <38CBF948.1DD06544@geoserve.net>

A long time ago, in a galaxy far, far away...The Open Lab servers were
halted in anticipation of a power outage that never came.  But the Rebel
Alliance brought them back up.  Have fun.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Sun Mar 12 20:24:36 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] [Fwd: Overflow and Loci]
Message-ID: <38CC4354.81F00A4E@geoserve.net>

Locians, I've been in contact with the Overflow developers.  I forward the
messages to the list and then  CC future messages.  Jeff
-------------- next part --------------
An embedded message was scrubbed...
From: "J.W. Bizzaro" <bizzaro@geoserve.net>
Subject: Overflow and Loci
Date: Sat, 11 Mar 2000 01:14:47 +0000
Size: 1928
Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000313/343ed4a7/attachment.mht
From bizzaro at geoserve.net  Sun Mar 12 20:25:08 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] [Fwd: Overflow and Loci]
Message-ID: <38CC4374.35755C34@geoserve.net>

 
-------------- next part --------------
An embedded message was scrubbed...
From: Jean-Marc Valin <jean-marc.valin@hermes.usherb.ca>
Subject: Re: Overflow and Loci
Date: Fri, 10 Mar 2000 23:03:33 -0500
Size: 2011
Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000313/ae376937/attachment.mht
From bizzaro at geoserve.net  Sun Mar 12 20:25:56 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] [Fwd: Overflow and Loci]
Message-ID: <38CC43A4.D6B97592@geoserve.net>

 
-------------- next part --------------
An embedded message was scrubbed...
From: "J.W. Bizzaro" <bizzaro@geoserve.net>
Subject: Re: Overflow and Loci
Date: Sun, 12 Mar 2000 06:42:47 +0000
Size: 8152
Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000313/3079c38a/attachment.mht
From bizzaro at geoserve.net  Sun Mar 12 20:27:03 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] [Fwd: Overflow and Loci]
Message-ID: <38CC43E7.B0F8E577@geoserve.net>

 
-------------- next part --------------
An embedded message was scrubbed...
From: Jean-Marc Valin <jean-marc.valin@hermes.usherb.ca>
Subject: Re: Overflow and Loci
Date: Sun, 12 Mar 2000 15:34:10 -0500
Size: 3032
Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000313/12542ed8/attachment.mht
From bizzaro at geoserve.net  Sun Mar 12 20:26:38 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] [Fwd: Overflow and Loci]
Message-ID: <38CC43CE.AACAE0AC@geoserve.net>

 
-------------- next part --------------
An embedded message was scrubbed...
From: Dominic Letourneau <doumdi@yahoo.com>
Subject: Re: Overflow and Loci
Date: Sun, 12 Mar 2000 05:46:02 -0800 (PST)
Size: 4533
Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000313/4fcd7660/attachment.mht
From bizzaro at geoserve.net  Mon Mar 13 02:00:52 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] The ABCs of XML
Message-ID: <38CC9224.AA21A7B3@geoserve.net>

Brad et al may find interesting what this article says about XML vs. CORBA
for RPC:

http://linuxtoday.com/stories/18230.html

Jeff


From admin at planetX.bastards.nl  Mon Mar 13 05:13:01 2000
From: admin at planetX.bastards.nl (jarl van katwijk)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] GMS
Message-ID: <200003131013.LAA06996@planetX.bastards.nl>

Hi Loci people,

I'm developing GMS, a project very similair to Loci (http://sunsite.auc.dk/gms).
I just recenly rewrote the GUI to be an seperate application communicating trough
Corba with the main engine, and the next phase is coding a better GUI.
Now that I seem Loci I'm considering the possibilities of integrating the GUI into
GMS. This will mean I have to port the python code to C.

I want to know whether the Loci GUI is mainly finished, or should I wait till it's
more mature,cq untill all planned features are done? 

thnx,
jarl


From bizzaro at geoserve.net  Mon Mar 13 06:42:06 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] GMS
References: <200003131013.LAA06996@planetX.bastards.nl>
Message-ID: <38CCD40E.21C2A9EB@geoserve.net>

Hi Jarl!  It's good to hear from you.  I've been meaning to write to you
since I saw the comment on Gnotices about GMS.  We (Loci) are talking with
the Overflow people about a collaboration, and I thought perhaps GMS would
like to join the party.

jarl van katwijk wrote:
> 
> I just recenly rewrote the GUI to be an seperate application communicating trough
> Corba with the main engine, and the next phase is coding a better GUI.
> Now that I seem Loci I'm considering the possibilities of integrating the GUI into
> GMS. This will mean I have to port the python code to C.

Hmmm.  I'm not sure what you mean by 'integrating the GUI'.  Are you
interested in getting GMS to work with Loci's actual GUI, or are you looking
to make something like it in C?

If you are looking to use our GUI with your own C program, Python has
language bindings to C (Python is in fact written in C).

> I want to know whether the Loci GUI is mainly finished, or should I wait till it's
> more mature,cq untill all planned features are done?

The design/plan for the GUI is pretty well thought out.  We have only to put
more of our ideas into code.  Again, what were you hoping to do?

(BTW, are you subscribed to this list?)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Mon Mar 13 06:44:47 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:19 2006
Subject: [Pipet Devel] xml error
Message-ID: <38CCD4AF.A0CA2E4@geoserve.net>

Brad,

I'm getting this XML error when trying to 'add' a container to the
workspace:

--------------------------
Fatal error: Couldn't open resource
'././middle/temp/xml/workspace1/container1.xml' at
././middle/temp/xml/workspace1/container1.xml:1:0
--------------------------

It appears the temp directory doesn't exist.

BTW, I just made a bunch of commits, so update your copy.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Mon Mar 13 06:56:46 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] xml error
References: <38CCD4AF.A0CA2E4@geoserve.net>
Message-ID: <38CCD77E.A845C3E1@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> --------------------------
> Fatal error: Couldn't open resource
> '././middle/temp/xml/workspace1/container1.xml' at
> ././middle/temp/xml/workspace1/container1.xml:1:0
> --------------------------
> 
> It appears the temp directory doesn't exist.

Sorry, that's wrong.  The temp directory does exist during runtime.

It appears the middle is trying to find...

    temp/xml/workspace1/container1.xml

while the actual locale is...

    temp/session1/xml/workspace1/container1.xml

Perhaps an error due to the addition of sessions.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Mon Mar 13 07:22:28 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131013.LAA06996@planetX.bastards.nl>
Message-ID: <38CCDD84.840767F2@geoserve.net>

jarl van katwijk wrote:
> 
> I'm developing GMS, a project very similair to Loci (http://sunsite.auc.dk/gms).

Jarl, looking at GMS, it appears you have accomplished much of what we haven't
even touched yet: Developing an API for getting independent programs to talk
nice to each other and the core.

On the other hand, we have been concentrating on developing a nice GUI plus an
API for getting several different GUIs to talk to our core.

Perhaps we can merge projects.  Brad (who has been developing our core) would
kill me if we had to scrap Loci's own core, but why don't we take a look at
each other's designs and see where we can meet?

As I mentioned, Python is written in C and has language bindings to it, so we
may be able to make a Python wrapper for your core that will allow us to make
Python interfaces.  And not just GUI interfaces: We were planning on making a
natural language interface that will let the user 'talk' to Loci.

A C core would be faster, which is an advantage to us.  And Python interfaces
are easier to make than those in C, which would be an advantage to you.

What do you think?  Again, Loci has a very well thought out design, but I
think we may be able to reach a middle ground.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Mon Mar 13 09:49:51 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131013.LAA06996@planetX.bastards.nl> <38CCDD84.840767F2@geoserve.net>
Message-ID: <38CD000F.1E2F94F2@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> As I mentioned, Python is written in C and has language bindings to it, so we
> may be able to make a Python wrapper for your core that will allow us to make
> Python interfaces.

Oh, I'm forgetting that you're using CORBA.  There are ORBit bindings to
Python (2 projects in fact).  So, maybe we can handle the GUI part of the
project and you can handle the infrustructure.

Loci of course has some design plans that are very elaborate and which come
from thinking about this type of project as strictly connecting icons together
with lines.  But we can work toward those goals as time passes.  Right now I
am most anxious about getting some help with the infrastructure and making
something that works.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Mon Mar 13 09:57:36 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] loci] GMS
References: <200003131013.LAA06996@planetX.bastards.nl> <38CCDD84.840767F2@geoserve.net>
Message-ID: <38CD01DF.8B11893@casema.net>

| Hi Jarl!  It's good to hear from you.  I've been meaning to write to you
| since I saw the comment on Gnotices about GMS.  We (Loci) are talking with
| the Overflow people about a collaboration, and I thought perhaps GMS would
| like to join the party.


Hi again,
I've been searching a long time for similair projects and now there're two at once..
nice-n-sweet luxory. I've also checked out UML (mentioned on the news.gnome.org
discussion)
which has several interesting features. I recommend reading this, and in particulair
Booch.


>
> I just recenly rewrote the GUI to be an seperate application communicating trough
> Corba with the main engine, and the next phase is coding a better GUI.
> Now that I seem Loci I'm considering the possibilities of integrating the GUI into
> GMS. This will mean I have to port the python code to C.

| Hmmm.  I'm not sure what you mean by 'integrating the GUI'.  Are you
| interested in getting GMS to work with Loci's actual GUI, or are you looking
| to make something like it in C?
Yes and no.. a GUI for gms only needs to 'speak' to the CORBA api the 'core' exports
and must offer a corba api to the same core for displaying dialogs etc. (a gms
core-gui
couple has 2 corba communication lines) The speed and portability of C makes it a
good candidate, but probably Python will be good enough. As long as Python has good
gtk- and corba bindings.

> I want to know whether the Loci GUI is mainly finished, or should I wait till it's
> more mature,cq untill all planned features are done?

| The design/plan for the GUI is pretty well thought out.  We have only to put
| more of our ideas into code.  Again, what were you hoping to do?
I'm hoping it will be easy to rewrite the Loci gui to the gms ADM-DFP communication
api and to replace Loci's Class-system by gms's MO's. As you can see on the gms site
my gui is quite simple, it uses listing instead of a nice canvas window. But you
already got
the graphics and code done, I wanna use that part.

| (BTW, are you subscribed to this list?)
yes :)

| Jarl, looking at GMS, it appears you have accomplished much of what we haven't
| even touched yet: Developing an API for getting independent programs to talk
| nice to each other and the core.
Version 0.4.0pre6 (allmost done) will have the final plugin-registration API. I still

need to spend much more time into the security\authentication code though.

| On the other hand, we have been concentrating on developing a nice GUI plus an
| API for getting several different GUIs to talk to our core.
Hmm.. sounds tempting :)
I didn't got the opportunity to have a detailed look at your sources, will do this
evening.

| Perhaps we can merge projects.  Brad (who has been developing our core) would
| kill me if we had to scrap Loci's own core, but why don't we take a look at
| each other's designs and see where we can meet?
It sounds like the logical thing to do, but there seems to be one mayor disign diff.
between
the two projects: the system's objects. Gms uses a 'situation-oriented' object, every
object
stores\processes\filters\etc data, depending on the function\situation of the object.
Loci
seems to use a 'function-orientation' why of implementing objects, a 'special'
store-object,
etc.
Why did you choose this approach? Or am I totally wrong in the way I think Loci
works?


| As I mentioned, Python is written in C and has language bindings to it, so we
| may be able to make a Python wrapper for your core that will allow us to make
| Python interfaces.  And not just GUI interfaces: We were planning on making a
| natural language interface that will let the user 'talk' to Loci.
Ic. nice feature. What do you think about my idead to attach a GAP to GMS which is
using
the regulair GUI\client API but is in fact a engine which detects high fitness
structures
and applies those to other instances of gms tunning around the inet?
This would solve the biggest problem that Loci and gms will have once they reach
production
phase, which is that 'regulair' user wont be able to configure the dataflow-diagram,
it would
just be too complicated for them.

| A C core would be faster, which is an advantage to us.  And Python interfaces
| are easier to make than those in C, which would be an advantage to you.
| What do you think?  Again, Loci has a very well thought out design, but I
| think we may be able to reach a middle ground.
One thing: what are Loci's? Are they binairies, or scripts? GMS uses dynamic loadable

binairies, for speed reasons mainly.




From jarl at casema.net  Mon Mar 13 09:59:02 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] loci] GMS
References: <200003131013.LAA06996@planetX.bastards.nl> <38CCDD84.840767F2@geoserve.net>
Message-ID: <38CD0236.D4AF67D6@casema.net>

| Hi Jarl!  It's good to hear from you.  I've been meaning to write to you
| since I saw the comment on Gnotices about GMS.  We (Loci) are talking with
| the Overflow people about a collaboration, and I thought perhaps GMS would
| like to join the party.


Hi again,
I've been searching a long time for similair projects and now there're two at once..
nice-n-sweet luxory. I've also checked out UML (mentioned on the news.gnome.org
discussion)
which has several interesting features. I recommend reading this, and in particulair
Booch.


>
> I just recenly rewrote the GUI to be an seperate application communicating trough
> Corba with the main engine, and the next phase is coding a better GUI.
> Now that I seem Loci I'm considering the possibilities of integrating the GUI into
> GMS. This will mean I have to port the python code to C.

| Hmmm.  I'm not sure what you mean by 'integrating the GUI'.  Are you
| interested in getting GMS to work with Loci's actual GUI, or are you looking
| to make something like it in C?
Yes and no.. a GUI for gms only needs to 'speak' to the CORBA api the 'core' exports
and must offer a corba api to the same core for displaying dialogs etc. (a gms
core-gui
couple has 2 corba communication lines) The speed and portability of C makes it a
good candidate, but probably Python will be good enough. As long as Python has good
gtk- and corba bindings.

> I want to know whether the Loci GUI is mainly finished, or should I wait till it's
> more mature,cq untill all planned features are done?

| The design/plan for the GUI is pretty well thought out.  We have only to put
| more of our ideas into code.  Again, what were you hoping to do?
I'm hoping it will be easy to rewrite the Loci gui to the gms ADM-DFP communication
api and to replace Loci's Class-system by gms's MO's. As you can see on the gms site
my gui is quite simple, it uses listing instead of a nice canvas window. But you
already got
the graphics and code done, I wanna use that part.

| (BTW, are you subscribed to this list?)
yes :)

| Jarl, looking at GMS, it appears you have accomplished much of what we haven't
| even touched yet: Developing an API for getting independent programs to talk
| nice to each other and the core.
Version 0.4.0pre6 (allmost done) will have the final plugin-registration API. I still

need to spend much more time into the security\authentication code though.

| On the other hand, we have been concentrating on developing a nice GUI plus an
| API for getting several different GUIs to talk to our core.
Hmm.. sounds tempting :)
I didn't got the opportunity to have a detailed look at your sources, will do this
evening.

| Perhaps we can merge projects.  Brad (who has been developing our core) would
| kill me if we had to scrap Loci's own core, but why don't we take a look at
| each other's designs and see where we can meet?
It sounds like the logical thing to do, but there seems to be one mayor disign diff.
between
the two projects: the system's objects. Gms uses a 'situation-oriented' object, every
object
stores\processes\filters\etc data, depending on the function\situation of the object.
Loci
seems to use a 'function-orientation' why of implementing objects, a 'special'
store-object,
etc.
Why did you choose this approach? Or am I totally wrong in the way I think Loci
works?


| As I mentioned, Python is written in C and has language bindings to it, so we
| may be able to make a Python wrapper for your core that will allow us to make
| Python interfaces.  And not just GUI interfaces: We were planning on making a
| natural language interface that will let the user 'talk' to Loci.
Ic. nice feature. What do you think about my idead to attach a GAP to GMS which is
using
the regulair GUI\client API but is in fact a engine which detects high fitness
structures
and applies those to other instances of gms tunning around the inet?
This would solve the biggest problem that Loci and gms will have once they reach
production
phase, which is that 'regulair' user wont be able to configure the dataflow-diagram,
it would
just be too complicated for them.

| A C core would be faster, which is an advantage to us.  And Python interfaces
| are easier to make than those in C, which would be an advantage to you.
| What do you think?  Again, Loci has a very well thought out design, but I
| think we may be able to reach a middle ground.
One thing: what are Loci's? Are they binairies, or scripts? GMS uses dynamic loadable

binairies, for speed reasons mainly.




From chapmanb at arches.uga.edu  Mon Mar 13 10:19:29 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
Message-ID: <200003131522.KAA154706@archa11.cc.uga.edu>

>> As I mentioned, Python is written in C and has language bindings to 
it, so
> we
>> may be able to make a Python wrapper for your core that will allow 
us to
 make
>> Python interfaces.
> 
> Oh, I'm forgetting that you're using CORBA.  There are ORBit 
bindings to
> Python (2 projects in fact).  So, maybe we can handle the GUI part 
of the
> project and you can handle the infrustructure.

I also had good luck making ORBit and Fnorb (an almost all python 
ORB implementation) talk to each other (with the stuff I did on the 
biopython project), so we shouldn't even have to wait for/develop 
python bindings for ORBit. We could start ASAP, provided Jarl is 
interested in trying to combine with us. From my quick glance at the 
GMS code, it looks like we both have a lot to offer each other, and I 
would definately be for merging the projects, if possible.
    Hopefully we will be able to find a compromise that lets Jarl keep 
creative freedom over the goals of GMS, and lets Loci take care of 
its objectives. I think we are too close in design goals to not try 
working together for a better overall final product. 

Brad





From chapmanb at arches.uga.edu  Mon Mar 13 10:46:36 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] Loci code
Message-ID: <200003131549.KAA88902@archa12.cc.uga.edu>

Hey Jarl;
    I just read your message and your mention of taking a look at the 
code. You might want to wait for a couple of days because we are in 
the middle of a major rewrite (we are separating the gui from the rest 
of the code) which is nearly done, but is not reflected very well in 
the snapshot. Hopefully we can then get a new snapshot you can take a 
look at that will give you a better estimation of the design of Loci.

Brad



From jarl at casema.net  Mon Mar 13 11:10:23 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu>
Message-ID: <38CD12EF.733DA69C@casema.net>

> >
> > Oh, I'm forgetting that you're using CORBA.  There are ORBit
> bindings to
> > Python (2 projects in fact).  So, maybe we can handle the GUI part
> of the
> > project and you can handle the infrustructure.
>
> I also had good luck making ORBit and Fnorb (an almost all python
> ORB implementation) talk to each other (with the stuff I did on the
> biopython project), so we shouldn't even have to wait for/develop
> python bindings for ORBit. We could start ASAP, provided Jarl is
> interested in trying to combine with us. From my quick glance at the
> GMS code, it looks like we both have a lot to offer each other, and I
> would definately be for merging the projects, if possible.
>     Hopefully we will be able to find a compromise that lets Jarl keep
> creative freedom over the goals of GMS, and lets Loci take care of
> its objectives. I think we are too close in design goals to not try
> working together for a better overall final product.
>

Sounds promising.. I'll try to understand the Loci sourcecode this
evening,
after which we (I?) can try some kinda merge 'pilot' to get a clear
picture
about the problems of a loci-gms marriage. I've been doing professional
porting last 2 years.. so I'll be able to see any problem quite quickly,
though
I've never used Python myself ;)

bye,
jarl



From jarl at casema.net  Mon Mar 13 11:12:21 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] Loci code
References: <200003131549.KAA88902@archa12.cc.uga.edu>
Message-ID: <38CD1365.2E5C5240@casema.net>

Brad Chapman wrote:

> Hey Jarl;
>     I just read your message and your mention of taking a look at the
> code. You might want to wait for a couple of days because we are in
> the middle of a major rewrite (we are separating the gui from the rest
> of the code) which is nearly done, but is not reflected very well in
> the snapshot. Hopefully we can then get a new snapshot you can take a
> look at that will give you a better estimation of the design of Loci.
>
> Brad
>

IC. This is good news, GMS is currently in the same phase, soon we'll
have
both projects with seperate GUI's and Core's.





From bizzaro at geoserve.net  Mon Mar 13 14:07:52 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu> <38CD12EF.733DA69C@casema.net>
Message-ID: <38CD3C88.B7A277ED@geoserve.net>

Jarl van Katwijk wrote:
> 
> Sounds promising.. I'll try to understand the Loci sourcecode this
> evening, after which we (I?) can try some kinda merge 'pilot' to
> get a clear picture about the problems of a loci-gms marriage. I've
> been doing professional porting last 2 years.. so I'll be able to
> see any problem quite quickly,

Jarl, right now Loci uses a data stream and XML tags to send messages between
GUI and core (an XML RPC of a sort).  This is my own personal design decision,
and it is very much incompatible with CORBA.  But, we have really only begun
to implement this API.  As Brad mentioned, we are trying to separate GUI from
core.  This is the immediate result of developing a functional API and is an
indication that we aren't married to it.

In short, I talked to Brad, and we're both willing to switch to whatever CORBA
interface GMS requires.

> though I've never used Python myself ;)

Once you try it, you'll like it.  It's a very well designed language, as I'm
sure you have heard.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Mon Mar 13 14:54:19 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131013.LAA06996@planetX.bastards.nl> <38CCDD84.840767F2@geoserve.net> <38CD01DF.8B11893@casema.net>
Message-ID: <38CD476B.BFC13AAE@geoserve.net>

Jarl van Katwijk wrote:
> 
> I've been searching a long time for similair projects and now there're two at once..
> nice-n-sweet luxory.

Yes.  We should both be in contact with the Overflow developers, as they are
willing to collaborate in certain areas.

> I'm hoping it will be easy to rewrite the Loci gui to the gms ADM-DFP communication
> api and to replace Loci's Class-system by gms's MO's. As you can see on the gms site
> my gui is quite simple, it uses listing instead of a nice canvas window. But you
> already got the graphics and code done, I wanna use that part.

There are many features that a Loci-like GUI can bring to the project,
features that go well beyond messaging.  If you look through our mailing list
archives:

    http://bioinformatics.org/pipermail/pipet-devel/

>from the last few months, you will see some discussion of these.  One
interesting idea is to use graphical pipes to 'construct a command-line'
statement.  Another is to build a filesystem browser by wrapping some UNIX
utilities.

> | On the other hand, we have been concentrating on developing a nice GUI plus an
> | API for getting several different GUIs to talk to our core.
> Hmm.. sounds tempting :)
> I didn't got the opportunity to have a detailed look at your sources, will do this
> evening.

Again, this has to do with data streaming.

> It sounds like the logical thing to do, but there seems to be one mayor disign diff.
> between the two projects: the system's objects. Gms uses a 'situation-oriented'
> object, every object stores\processes\filters\etc data, depending on the
> function\situation of the object. Loci seems to use a 'function-orientation' why of
> implementing objects, a 'special' store-object, etc. Why did you choose this
> approach? Or am I totally wrong in the way I think Loci works?

I'm not so sure there is a difference.  You're saying that GMS will _either_
store, process, or filter data, depending on the situation, correct?  But does
each object have _all_ of these capabilities?

With Loci, we were trying to classify objects according to function:

    * Executables process data
    * Files store data
    * Directories store files and executables
    * Viewers/visualizers display data
    * Converters/filters change data format or protocol
    * Flow controllers...control data flow

But I wasn't considering the real needs of the core when defining these
things.  Remember, we haven't even looked into what GMS has already
accomplished.  I suppose it may be that each of these has storage
requirements, etc. that may not be apparent to me.

Loci was being built from the GUI on back.  I think, as a result, we have come
up with some good ideas.  GMS seems to have been built with more emphasis on
the infrastructure.  Again, our projects are very complementary.

> Ic. nice feature. What do you think about my idead to attach a GAP to GMS which is
> using the regulair GUI\client API but is in fact a engine which detects high
> fitness structures and applies those to other instances of gms tunning around the
> inet? This would solve the biggest problem that Loci and gms will have once they
> reach production phase, which is that 'regulair' user wont be able to configure
> the dataflow-diagram, it would just be too complicated for them.

I want to look into this and get Brad's opinion, since I'm not entirely sure
what you're talking about.  We have looked into issues of node configuration
with the realization that some 'programs' or constructs built for this system
will be complex.  I think Loci solves much of this problem by saving
constructs, what we call 'Work Flow Diagrams', as XML.  In fact, we are using
XML as a descriptive meta-data for all of our nodes.  This allows Loci to
combine and recombine nodes and save the constructs.  Have you seen some of
our discussion threads about this?

> One thing: what are Loci's? Are they binairies, or scripts? GMS uses dynamic loadable
> binairies, for speed reasons mainly.

Well, I suppose you're asking about the wrappers and not the programs.  Again,
we haven't come up with a standard API for doing this.  What we have done uses
some Python and some XML.  You're absolutely right that C is faster, but
running programs across the Internet tends to take away any advantage that may
give.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Mon Mar 13 14:59:52 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu>
Message-ID: <38CD48B8.3F261420@geoserve.net>

Brad Chapman wrote:
> 
>     Hopefully we will be able to find a compromise that lets Jarl keep
> creative freedom over the goals of GMS, and lets Loci take care of
> its objectives. I think we are too close in design goals to not try
> working together for a better overall final product.

Once again, there are many, many, many ideas we have that came from years of
thought and discussion.  I don't want to throw any of that out but see how we
can implement them in a Loci/GMS system.  We need to have some serious
discussions with Jarl, Jean-Marc and Dominic about all of these ideas.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Mon Mar 13 15:06:38 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu> <38CD48B8.3F261420@geoserve.net>
Message-ID: <38CD4A4E.D85CBD1A@casema.net>

> >
> >     Hopefully we will be able to find a compromise that lets Jarl keep
> > creative freedom over the goals of GMS, and lets Loci take care of
> > its objectives. I think we are too close in design goals to not try
> > working together for a better overall final product.
>
> Once again, there are many, many, many ideas we have that came from years of
> thought and discussion.  I don't want to throw any of that out but see how we
> can implement them in a Loci/GMS system.

Ofcourse, the result should only get better out of a merge.

>  We need to have some serious
> discussions with Jarl, Jean-Marc and Dominic about all of these ideas.
>

As I mentioned earlier: the first subject of discussion should be the differance

between the classes\objects 'filosophy' of both systems.. I think your choose
to implement different types based upon functionality isn't the way it should be

done. I'm supporting the 'one global do-everything object".

I'm very curious why Loci classes are implemented the way they are...

bye,
jarl



From bizzaro at geoserve.net  Mon Mar 13 15:23:22 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu> <38CD48B8.3F261420@geoserve.net> <38CD4A4E.D85CBD1A@casema.net>
Message-ID: <38CD4E3A.5738D670@geoserve.net>

jarl van katwijk wrote:
> 
> As I mentioned earlier: the first subject of discussion should be the differance
> between the classes\objects 'filosophy' of both systems.. I think your choose
> to implement different types based upon functionality isn't the way it should be
> done. I'm supporting the 'one global do-everything object".
> 
> I'm very curious why Loci classes are implemented the way they are...

Perhaps you can give us a synopsis/summary of the GMS object philosophy.  It
would then be easier for me to point out differences, because right now I'm
not so sure how GMS is different.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Mon Mar 13 15:23:36 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131013.LAA06996@planetX.bastards.nl> <38CCDD84.840767F2@geoserve.net> <38CD01DF.8B11893@casema.net> <38CD476B.BFC13AAE@geoserve.net>
Message-ID: <38CD4E48.F4139653@casema.net>

>
> Yes.  We should both be in contact with the Overflow developers, as they are
> willing to collaborate in certain areas.

Sure, they seem to have done some work for high performance processing, something I like
to have a good look at.
I also liked seeing they are doing a speech recognision system.. I've already implemented
festival (speech synthesis), and was looking at IBM's viavoice, but an open source speech
recognision system is much better.

>
>
> > I'm hoping it will be easy to rewrite the Loci gui to the gms ADM-DFP communication
> > api and to replace Loci's Class-system by gms's MO's. As you can see on the gms site
> > my gui is quite simple, it uses listing instead of a nice canvas window. But you
> > already got the graphics and code done, I wanna use that part.
>
> There are many features that a Loci-like GUI can bring to the project,
> features that go well beyond messaging.  If you look through our mailing list
> archives:
>
>     http://bioinformatics.org/pipermail/pipet-devel/
>
> from the last few months, you will see some discussion of these.  One
> interesting idea is to use graphical pipes to 'construct a command-line'
> statement.  Another is to build a filesystem browser by wrapping some UNIX
> utilities.
>

I've downloaded the complete archive yesterday.. will read them

>
> > | On the other hand, we have been concentrating on developing a nice GUI plus an
> > | API for getting several different GUIs to talk to our core.
> > Hmm.. sounds tempting :)
> > I didn't got the opportunity to have a detailed look at your sources, will do this
> > evening.
>
> Again, this has to do with data streaming.
>
> > It sounds like the logical thing to do, but there seems to be one mayor disign diff.
> > between the two projects: the system's objects. Gms uses a 'situation-oriented'
> > object, every object stores\processes\filters\etc data, depending on the
> > function\situation of the object. Loci seems to use a 'function-orientation' why of
> > implementing objects, a 'special' store-object, etc. Why did you choose this
> > approach? Or am I totally wrong in the way I think Loci works?
>
> I'm not so sure there is a difference.  You're saying that GMS will _either_
> store, process, or filter data, depending on the situation, correct?  But does
> each object have _all_ of these capabilities?

Yes, look at the messaging_mobject_structs.h file. Every object has a 'procedure' that can
mutate the data, it has a history-file is which data is stored (upto 32 entries in memory,
beyond this number on disk), etc. Look for yourself :)


>
>
> With Loci, we were trying to classify objects according to function:
>
>     * Executables process data
>     * Files store data
>     * Directories store files and executables
>     * Viewers/visualizers display data
>     * Converters/filters change data format or protocol
>     * Flow controllers...control data flow
>
> But I wasn't considering the real needs of the core when defining these
> things.  Remember, we haven't even looked into what GMS has already
> accomplished.  I suppose it may be that each of these has storage
> requirements, etc. that may not be apparent to me.

Yes, I have these 'types':
    Sensors:     objects that gather datastreams from the outside world into the pipeline
    Collectors: objects that gather datastream outof the pipeline
    Filters:         objects that have no special situation, they are suppost to prosess
data, filter, route, etc.
    Visuals:        object that output datastreams from the system, display it, etc.


> > Ic. nice feature. What do you think about my idead to attach a GAP to GMS which is
> > using the regulair GUI\client API but is in fact a engine which detects high
> > fitness structures and applies those to other instances of gms tunning around the
> > inet? This would solve the biggest problem that Loci and gms will have once they
> > reach production phase, which is that 'regulair' user wont be able to configure
> > the dataflow-diagram, it would just be too complicated for them.
>
> I want to look into this and get Brad's opinion, since I'm not entirely sure
> what you're talking about.  We have looked into issues of node configuration
> with the realization that some 'programs' or constructs built for this system
> will be complex.  I think Loci solves much of this problem by saving
> constructs, what we call 'Work Flow Diagrams', as XML.  In fact, we are using
> XML as a descriptive meta-data for all of our nodes.  This allows Loci to
> combine and recombine nodes and save the constructs.  Have you seen some of
> our discussion threads about this?

Not yet, but I'm sure noirmal desktop users do not want to program\configure their system.
They exspect the OS to 'know' what to do with data(-streams). But again: I will read the
mailing archive :)

>
> > One thing: what are Loci's? Are they binairies, or scripts? GMS uses dynamic loadable
> > binairies, for speed reasons mainly.
>
> Well, I suppose you're asking about the wrappers and not the programs.  Again,
> we haven't come up with a standard API for doing this.  What we have done uses
> some Python and some XML.  You're absolutely right that C is faster, but
> running programs across the Internet tends to take away any advantage that may
> give.

The plugins are loaded locally, communication between different instances of gms will take
place tru a sensor\visual pair. But having some scripting\XML API will surely be usefull
somehow.

bye,
jarl



From jarl at casema.net  Mon Mar 13 15:27:50 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu> <38CD12EF.733DA69C@casema.net> <38CD3C88.B7A277ED@geoserve.net>
Message-ID: <38CD4F46.9D652E9F@casema.net>

>
>
> Jarl, right now Loci uses a data stream and XML tags to send messages between
> GUI and core (an XML RPC of a sort).  This is my own personal design decision,
> and it is very much incompatible with CORBA.  But, we have really only begun
> to implement this API.  As Brad mentioned, we are trying to separate GUI from
> core.  This is the immediate result of developing a functional API and is an
> indication that we aren't married to it.
>
> In short, I talked to Brad, and we're both willing to switch to whatever CORBA
> interface GMS requires.

Don't rush :)
After you've looked at the gms api you'll notice it isn't complete. There many
features I hadn;t made my mind up about, i'm only hoping Loci can fill in thesse
gaps.




From jarl at casema.net  Mon Mar 13 15:44:34 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu> <38CD48B8.3F261420@geoserve.net> <38CD4A4E.D85CBD1A@casema.net> <38CD4E3A.5738D670@geoserve.net>
Message-ID: <38CD5332.75EC8614@casema.net>

> >
> > As I mentioned earlier: the first subject of discussion should be the differance
> > between the classes\objects 'filosophy' of both systems.. I think your choose
> > to implement different types based upon functionality isn't the way it should be
> > done. I'm supporting the 'one global do-everything object".
> >
> > I'm very curious why Loci classes are implemented the way they are...
>
> Perhaps you can give us a synopsis/summary of the GMS object philosophy.  It
> would then be easier for me to point out differences, because right now I'm
> not so sure how GMS is different.
>

(Please first do a 'lpr messaging_mobject_structs.h' and have a look at the
srtuctures...)

Every MO has the same structure, features, fields and capabilities. This is done
because I want to have high flexibility, stability (no need for different processing
handlers), extendability (missing features can be added 'easely'),  and analizability
(the GAP core will requier a single type based database structure).

This setup also makes it possible to realise a setup like the one Loci can offer,
simply by limiting MO's in their functionality.



From justin at ukans.edu  Mon Mar 13 21:14:00 2000
From: justin at ukans.edu (Justin Bradford)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] orbit/python bindings (was Re: [Pipet Devel] GMS)
In-Reply-To: <38CD000F.1E2F94F2@geoserve.net>
Message-ID: <Pine.OSF.4.03.10003132001090.19761-100000@busboy.sped.ukans.edu>

On Mon, 13 Mar 2000, J.W. Bizzaro wrote:

> Oh, I'm forgetting that you're using CORBA.  There are ORBit bindings to
> Python (2 projects in fact).  So, maybe we can handle the GUI part of the
> project and you can handle the infrustructure.

Let's be honest. There are two codebases for Python bindings to ORBit, but
really only one (active) project: orbit-python

I've been talking with Jason Tackaberry (the guy working on orbit-python)
and he plans to continue active development and maintainence of his
project beyond his class project (where orbit-python originated).

He supports dynamically loaded IDLs, already, and plans to implement
support for the Interface Repository. These were, by far, the most
commonly requested new features for PyORBit. PyORBit has an IDL compiler
which generates a Python module for a specifc IDL. This likely has
performance benefits over the dynamic method, but it's a lot more
cumbersome.

So I recommend that loci uses orbit-python. When I do get a little more
free time, I'd rather work on loci than the bindings, anyway.

Justin





From bizzaro at geoserve.net  Mon Mar 13 21:30:11 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:20 2006
Subject: [Pipet Devel] orbit/python bindings (was Re: [Pipet Devel] GMS)
References: <Pine.OSF.4.03.10003132001090.19761-100000@busboy.sped.ukans.edu>
Message-ID: <38CDA433.84FBE98B@geoserve.net>

Justin Bradford wrote:
> 
> So I recommend that loci uses orbit-python. When I do get a little more
> free time, I'd rather work on loci than the bindings, anyway.

Hey Justin!  Thanks for the tip.

BTW, do you know of anyone working on Python bindings to Bonobo?  I saw Miguel
de Icaza speek last week and give a demo of Bonobo.  He showed an 'SVG Bonobo
component'.  I think this would be great to use for visuals, but we need the
Python bindings to do it.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Mon Mar 13 21:47:12 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] [Fwd: Overflow/Loci/GMS collaboration]
Message-ID: <38CDA830.17908993@geoserve.net>

>From Jean-Marc...
-------------- next part --------------
An embedded message was scrubbed...
From: Jean-Marc Valin <jean-marc.valin@hermes.usherb.ca>
Subject: Re: Overflow/Loci/GMS collaboration
Date: Mon, 13 Mar 2000 15:43:34 -0500
Size: 5025
Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000314/b30d2585/attachment.mht
From bizzaro at geoserve.net  Mon Mar 13 21:51:52 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu> <38CD48B8.3F261420@geoserve.net> <38CD4A4E.D85CBD1A@casema.net> <38CD4E3A.5738D670@geoserve.net> <38CD5332.75EC8614@casema.net>
Message-ID: <38CDA948.420012DE@geoserve.net>

jarl van katwijk wrote:
> 
> (Please first do a 'lpr messaging_mobject_structs.h' and have a look at the
> srtuctures...)

I'll save everyone else the trouble and attach the file.

Jeff
-------------- next part --------------
/*
 * Messaging Object 
 * this is the structure in which data resides inside the
 * messaging system, NO other structure will be used
 *
 * begin : 19-6-1999, BastardAI
 *          8-7-1999, Note Java-varinaming style
 *
 */

#ifndef MOSTRUCT
#define MOSTRUCT
#include <gnome.h>
#include <orb/orbit.h>
#include "messaging_mobject_constants.h"

typedef struct _messaging_data_object messaging_data_object;
typedef struct _messaging_object messaging_object;
typedef struct _messaging_relation messaging_relation;

struct _messaging_data_object {
	gint			name;				/* runtime name\number\key\id */
	gchar			*rawdata;			/* raw data */
	gint			rawdataLen;			/* length of rawdata memory block */
	GString         *String;			/* string data representation */
										/* TODO: String -> non-pointer */
	gint			Integer;			/* Integer data representation */
	gint			Percentage;			/* Percentage data representation */
	gboolean		Boolean;			/* Boolean data representation */
					/* NOTE: The unique data type equals the rawdata */
};

struct _messaging_relation {
	messaging_object	*MO; 	/* Other MO */
	rType_t				rType;	/* Relation description */
};

struct _messaging_object {
	gint 			name;   	  	 	/* runtime name\number\key\id */
	messaging_data_object *MDO;			/* data block of current MO */
	gint            dataSequence;		/* followup number of data block ,
	                                       N = first; N--; 0 = last */
    GString         *dataDescription;   /* Description of data block */
    gboolean        dataPassedOver;     /* db internal detection dead ends */
	gint			dataBlocked;		/* current amount of cached data */
	messaging_data_object *dataBlockedList[32];/* list of blocked data chunks */
	GString			*historyFilename;	/* Filename of .. */
	GIOChannel		*historyIO;			/* .. logfile after dataBlockedList */
	GString			*dname;		   		/* displayed name */
	GString         *dDescription; 		/* displayed description */
	moType_t		moType;		   		/* messaging_object type */
	gint			currentRelations;	/* N of current relations */
	messaging_relation	*relation[256];	/* other MObjects to which current
											one has some sort of relation to.
					   TODO: N=256=fixed, hmmm, a llist should replace ?? */
	GString			*allowedFilter;		/* This string must be matched by
										   offered data in order to accept it */
	gint			(*procedure)(messaging_object *MO);	
										/* operation on data by current mobject */
	statusType_t	status; 			/* status */
	gpointer		configInit;			/* initialize configuration of MO */
	gpointer		configRuntime;		/* runtime configuration of MO */
	gboolean		UIsetup;			/* FALSE until fully setup, see NOTE1 below */
	//GtkWidget*		visualPlug;			/* Socket that holds the visualPanel, also NOTE2 */
	GtkWidget*		(*visualPlug_buildProcedure)(messaging_object *MO,gint XID); /* builds visual UI */
	GtkWidget*		(*configPlug_buildProcedure)(messaging_object *MO,gint XID); /* builds config UI */
	//GtkWidget*		configPlug;			/* Holds the gtkplug widget of the config panel */
	gint			(*init)(messaging_object *MO); /* initialize code */
	gint			(*destruct)(messaging_object *MO); /* destroy code */
};

/* NOTE1 : plugins can be registerd to a ADM, the uisetup field is true after:
			1- the mo is registered to the dfp
			2- if moType==VisualMO, the mo visualSocket must be registered to the ADM
				(= being != NULL)
	Registering to a ADM is not needed for a MO to be operational, but as long as 
	UIsetup == FALSE the MO cannot be manually configured and cannot output information.
	This is only very important to VisualMO's :) */

/* NOTE2 : These sockets are pointers to widgets OUTSIDE the DFP\Plugin data block. These
	are ONLY used by the ADM, which requests the values and uses it locally. The GtkWidget
	wont be a GtkSocket, but the GUI structure that contains the socket */

#endif
From bizzaro at geoserve.net  Mon Mar 13 22:08:10 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Overflow/Messaging/Loci alliance mailing list
Message-ID: <38CDAD1A.78B3BC52@geoserve.net>

Jean-Marc and I agree that there should be a separate mailing list to discuss
the collaboration between our three projects.  I can set that up here at The
Open Lab.  I can provide any service in fact: CVS, Web, FTP, etc.

The only problem is, what do we call the list?  :-)  The question really is,
what do we call the collaboration?  Overflow/Messaging/Loci is rather long. 
'OML' is rather meaningless.

How about 'Data' or 'Datum'.  Each project could then use the word as a
prefix, like...

    Data-Overflow
    Data-Messaging
    Data-Loci

?

Any other suggestions?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Mon Mar 13 22:46:00 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca>
Message-ID: <38CDB5F8.90F6A49@geoserve.net>

Jean-Marc Valin wrote:
> 
> There's another option that might be worth considering: "plugging" the GMS (or
> Loci) core on top of the Overflow core. I don't think it would be that hard,
> since we already have a node that can exec an executable and return it's stdout.

That's along the line of what GMS has accomplished and Loci has been trying to
accomplish.  This is something we'll need to discuss since we have 3 ideas
about how this can be done.

But if Overflow were to be a node in GMS, Overflow would have to handle input
from and output to the outside world.  I imagine Overflow might need a special
node for doing that.  An idea that we had for Loci, was that unconnected lines
on a workspace would represent data that would flow to and from the outside. 
And since Loci was going to nest workspace in a workspace in a workspace,
etc., and each nested workspace is itself a node, unconnected lines would mean
data flows to a parent.  I'd like to see us do that in our collaboration.

> A nice feature we have (I didn't see it in Loci/GMS, but maybe it's there too)
> is "subnets" and "iterators", which allow the C equivalent of functions
> (reusable in any "program") and loops. For example, this allows me "for each
> file in a list", "for each 10 ms frame in the file", to calculate a
> powerspectrum.

Loci was going to (again, much of the design hasn't been implemented)
represent connected nodes as linked XML.  The linked XML would then be parsed
by an interpreter, which would allow for data flow controls (loops, etc.) to
be inserted.  The only thing we haven't figured out is how to determine the
conditions for flow control (e.g., what data is used with an if-then
control?).  Remember, Loci wanted to be able to deal with any type of data.

> Another consideration, besides the GUI and the core, is the object system. In
> Overflow, all nodes return a pointer (reference counted) to an Object, which is
> the base class for all other types (Int, String, Vector, ...). ?ll these types
> can be saved/parsed to file. Do you have something similar in Loci, or are you
> using files without caring for the content of the file? (or something else)

Loci wasn't going to care about the content of the file.  We were going to use
XML meta-data to accompany each node, and that could be saved to a file.  I'd
like to hear what Jarl has to say about this and how GMS does it.

> As for the GUI, I don't think it's a priority for now... we might end up
> rewriting one that best handles the new core (if we merge the 3 projects). Or we
> can always keep 3 different GUIs that use the same core differently...

Well, I okayed this with our one other dedicated coder (Brad), and we want
Loci to provide a consistent (and very cool, I might add) GUI across the
projects.  So, we'll back off of the infrastucture coding and dedicate
ourselves to user interface and general design issues.

> Last thing, let's see how much time we have. For the next year, I'm gonna code a
> lot for Overflow, since I use it for my master (in speech processing). What
> about you and the GMS guys?

Oh, I just got my Master's and won't be back in school for the Ph.D. program
until September.  So, I have lots of time for the next 6 months.  And, if I
may speak for Brad, he wants to use such an application for his thesis work,
so he's very anxious to keep things moving.  From Loci's point-of-view,
merging or collaborating with GMS and Overflow could take away a year or more
of development time (Plus, otherwise, all 3 projects would have a great deal
of redundency between them).

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Mon Mar 13 23:23:35 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net>
Message-ID: <38CDBEC7.6C48A3C7@hermes.usherb.ca>

> That's along the line of what GMS has accomplished and Loci has been trying to
> accomplish.  This is something we'll need to discuss since we have 3 ideas
> about how this can be done.
> 
> But if Overflow were to be a node in GMS, Overflow would have to handle input
> from and output to the outside world.  I imagine Overflow might need a special
> node for doing that.  An idea that we had for Loci, was that unconnected lines
> on a workspace would represent data that would flow to and from the outside.
> And since Loci was going to nest workspace in a workspace in a workspace,
> etc., and each nested workspace is itself a node, unconnected lines would mean
> data flows to a parent.  I'd like to see us do that in our collaboration.

The reason I was suggesting the Overflow core is that we already handle IO with
files and pipes, so Overflow nodes can also be executables (it's tested and it
works). From what I know now, I see two options:
1) Wrap all Overflow as a Loki/GMS node
2) Use Overflow's core and implement the missing functionnalities.

I'd personnaly prefer 2) but the main question is whether the design we have can
handle the new features from Loci/GMS. I think it is general enough but again,
you might prove me wrong. Can you try find out things in Loci you don't think
Overflow can do, and I'll try to see whether that's the case. As for 1) I'm sure
you can already do it with what you have. Simply edit an Overflow document that
takes its inputs from a file and sends its output to another file (or stdout)
and you're done.

I forgot... another thing we have in Overflow that's essential (well, for what I
do with it) is the ability to "automatically" cache the results that will be
reused in another iteration of a loop or by another node, so nothing gets
computed twice.

I don't know whether that was your original goal, but what we're about to do by
merging the 3 proejcts is this: a graphical shell (we could call it vsh, as in
visual shell). Though with Overflow, it would also be a scientific application
(the original Overflow goal was to be a "visual Matlab").

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Tue Mar 14 01:16:27 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDBEC7.6C48A3C7@hermes.usherb.ca>
Message-ID: <38CDD93B.762E508F@geoserve.net>

Jean-Marc Valin wrote:
> 
> The reason I was suggesting the Overflow core is that we already handle IO with
> files and pipes, so Overflow nodes can also be executables (it's tested and it
> works). From what I know now, I see two options:
> 1) Wrap all Overflow as a Loki/GMS node
> 2) Use Overflow's core and implement the missing functionnalities.

If we chose 2), what would we use GMS's core for?  I'd like to hear Jarl's
thoughts on Overflow's core.  GMS may provide the missing functionality for
Overflow, and merging C and C++ shouldn't be all THAT difficult.  I was
thinking about 1), but sure, we should consider 2) as well.

> I'd personnaly prefer 2) but the main question is whether the design we have can
> handle the new features from Loci/GMS. I think it is general enough but again,
> you might prove me wrong. Can you try find out things in Loci you don't think
> Overflow can do, and I'll try to see whether that's the case.

Okay, here's one: Overflow runs from a batch file created by the GUI, right? 
Loci's design (and I think GMS's too) lets the core work while the user
continues to operate the GUI.  This is necessary for Internet-distributed
nodes running on large datasets, because there can be a very long delay before
anything is returned.

So, Loci would allow the user to execute a node and THEN start adding
connections...or even change connections at a later time.  Loci does this by
making a 'dynamic batch file', which really has to be a database (since we are
using XML, we were going to use an XML database).

How does Overflow start an executable as a node?  You have to feed it a
command-line statement, right?  I mentioned already that I have a neat way to
generate a command-line statement by connecting nodes :-)  GMS/Jarl seems to
have developed a good system for wrapping executables.  Have you seen it? 
That's one way I think GMS complements Overflow, another is GMS's use of
CORBA.

I like how Overflow uses XML.  Looking at some of your XML files, it seems we
were thinking along the same line.  For example, here is a Loci XML script
that tells the core what connections to make ('front' is the GUI and 'middle'
is the core):

----------------------------------------------------

<front>
  <add_workspace>
    <workspace id='' >
    </workspace>
  </add_workspace>
</front>
<middle>
  <success>
    <workspace id='workspace1' >
    </workspace>
  </success>
</middle><!--mouse click at 101.0 83.0 -->
<!--clicked on workspace -->
<front>
  <add>
    <workspace id='workspace1' >
    </workspace>
    <locus type='document' >
    </locus>
  </add>
</front>
<middle>
  <success>
    <locus id='workspace1/document1.xml'  num_outputs='1' 
gui_name='file_select'  num_inputs='1' >
    </locus>
  </success>
</middle><!--clicked on document -->
<!--was workspace now document -->
<!--mouse click at 184.0 73.0 -->
<!--clicked on workspace -->
<!--was document now workspace -->
<front>
  <add>
    <workspace id='workspace1' >
    </workspace>
    <locus type='viewer' >
    </locus>
  </add>
</front>
<middle>
  <success>
    <locus id='workspace1/viewer1.xml'  num_outputs='0'  gui_name='text_box' 
num_inputs='1' >
    </locus>
  </success>
</middle><!--clicked on viewer -->
<!--was workspace now viewer -->
<!--mouse click at 182.0 76.0 -->
<!--clicked on viewer -->
<!--locus dropped at 259.0 116.0 -->
<!--mouse click at 100.0 184.0 -->
<!--clicked on document -->
<!--was viewer now document -->
<!--dot group xy 259.0 212.0 -->
<!--dot center world xy 259.0 212.0 -->
<!--dropped locus output line # 0 -->
<!--dropped line on mate viewer -->
<!--dropped on mate input line # 0 -->
<front>
  <connect>
    <input id='workspace1/viewer1.xml' >
    </input>
    <output id='workspace1/document1.xml' >
    </output>
  </connect>
</front>
<middle>
  <success>
  </success>
</middle><!--connecting locus document line # 0 to mate viewer line # 0 -->
<!--locus position 101.0 83.0 -->
<!--mate position 259.0 116.0 -->
<!--half_pos 180.0 99.5 -->
<!--dots xy recorded 101.0 183.0 -->
<!--locus linemovexy -79.0 -112.5 -->
<!--dots xy recorded 184.0 173.0 -->
<!--mate linemovexy -79.0 -116.5 -->

----------------------------------------------------

And then the core/middle uses that to modify the dynamic batch file or
database.  Here's the 'document' node:

----------------------------------------------------

<loci type='document' >

  <widget>file_select</widget>

  <input xml:link='simple'  href='baselocus.xml' >
  </input>
  <output xml:link='simple'  href='workspace1/viewer1.xml' >
  </output>

</loci>

----------------------------------------------------

and here is the 'viewer' node:

----------------------------------------------------

<loci type='viewer' >

  <widget>text_box</widget>

  <input xml:link='simple'  href='workspace1/document1.xml' >
  </input>

</loci>

----------------------------------------------------

I think there's a lot we can collaborate on regarding XML.

> As for 1) I'm sure
> you can already do it with what you have. Simply edit an Overflow document that
> takes its inputs from a file and sends its output to another file (or stdout)
> and you're done.

I guess what we end up doing depends upon how much you and Dominic want to
change in Overflow.  As for 1), this requires the least amount of work for you
guys.  But if you're willing to take on new features and paradigms, I'm
thrilled to hear it :-)

> I forgot... another thing we have in Overflow that's essential (well, for what I
> do with it) is the ability to "automatically" cache the results that will be
> reused in another iteration of a loop or by another node, so nothing gets
> computed twice.

I haven't even thought about that, since we haven't developed any flow control
nodes for Loci.  Loci does, however, always keep reference to the location of
a block of data (often kept as 'document' nodes) so we probably wouldn't have
thrown that stuff away.

Speaking of 'references', an important consideration we made when designing
Loci is that it's important not to transfer everything everytime.  To solve
this, we keep a URI to programs, etc. and an XML REPRESENTATION of these
things.  In most cases, only the representation needs to be transfered to the
local computer.

> I don't know whether that was your original goal, but what we're about to do by
> merging the 3 proejcts is this: a graphical shell (we could call it vsh, as in
> visual shell). Though with Overflow, it would also be a scientific application
> (the original Overflow goal was to be a "visual Matlab").

I like the sound of that.  You know, Loci uses icons to represent nodes, and
some of the icons (will) look like those seen on a GUI desktop: folders,
trash, documents, etc.  And we're following the Xerox/Apple model in many
cases: Drag-and-drop a node from a folder onto the desktop, double-click on an
icon to open a windowlet (little window that stays with the icon and in Loci),
etc.  PLUS WE HAVE A PLAN FOR GRAPHICAL THEMES: The user can choose desktop
colors (and wallpaper when the canvas gets that item), icon styles, symbol
(goes under the icon to represent node class) styles that can be
mixed-and-matched with icons, plus Gtk themes that can be applied to Loci
widgets alone.  The way I planned it, Loci COULD substitute for a user's
desktop.

As for a scientific application, Brad, myself and many others involved with
Loci are computational biologists who hope to use this for scientific
research.  Hmmmm....There really are many, many more ideas that we've had, but
we'll get to those in time :-)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From gvd at redpoll.pharmacy.ualberta.ca  Tue Mar 14 01:27:54 2000
From: gvd at redpoll.pharmacy.ualberta.ca (Gary Van Domselaar)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Overflow/Messaging/Loci alliance mailing list
References: <38CDAD1A.78B3BC52@geoserve.net>
Message-ID: <38CDDBEA.566BEB9F@redpoll.pharmacy.ualberta.ca>

"J.W. Bizzaro" wrote:
> 
> Jean-Marc and I agree that there should be a separate mailing list to discuss
> the collaboration between our three projects.  I can set that up here at The
> Open Lab.  I can provide any service in fact: CVS, Web, FTP, etc.
> 
> The only problem is, what do we call the list?  :-)  The question really is,
> what do we call the collaboration?  Overflow/Messaging/Loci is rather long.
> 'OML' is rather meaningless.
> 
> How about 'Data' or 'Datum'.  Each project could then use the word as a
> prefix, like...
> 
>     Data-Overflow
>     Data-Messaging
>     Data-Loci
> 
> ?
> 
> Any other suggestions?

How about 'The Data Pipe Alliance'?  Sounds professional and cool.
Whatever you decide I can set up on the forthcoming development
environment for bioinformatics.org.

Regards,
g.
-- 
                                  Gary Van Domselaar
                               gary@bioinformatics.org
                          http://www.bioinformatics.org/~gary
                ----------------------------------------------------
                           bioinformatics.org: The Open Lab
                            http://www.bioinformatics.org/


From bizzaro at geoserve.net  Tue Mar 14 02:06:42 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDBEC7.6C48A3C7@hermes.usherb.ca>
Message-ID: <38CDE502.BB358A2E@geoserve.net>

Jean-Marc Valin wrote:
> 
> (we could call it vsh, as in visual shell).

It took me a while to find it, but VSH is an ancient file manager for UNIX:

    http://www.cs.indiana.edu/hyplan/kinzler/home/doc/vsh.txt

It was originally written in 1983 (!), ported to Linux in 1992, and not
touched since 1994.  It's small, but I couldn't get it to compile.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Tue Mar 14 02:02:55 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net>
 <38CDBEC7.6C48A3C7@hermes.usherb.ca> <38CDD93B.762E508F@geoserve.net>
Message-ID: <38CDE41F.5A263F64@hermes.usherb.ca>

"J.W. Bizzaro" a ?crit :
> > 1) Wrap all Overflow as a Loki/GMS node
> > 2) Use Overflow's core and implement the missing functionnalities.
> 
> If we chose 2), what would we use GMS's core for?  I'd like to hear Jarl's
> thoughts on Overflow's core.  GMS may provide the missing functionality for
> Overflow, and merging C and C++ shouldn't be all THAT difficult.  I was
> thinking about 1), but sure, we should consider 2) as well.

OK, let's talk about 2) (I consider 1 to be trivial). All our nodes are classes
and the code that's ran is in the getOutput method. Maybe it's possible to call
the GMS functions from there. Again, the main reason I'd like 2) is speed: not
having to call an executable for a sum, logical and, even an FFT. 

> Okay, here's one: Overflow runs from a batch file created by the GUI, right?

No (well you can do it, but you don't have to). When you use vflow to run the
"program", vflow starts another thread, without needing to write XML to disk.
You can modify the program while it's running, but it won't affect it until you
restart it. I think it would be technically feasable to have the modifications
take effect right now, by having a special node that "encapsulates" a Network
(we already have what it takes to do that). That is, in Overflow a Network (set
of connected nodes) is a subclass of the Node class (and thus can be used as a
simple node). 

> Loci's design (and I think GMS's too) lets the core work while the user
> continues to operate the GUI.  This is necessary for Internet-distributed
> nodes running on large datasets, because there can be a very long delay before
> anything is returned.


> So, Loci would allow the user to execute a node and THEN start adding
> connections...or even change connections at a later time.  Loci does this by
> making a 'dynamic batch file', which really has to be a database (since we are
> using XML, we were going to use an XML database).

I'm not sure I understand what you mean here?

> How does Overflow start an executable as a node?  You have to feed it a
> command-line statement, right?  I mentioned already that I have a neat way to
> generate a command-line statement by connecting nodes :-)  GMS/Jarl seems to
> have developed a good system for wrapping executables.  Have you seen it?
> That's one way I think GMS complements Overflow, another is GMS's use of
> CORBA.

You can have a set of nodes that generate a command line in Overflow, then feed
that command-line (string) to the "Exec" node.

> I like how Overflow uses XML.  Looking at some of your XML files, it seems we
> were thinking along the same line.  For example, here is a Loci XML script
> that tells the core what connections to make ('front' is the GUI and 'middle'
> is the core):

Is the "program" in Loci stored as a script? The way it works in Overflow, is
that the XML is simply storing a set of objects: Networks, Nodes and Links. It
stores how the nodes are connected together (XML was just a convenient way to
serialize our objects). Another thing you need to know about Overflow. It works
using a "pull" on the nodes. That is, you first ask the last node for its
output. Then this node asks for the value of each of its input nodes. These
nodes do the same until everything is calculated. If, for some reason a value is
unused, it is not calculated. 

> I guess what we end up doing depends upon how much you and Dominic want to
> change in Overflow.  As for 1), this requires the least amount of work for you
> guys.  But if you're willing to take on new features and paradigms, I'm
> thrilled to hear it :-)

Dominic has a couple things he wants in Overflow (or anything it becomes):
Asynchronus events, multi-thread stuff (partially implemented), ... (Dominic,
anything I forgot?)


> I like the sound of that.  You know, Loci uses icons to represent nodes, and
> some of the icons (will) look like those seen on a GUI desktop: folders,
> trash, documents, etc.  And we're following the Xerox/Apple model in many
> cases: Drag-and-drop a node from a folder onto the desktop, double-click on an
> icon to open a windowlet (little window that stays with the icon and in Loci),
> etc.  PLUS WE HAVE A PLAN FOR GRAPHICAL THEMES: The user can choose desktop
> colors (and wallpaper when the canvas gets that item), icon styles, symbol
> (goes under the icon to represent node class) styles that can be
> mixed-and-matched with icons, plus Gtk themes that can be applied to Loci
> widgets alone.  The way I planned it, Loci COULD substitute for a user's
> desktop.

Sounds cool! I'm not really a GUI guy. The vflow GUI is about the best I can do
(I actually learnind GTK/GNOME while writing it).

> As for a scientific application, Brad, myself and many others involved with
> Loci are computational biologists who hope to use this for scientific
> research.  Hmmmm....There really are many, many more ideas that we've had, but
> we'll get to those in time :-)

I use Overflow for speech processing/recognition and Dominic uses it for image
processing and robot behaviours. I'll probably also write a real-time music
effect processor... when I have time.

Regards,
	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca



From jarl at casema.net  Tue Mar 14 02:17:44 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] orbit/python bindings (was Re: [Pipet Devel] GMS)
References: <Pine.OSF.4.03.10003132001090.19761-100000@busboy.sped.ukans.edu> <38CDA433.84FBE98B@geoserve.net>
Message-ID: <38CDE797.DAF42FEB@casema.net>

"J.W. Bizzaro" wrote:

> Justin Bradford wrote:
> >
> > So I recommend that loci uses orbit-python. When I do get a little more
> > free time, I'd rather work on loci than the bindings, anyway.
>
> Hey Justin!  Thanks for the tip.
>
> BTW, do you know of anyone working on Python bindings to Bonobo?  I saw Miguel
> de Icaza speek last week and give a demo of Bonobo.  He showed an 'SVG Bonobo
> component'.  I think this would be great to use for visuals, but we need the
> Python bindings to do it.

Bonobo is a docked gtk widget that has a Corba api to import & export datashunks,
they surtainly
are valueble as visuals.



From gvd at redpoll.pharmacy.ualberta.ca  Tue Mar 14 02:22:51 2000
From: gvd at redpoll.pharmacy.ualberta.ca (Gary Van Domselaar)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] GMS
References: <200003131522.KAA154706@archa11.cc.uga.edu> <38CD48B8.3F261420@geoserve.net> <38CD4A4E.D85CBD1A@casema.net> <38CD4E3A.5738D670@geoserve.net>
Message-ID: <38CDE8CB.B9C37585@redpoll.pharmacy.ualberta.ca>

"J.W. Bizzaro" wrote:
> 
> jarl van katwijk wrote:
> >
> > As I mentioned earlier: the first subject of discussion should be the differance
> > between the classes\objects 'filosophy' of both systems.. I think your choose
> > to implement different types based upon functionality isn't the way it should be
> > done. I'm supporting the 'one global do-everything object".
> >
> > I'm very curious why Loci classes are implemented the way they are...

Jarl,

Loci's philosophy follows that of the unix philosophy: one function, one
application.  Sophisticated, distributed data processing is accomplished
by pipelining these 'simple' applications toghether.  The
'all-inclusive' approach that GMS takes for brokering datastreams is
also valid.  One consideration for Loci, however, is that it is intended
ultimately to act as a graphical scripting language for distributed data
brokering.  The processing applications exist already; Loci wraps these
applications, and adds 'extras' to the workpath (like data format
conversion utilities) in order for the data to stream properly between
different applications.  Other 'elemental loci' include data storage
utilities, which can be customized to store an incoming data stream
based on user defined schemas, possibly so that they can process it
further later using non Loci-wrapped processing applications, or perhaps
to view the data with their favorite viewing application. In our world
of computational biology, this is a requirement, so modularity and
simplicity are a big part of the Loci philosophy.

An additional consideration arises from the following scenario: a simple
workflow diagram can be encapsulated into  a 'node' which is itself
capable of being inserted into another workpath, and so on, ad
infinitum.  An apparently simple workpath may be in reality composed of
multiply nested workpaths and can in principle be quite complex.  In
order to minimize bloat, our thinking was to minimize each type of
elementary node to its basic functionality.  

IMHO, I don't think there is a 'right' or 'wrong' way to design the
elementary objects.  This is all good food for thought, and I suggest we
(Loci, GMS, and Overflow) create a discussion forum where we can have
open discussions on the best way to implement this core functionality. 
The Locians are a very open-minded group; thats one of the reasons why
we discussed our project for so long before committing to writing code. 
Lucky that we did, since we have so recently made contacts with such
useful potential allies!

I am currently working on creating a web-based open source development
environment for our organization: bionformatics.org.  It is based on the
sourceforge development environment (http://sourceforge.net).  It is a
sophisticated and very user-friendly environment intended to foster
discussions and collaborations of the nature in which we are currently
engaged.  I can set up a forum for you, Jean-Marc (canadian, yaay;-) and
the locians to hash out the details of our architectures, design goals,
and hopefully generate a powerful collaboration for creating the
sophisticated data processsing environment that we are all so passionate
about.


> 
> Perhaps you can give us a synopsis/summary of the GMS object philosophy.  It
> would then be easier for me to point out differences, because right now I'm
> not so sure how GMS is different.

I can see from the dia flowchart on your homepage that you have a well
thought out architecture for your core operation, but some of the
acronyms are a little too obscure for me. If you could provide a couple
of paragraphs describing the operation of your 'core' for us, that too
would be greatly appreciated.

Regards,

g.
-- 
                                  Gary Van Domselaar
                               gary@bioinformatics.org
                          http://www.bioinformatics.org/~gary
                ----------------------------------------------------
                           bioinformatics.org: The Open Lab
                            http://www.bioinformatics.org/


From jarl at casema.net  Tue Mar 14 02:35:25 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] [Fwd: Overflow/Loci/GMS collaboration]
References: <38CDA830.17908993@geoserve.net>
Message-ID: <38CDEBBD.A6B4BD94@casema.net>

>
> There's another option that might be worth considering: "plugging" the GMS (or
> Loci) core on top of the Overflow core. I don't think it would be that hard,
> since we already have a node that can exec an executable and return it's stdout.
> Using sockets/corba might not be that hard. The advantage is that you could have
> a complex processing without too much overhead.
>

Maybe Overflow-subnets can be wrapped to become an element inside loic\gms?

>
> A nice feature we have (I didn't see it in Loci/GMS, but maybe it's there too)
> is "subnets" and "iterators", which allow the C equivalent of functions
> (reusable in any "program") and loops. For example, this allows me "for each
> file in a list", "for each 10 ms frame in the file", to calculate a
> powerspectrum.

It's on my TODO :)




From bizzaro at geoserve.net  Tue Mar 14 02:50:08 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Messaging/Loci alliance mailing list
References: <38CDAD1A.78B3BC52@geoserve.net> <38CDEC78.499D54E2@casema.net>
Message-ID: <38CDEF30.2A1CBBA1@geoserve.net>

BTW,

Jean-Marc Valin <valj01@gel.usherb.ca> and Dominic Letourneau
<doumdi@yahoo.com> are not on the Loci list, so e-mail should be addressed to
them as well.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 02:52:24 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] [Fwd: Overflow/Loci/GMS collaboration]
References: <38CDA830.17908993@geoserve.net> <38CDEBBD.A6B4BD94@casema.net>
Message-ID: <38CDEFB8.8912EA27@geoserve.net>

jarl van katwijk wrote:
> 
> > A nice feature we have (I didn't see it in Loci/GMS, but maybe it's there too)
> > is "subnets" and "iterators", which allow the C equivalent of functions
> > (reusable in any "program") and loops. For example, this allows me "for each
> > file in a list", "for each 10 ms frame in the file", to calculate a
> > powerspectrum.
> 
> It's on my TODO :)

You see, GMS and Overflow do complement each other ;-)

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 14 02:51:17 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net>
Message-ID: <38CDEF75.475FF3DC@casema.net>

> >
> > There's another option that might be worth considering: "plugging" the GMS (or
> > Loci) core on top of the Overflow core. I don't think it would be that hard,
> > since we already have a node that can exec an executable and return it's stdout.
>
> That's along the line of what GMS has accomplished and Loci has been trying to
> accomplish.  This is something we'll need to discuss since we have 3 ideas
> about how this can be done.

Gms does not 'wrap' external applications, it rather attaches them. Writting a
wrapper-plugins is perfectly possible though.

>
>
> But if Overflow were to be a node in GMS, Overflow would have to handle input
> from and output to the outside world.  I imagine Overflow might need a special
> node for doing that.

Not needed, an Overflow sub-net can 'accept' any data-description out of the pipeline

that it wants. The exsisting & 'to be coded' sensors will take care of the gathering
of data
and the visual will export the dataflows. In the GMS design dataflows MUST pass a
sensor-pipeline-visual path becouse of security\accessability.

> An idea that we had for Loci, was that unconnected lines
> on a workspace would represent data that would flow to and from the outside.
> And since Loci was going to nest workspace in a workspace in a workspace,
> etc., and each nested workspace is itself a node, unconnected lines would mean
> data flows to a parent.  I'd like to see us do that in our collaboration.

Unconnected lines? Like 'deadend dataflows'?

>
> Loci wasn't going to care about the content of the file.  We were going to use
> XML meta-data to accompany each node, and that could be saved to a file.  I'd
> like to hear what Jarl has to say about this and how GMS does it.

Likewise, gms has data in 5 types availieble to every node\object.
(Raw\string\integer\
percentage\boolean). Historic data is cached\stored on disk.

> > As for the GUI, I don't think it's a priority for now... we might end up
> > rewriting one that best handles the new core (if we merge the 3 projects). Or we
> > can always keep 3 different GUIs that use the same core differently...
>
> Well, I okayed this with our one other dedicated coder (Brad), and we want
> Loci to provide a consistent (and very cool, I might add) GUI across the
> projects.  So, we'll back off of the infrastucture coding and dedicate
> ourselves to user interface and general design issues.

Maybe the person that coded the core of Loci is interested switching from python to
C?

>
> > Last thing, let's see how much time we have. For the next year, I'm gonna code a
> > lot for Overflow, since I use it for my master (in speech processing). What
> > about you and the GMS guys?

I'm always working on gms.. only recently I started a new project (professional)
which will cool down coding in the midweeks :) Weekend are still dedicated to gms..



From bizzaro at geoserve.net  Tue Mar 14 03:22:31 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDEF75.475FF3DC@casema.net>
Message-ID: <38CDF6C7.53C8B4C8@geoserve.net>

jarl van katwijk wrote:
> 
> Gms does not 'wrap' external applications, it rather attaches them. Writting a
> wrapper-plugins is perfectly possible though.

I stand corrected.

> > An idea that we had for Loci, was that unconnected lines
> > on a workspace would represent data that would flow to and from the outside.
> > And since Loci was going to nest workspace in a workspace in a workspace,
> > etc., and each nested workspace is itself a node, unconnected lines would mean
> > data flows to a parent.  I'd like to see us do that in our collaboration.
> 
> Unconnected lines? Like 'deadend dataflows'?

What Loci would do in such a case is try to send the data to a parent node or
network, instead of just killing the dataflow.  We were going to define a
'dead-end' node for when the dataflow needs to be stopped.  Actually the
reverse is just as valid: unconnected lines stop the dataflow and special
nodes can send data to parents.  I just chose the former approach because
unconnected lines in a nested workspace can be mapped directly onto
unconnected lines on the parent node, which I think is more intuitive.  You
can see this in the following screenshot:

    http://bioinformatics.org/loci/documentation/screenshots/latest.gif

The red (unconnected) dots in the nested workspace (lower left) are mapped
onto the parent node (labeled 'composite') and appear as green (connected)
dots.

> Likewise, gms has data in 5 types availieble to every node\object.
> (Raw\string\integer\
> percentage\boolean). Historic data is cached\stored on disk.

Are you using XML?  What do you think about using it?

> Maybe the person that coded the core of Loci is interested switching from python to
> C?

Are you saying you want the GUI in C, or that you want Brad to help with the
core?

It's amusing that all 3 projects independently decided to separate the GUI
from the core.  But if we are to keep the GUI separate, does it really matter
what language the GUI is written in, or is this a personal preference?

I know it seems awkward to have 3 languages.  But I guess we'll have to see to
what extent our projects will be merged.  If we're looking at a single
distribution, and Overflow and GMS agree on either C or C++, then we can
consider C or C++ for the GUI.  But if our projects become separate
distributions (as with Gnome) that communicate through CORBA, language
shouldn't matter.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 14 08:55:45 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDEF75.475FF3DC@casema.net> <38CDF6C7.53C8B4C8@geoserve.net>
Message-ID: <38CE44E1.344F3315@casema.net>

> > Unconnected lines? Like 'deadend dataflows'?
>
> What Loci would do in such a case is try to send the data to a parent node or
> network, instead of just killing the dataflow.  We were going to define a
> 'dead-end' node for when the dataflow needs to be stopped.  Actually the
> reverse is just as valid: unconnected lines stop the dataflow and special
> nodes can send data to parents.  I just chose the former approach because
> unconnected lines in a nested workspace can be mapped directly onto
> unconnected lines on the parent node, which I think is more intuitive.  You
> can see this in the following screenshot:
>
>     http://bioinformatics.org/loci/documentation/screenshots/latest.gif
>
> The red (unconnected) dots in the nested workspace (lower left) are mapped
> onto the parent node (labeled 'composite') and appear as green (connected)
> dots.

Ic, gms is still very simple about connectivity logic, is this area loci is much more
mature.

>
>
> > Likewise, gms has data in 5 types availieble to every node\object.
> > (Raw\string\integer\
> > percentage\boolean). Historic data is cached\stored on disk.
>
> Are you using XML?  What do you think about using it?

Good Thing, I like using standards anyway, and XML seems to be a good candidate for
data-encapsulation. I did not study it yet.
Yesterday evening I did read the Loci, the Overflow and some other projects
documentation b.t.w.

>
>
> > Maybe the person that coded the core of Loci is interested switching from python to
> > C?
>
> Are you saying you want the GUI in C, or that you want Brad to help with the
> core?

the 2nd, coding the core on my own wont go quickly enough.

>
> It's amusing that all 3 projects independently decided to separate the GUI
> from the core.  But if we are to keep the GUI separate, does it really matter
> what language the GUI is written in, or is this a personal preference?

You're correct. As long as corba is used to combine all parts it's better to chose a
language for every part. Pythons can be better for the GUI, C is best for the core and
C++ can be used for a 'detail level'.

>
> I know it seems awkward to have 3 languages.  But I guess we'll have to see to
> what extent our projects will be merged.  If we're looking at a single
> distribution, and Overflow and GMS agree on either C or C++, then we can
> consider C or C++ for the GUI.  But if our projects become separate
> distributions (as with Gnome) that communicate through CORBA, language
> shouldn't matter.

Yep :)




From bizzaro at geoserve.net  Tue Mar 14 09:26:53 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net>
	 <38CDBEC7.6C48A3C7@hermes.usherb.ca> <38CDD93B.762E508F@geoserve.net> <38CDE41F.5A263F64@hermes.usherb.ca>
Message-ID: <38CE4C2D.62A696DB@geoserve.net>

Jean-Marc Valin wrote:
> 
> > > 1) Wrap all Overflow as a Loki/GMS node
> > > 2) Use Overflow's core and implement the missing functionnalities.
>
> OK, let's talk about 2) (I consider 1 to be trivial). All our nodes are classes
> and the code that's ran is in the getOutput method. Maybe it's possible to call
> the GMS functions from there. Again, the main reason I'd like 2) is speed: not
> having to call an executable for a sum, logical and, even an FFT.

The big problem I see with approach 2) is that Overflow has no knowledge of
where it is in a physical network, where other instances of Overflow might be,
what nodes are available at what location, how to request a particular remote
node, and how to send data between two remote instances of Overflow without it
first coming back to the local instance.  These are all problems Loci was
designed to deal with.  But if you're talking about Overflow being the real
core, as core it would have to deal with these problems.

So, I don't think plugging GMS into Overflow's port will make Overflow/GMS a
distributed system.  As an example, let's say we have computers A and B.  A is
the local computer run by the user, and B is a remote computer.  Both A and B
have copies of Overflow/GMS, except computer B has a node that computer A does
not have.  Now tell me, how will Overflow/GMS on A know (1) where computer B
is, (2) what nodes are on B and (3) how to request access to the node?

Alright, let's make the problem a little more complicated.  We have 3
computers: A, B and C.  A is again local while B and C are separate remote
computers.  Now, the user on computer A wants to pipe data from a node on A,
to a node on B, and then directly to a node on C.  How does (4) Overflow/GMS
on B know to send the data to C and not back to A?

The only way I see to address these problems, and what Loci was proposing to
do, is create a directory service and a meta-data transfer system.

In Loci, each instance of Loci's core is identical to the next, regardless of
location.  And each has the ability to contact another with a request.  If,
for example, a node exists on computer B that the user wants (access to), Loci
on A would ask Loci on B for it.  And the system is very much seamless and
symmetric: If a node exists on computer A that the user wants, Loci on A (same
computer) ASKS ITSELF FOR THE NODE.

And if the user wants to connect a node on B to a node on C, Loci on A sends B
the request with meta-data saying to contact C when done.  Next, B contacts C,
and then C contacts A that the connection was made.

> > So, Loci would allow the user to execute a node and THEN start adding
> > connections...or even change connections at a later time.  Loci does this by
> > making a 'dynamic batch file', which really has to be a database (since we are
> > using XML, we were going to use an XML database).
> 
> I'm not sure I understand what you mean here?

I mean that all of the information for each node (including meta-data) and all
of the connections made between nodes, are repesented as XML.  BUT, to make
parsing and editing this XML structure easier, we put the information into a
database.  The information is actually transformed from XML into a form better
suited for the database, and then when retrieved, the information is put back
into XML, which is better suited for transfer and storage.

> You can have a set of nodes that generate a command line in Overflow, then feed
> that command-line (string) to the "Exec" node.

Great.  That's pretty much what we were trying to do, but we have some rather
interesting rules for construction that I'll have to pass on to you.

> Is the "program" in Loci stored as a script?

What are you calling 'program'?  If you mean the network to be ran (or
workpath to be followed in Loci-speak), then yes, it's stored as a script...in
a database.

> The way it works in Overflow, is
> that the XML is simply storing a set of objects: Networks, Nodes and Links.

Exactly the same as Loci!

> It
> stores how the nodes are connected together (XML was just a convenient way to
> serialize our objects).

Right.  Except, for doing dynamic editing and parsing, we need to take the
data out of XML and put it into a database.

> Another thing you need to know about Overflow. It works
> using a "pull" on the nodes. That is, you first ask the last node for its
> output. Then this node asks for the value of each of its input nodes. These
> nodes do the same until everything is calculated. If, for some reason a value is
> unused, it is not calculated.

Excellent!  This is exactly how we planned on 'running' a network.

> Dominic has a couple things he wants in Overflow (or anything it becomes):
> Asynchronus events, multi-thread stuff (partially implemented), ... (Dominic,
> anything I forgot?)

Great.

> Sounds cool! I'm not really a GUI guy. The vflow GUI is about the best I can do
> (I actually learnind GTK/GNOME while writing it).

I think I am a GUI guy afterall.  I tend to need to see the results of my
work.

> I use Overflow for speech processing/recognition and Dominic uses it for image
> processing and robot behaviours. I'll probably also write a real-time music
> effect processor... when I have time.

Oh cool, a couple of AI/robotics guys.  I think our needs are indeed very
similar.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 09:36:08 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDEF75.475FF3DC@casema.net> <38CDF6C7.53C8B4C8@geoserve.net> <38CE44E1.344F3315@casema.net>
Message-ID: <38CE4E58.167D9030@geoserve.net>

Jarl van Katwijk wrote:
> 
> Ic, gms is still very simple about connectivity logic, is this area loci is much more
> mature.

It's not that we have all this programmed.  If we did, we'd be nearly done. 
It's just that we've been talking about this stuff for more than a year now
:-)

> Good Thing, I like using standards anyway, and XML seems to be a good candidate for
> data-encapsulation.

That's good to hear, since Loci and Overflow have invested heavily in XML.

> the 2nd, coding the core on my own wont go quickly enough.

I wouldn't say that we're expecting you to code the core on your own ;-)

> You're correct. As long as corba is used to combine all parts it's better to chose a
> language for every part. Pythons can be better for the GUI, C is best for the core and
> C++ can be used for a 'detail level'.

I don't know what the core will look like in the end.  It seems all 3 projects
have something to contribute to it.  But it only makes sense to have the core
in one language.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 14 09:32:18 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:21 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDBEC7.6C48A3C7@hermes.usherb.ca> <38CDD93B.762E508F@geoserve.net>
Message-ID: <38CE4D72.1A3F9C5B@casema.net>

> >
> > The reason I was suggesting the Overflow core is that we already handle IO with
> > files and pipes, so Overflow nodes can also be executables (it's tested and it
> > works). From what I know now, I see two options:
> > 1) Wrap all Overflow as a Loki/GMS node
> > 2) Use Overflow's core and implement the missing functionnalities.
>
> If we chose 2), what would we use GMS's core for?  I'd like to hear Jarl's
> thoughts on Overflow's core.  GMS may provide the missing functionality for
> Overflow, and merging C and C++ shouldn't be all THAT difficult.  I was
> thinking about 1), but sure, we should consider 2) as well.

I read the Overflow documentation, and I'm not very enthousiastic about it. I think
it's structures are to small, it's more suiteble for realtime processing as it is for
application intergration. Using the Overflow core will requere a lot of
configurartion before it can do something usefull, even when on got a lot 'subnets'.
Also its components wont be manageble, they will be in the system in too great
numbers for a security\authentication system to handle. 2ndly, as I memtioned before
in some emails, I think nodes of different 'types' wont be as  powerfull as a general
object type, even when this general type has more cpu\memory overhead. Modern C
programming libraries, like Glib, will greatly remove this overhead anyway. And 3thly
I dont really like the way Overflow enforces high speed and high configuarability;
these features should mainly remain inside the wrapped applications themselfs. GMS
(and also Loci) want to accomplish 'Application Intergration", they dont want to be
"Doing the work themselfs".
Having the structures of Overflow inside nodes does make sense, this way it offers a
methode for writing nodes\plugins in a 'scripting' language. 4thly the gms core is
gonna be very fast: too, it already executes binairy code instead of some form of
scripting and I'm gonnan optimize the dataflow prosessor a lot.  So there's no need
to chose  the "overflow way' for speed reasons.
Dont take me wrong, there's surtainly a lot that the 'overflow-way' can do, but I
think it's not a good base for application intergration.


> > I forgot... another thing we have in Overflow that's essential (well, for what I
> > do with it) is the ability to "automatically" cache the results that will be
> > reused in another iteration of a loop or by another node, so nothing gets
> > computed twice.
>
> I haven't even thought about that, since we haven't developed any flow control
> nodes for Loci.  Loci does, however, always keep reference to the location of
> a block of data (often kept as 'document' nodes) so we probably wouldn't have
> thrown that stuff away.
>

GMS also does keep track of histori data. Calculation results reusage is a nice
idea..




From bizzaro at geoserve.net  Tue Mar 14 09:55:42 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDBEC7.6C48A3C7@hermes.usherb.ca> <38CDD93B.762E508F@geoserve.net> <38CE4D72.1A3F9C5B@casema.net>
Message-ID: <38CE52EE.B242E31A@geoserve.net>

Jarl van Katwijk wrote:
> 
> it's structures are to small, it's more suiteble for realtime processing as it is for
> application intergration.

But that's exactly why we're combining efforts: Overflow does something Loci
and GMS don't, and visa versa.  Overflow is not an application integration
environment, but Loci and GMS can't provide such low-level building blocks for
high-speed graphical application development.

> Using the Overflow core will requere a lot of
> configurartion before it can do something usefull, even when on got a lot 'subnets'.
> Also its components wont be manageble, they will be in the system in too great
> numbers for a security\authentication system to handle.

I don't know.  If Overflow nodes are connected locally, do they need security
authentication???

> 2ndly, as I memtioned before
> in some emails, I think nodes of different 'types' wont be as  powerfull as a general
> object type, even when this general type has more cpu\memory overhead. Modern C
> programming libraries, like Glib, will greatly remove this overhead anyway.

Okay, you have a valid point.  I'm willing to have it your way.

> And 3thly
> I dont really like the way Overflow enforces high speed and high configuarability;
> these features should mainly remain inside the wrapped applications themselfs.

Right, well Overflow is for application development; Loci and GMS are for
application integration.  We can have both worlds, can't we?

> 4thly the gms core is
> gonna be very fast: too, it already executes binairy code instead of some form of
> scripting and I'm gonnan optimize the dataflow prosessor a lot.  So there's no need
> to chose  the "overflow way' for speed reasons.

Then I invite Jean-Marc and Dominic to take a look at what GMS has to offer.

> Dont take me wrong, there's surtainly a lot that the 'overflow-way' can do, but I
> think it's not a good base for application intergration.

Again, I think we can do both.

> GMS also does keep track of histori data. Calculation results reusage is a nice
> idea..

That's good.  Loci was going to keep track of every step in network
construction, simply by the fact that the XML command stream could be saved.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 10:10:23 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Messaging/Loci alliance mailing list
References: <38CDAD1A.78B3BC52@geoserve.net> <38CDDBEA.566BEB9F@redpoll.pharmacy.ualberta.ca>
Message-ID: <38CE565F.FFC31F55@geoserve.net>

Gary Van Domselaar wrote:
> 
> How about 'The Data Pipe Alliance'?  Sounds professional and cool.
> Whatever you decide I can set up on the forthcoming development
> environment for bioinformatics.org.

How about gNodes ?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 14 10:31:50 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDBEC7.6C48A3C7@hermes.usherb.ca> <38CDD93B.762E508F@geoserve.net> <38CE4D72.1A3F9C5B@casema.net> <38CE52EE.B242E31A@geoserve.net>
Message-ID: <38CE5B66.14F14EE4@casema.net>

> >
> > it's structures are to small, it's more suiteble for realtime processing as it is for
> > application intergration.
>
> But that's exactly why we're combining efforts: Overflow does something Loci
> and GMS don't, and visa versa.  Overflow is not an application integration
> environment, but Loci and GMS can't provide such low-level building blocks for
> high-speed graphical application development.
>

GMS can provide this kinda lowlevel building blocks, but they wont be used i

>
> > Using the Overflow core will requere a lot of
> > configurartion before it can do something usefull, even when on got a lot 'subnets'.
> > Also its components wont be manageble, they will be in the system in too great
> > numbers for a security\authentication system to handle.
>
> I don't know.  If Overflow nodes are connected locally, do they need security
> authentication???

Yes. Every time a node is created OR a relation betweem them is created, I want to have
some
sort of authetication takeing place. I'm thinking about manual OK-ing a node once to make
it
loadable for the system, and calculating a CRC to authenticate. Overflow has a fixed
number of
static nodes, so securing node-creating will be much easier.

But the real problem comes for the authentication of relations between nodes. It wont be
possible
to allow a relation between (overflow type) node A en node B, because it cant be determind
what
the chain-of-nodes ending into node A will do. In gms authentication between a Collector
and a
Visual will be implemented, and there wont be taken heed of the filters in between. All
collectors
and visuals are crc-unique, so the relation between them can be authenticated.  Not having
such
a system will lead to unacceptable exploiting.
The security system I've in mind is taking much more into account, but relation-creation
is by far
the most important part.

>
> > And 3thly
> > I dont really like the way Overflow enforces high speed and high configuarability;
> > these features should mainly remain inside the wrapped applications themselfs.
>
> Right, well Overflow is for application development; Loci and GMS are for
> application integration.  We can have both worlds, can't we?

Yes. But not 100% of both :)

bye,
jarl



From jarl at casema.net  Tue Mar 14 10:34:21 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Messaging/Loci alliance mailing list
References: <38CDAD1A.78B3BC52@geoserve.net> <38CDDBEA.566BEB9F@redpoll.pharmacy.ualberta.ca> <38CE565F.FFC31F55@geoserve.net>
Message-ID: <38CE5BFD.92A90DD5@casema.net>

"J.W. Bizzaro" wrote:

> Gary Van Domselaar wrote:
> >
> > How about 'The Data Pipe Alliance'?  Sounds professional and cool.
> > Whatever you decide I can set up on the forthcoming development
> > environment for bioinformatics.org.
>
> How about gNodes ?
>

Or simply Gnu Application Intergration (GAI)?
(Based upon the name 'Enterprise Application Intergration')



From bizzaro at geoserve.net  Tue Mar 14 10:51:13 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Messaging/Loci alliance mailing list
References: <38CDAD1A.78B3BC52@geoserve.net> <38CDDBEA.566BEB9F@redpoll.pharmacy.ualberta.ca> <38CE565F.FFC31F55@geoserve.net> <38CE5BFD.92A90DD5@casema.net>
Message-ID: <38CE5FF1.AE10258A@geoserve.net>

Jarl van Katwijk wrote:
> 
> Or simply Gnu Application Intergration (GAI)?

But that would be pronounced 'GAY' :-P

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From gvd at redpoll.pharmacy.ualberta.ca  Tue Mar 14 10:51:35 2000
From: gvd at redpoll.pharmacy.ualberta.ca (Gary Van Domselaar)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net> <38CDEF75.475FF3DC@casema.net> <38CDF6C7.53C8B4C8@geoserve.net>
Message-ID: <38CE6006.AACD05A1@redpoll.pharmacy.ualberta.ca>

> It's amusing that all 3 projects independently decided to separate the GUI
> from the core.  But if we are to keep the GUI separate, does it really matter
> what language the GUI is written in, or is this a personal preference?

considering that the GUI is used to visually construct data workpaths,
speed is not an issue, and that most user feedback requests are for
interface modifications, I suggest we stick with Python, considering it
is such an excellent rapid prototyping language, and very easy to pick
up.  

> 
> I know it seems awkward to have 3 languages.  But I guess we'll have to see to
> what extent our projects will be merged.  If we're looking at a single
> distribution, and Overflow and GMS agree on either C or C++, then we can
> consider C or C++ for the GUI.  But if our projects become separate
> distributions (as with Gnome) that communicate through CORBA, language
> shouldn't matter.

I can see why we would want to use CORBA's IIOP to communicate between
'cores' in a network distributed environment, and also to control
wrapped applications, which likely are also distributed, but if the core
and GUI reside on the same box, is there a need to communicate to the
GUI through CORBA? Loci and Overflow have already developed an XML
protocol for core<-->GUI communication.  Is there a better argument for
switching to CORBA?

regards,

g.
-- 
                                  Gary Van Domselaar
                               gary@bioinformatics.org
                          http://www.bioinformatics.org/~gary
                ----------------------------------------------------
                           bioinformatics.org: The Open Lab
                            http://www.bioinformatics.org/


From jarl at casema.net  Tue Mar 14 10:57:48 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Messaging/Loci alliance mailing list
References: <38CDAD1A.78B3BC52@geoserve.net> <38CDDBEA.566BEB9F@redpoll.pharmacy.ualberta.ca> <38CE565F.FFC31F55@geoserve.net> <38CE5BFD.92A90DD5@casema.net> <38CE5FF1.AE10258A@geoserve.net>
Message-ID: <38CE617C.F47DBC5B@casema.net>

"J.W. Bizzaro" wrote:

> Jarl van Katwijk wrote:
> >
> > Or simply Gnu Application Intergration (GAI)?
>
> But that would be pronounced 'GAY' :-P
>

Hehe.. not good :)
I'm Dutch, so I didn't notice this 'problem'.




From gvd at redpoll.pharmacy.ualberta.ca  Tue Mar 14 11:10:19 2000
From: gvd at redpoll.pharmacy.ualberta.ca (Gary Van Domselaar)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Messaging/Loci alliance mailing list
References: <38CDAD1A.78B3BC52@geoserve.net> <38CDDBEA.566BEB9F@redpoll.pharmacy.ualberta.ca> <38CE565F.FFC31F55@geoserve.net> <38CE5BFD.92A90DD5@casema.net> <38CE5FF1.AE10258A@geoserve.net>
Message-ID: <38CE646B.8E1D946C@redpoll.pharmacy.ualberta.ca>

"J.W. Bizzaro" wrote:
> 
> Jarl van Katwijk wrote:
> >
> > Or simply Gnu Application Intergration (GAI)?
> 
> But that would be pronounced 'GAY' :-P

Its a little ambiguously (gay ;-).  bonobo is associated with
application integration (as in component software integration).  We are
dealing with data connectivity brokering, so perhaps to keep some kind
of data reference in the title?  Petty bickering I know, but first
impressions are lasting ones!

regards.
g.
-- 
                                  Gary Van Domselaar
                               gary@bioinformatics.org
                          http://www.bioinformatics.org/~gary
                ----------------------------------------------------
                           bioinformatics.org: The Open Lab
                            http://www.bioinformatics.org/


From chapmanb at arches.uga.edu  Tue Mar 14 11:48:02 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Code updates
Message-ID: <200003141651.LAA83906@archa11.cc.uga.edu>

Hey everyone;
    Just wanted to let everyone know that I just made a huge cvs 
commit, and now the front and middle are officially separated! Yay! 
All communication between them is now going through sockets, and I 
should have restored all of the functionality that we had previously, 
with a few exceptions:

1. My "wrapping"/connection code isn't currently compatible with the 
structure of everything, so it isn't used right now. It probably won't 
be very useful in the future either, with GMS and Overflow and all so 
it'll probably be out for a while, if not permanently.

2. The database connectivity stuff only half works because of some 
issues I'm having with sockets (more on this below). You can only 
connect to a gadfly db, and can't view the contents of tables. 

If anything else generates errors, please post on it. I've fixed the 
error Jeff noticed with containers (thanks for this!), and so 
everything except for what I've mentioned _should_ work right.
    I also added a doc directory which has the communication protocols 
I'm using kind of outlined.
    Maybe if Jeff is feeling like he has free time, and the code seems 
okay, he might be able to develop a new snapshot from this, so people 
can check it out:) If you are taking a look at the code, the parts to 
look at are basically divided as following:

front/gnome/library and subdirectories: The code for the gnome/gtk 
based front end. 

middle/library/modules/xml: This is all of the code for the current 
middle/core. Basically it contains code to create a server to 
communicate with the front via XML through sockets, code to create 
XML descriptions of the workspace (which can be seen in middle/temp 
when the Loci program is running), and code for dealing with the 
filesystem and databases.

middle/library/modules/wrap: The old barely started "wrapping" 
and "connectivity" code. This isn't currently used, so it can be 
ignored.

There is tons o' commenting, but if stuff isn't clear, please feel 
free to post. Any comments/criticism are highly encouraged. I 
apologize, I'm having trouble getting any of the docstring processing 
programs to work without complaining, so there isn't any nice api 
documentation like there should be.

    Okay, the rest of this has to do with problems I'm having with the 
socket connection between the front and back. Basically, I'm having 3 
issues:

1. Speed: Both the database and filesystem container views come up 
*really* slowly. This is due (I think) to all the extra dom/xml 
generation, and also the lag time waiting for the front.

2. Password passing: I've been passing around passwords for databases 
in an encrypted format (using the rotor module in the standard 
distribution). When encrypted, a password is a jumble of random 
characters. The problem I'm having is in passing these through the 
socket. When the server receieves a message, it receieves it in small 
pieces of strings, and then adds them up (until it gets a newline, the 
end of message signal). The problem is that python doesn't like to add 
the weird characters generated in the encryption, and so complains 
when it tries to add these.

3. Data passing: One thing which we haven't talked about, although I 
should have thought of it before, is how to pass a large amount of 
"unstructured info" between the front and the middle. For instance, 
how should I pass back the columns and row information for a database? 
    If I format this as XML, this will take impossibly forever for 
large tables (which I have quite a few of in my databases), but I 
can't think of another way way to pass info cleanly and reusably. 

Does anyone have any thoughts on fixes for these problems? 
Suggestions? Am I misssing something simple? 

Thanks for reading! 

Brad



From chapmanb at arches.uga.edu  Tue Mar 14 11:59:04 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
Message-ID: <200003141702.MAA110152@archa12.cc.uga.edu>

Hey all;
    Thanks for the interesting discussions on collaborations. I'm very 
excited to see this happening. I'll let you all do the decision making 
for most of the design issues, since you've all got lots of excellent 
design ideas and I'm basically just a coding monkey :-)
    I'd be more than happy to help in any way I can, which includes 
both doing GUI/front end stuff in python and helping with the GMS 
stuff in C. I'll probably be pretty sad in C at first, since I only 
have C++ experience, and all I know about C is from staring at other 
people's code, but I'm willing to work on it :) I'll be spending some 
time looking at both GMS and Overflow code over the next few days and 
trying to think of how we can integrate stuff with maximum reuse of 
existing code (definately a worthy goal). We have lots of choices 
for integrating the three languages, and hopefully we can do it 
cleanly. 
    Anyways, that's my short 2 cents....

Brad




From jean-marc.valin at hermes.usherb.ca  Tue Mar 14 13:20:29 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
References: <38CE50AC.EC48D4A@geoserve.net>
Message-ID: <38CE82ED.39CCB98@hermes.usherb.ca>

> I read the Overflow documentation, and I'm not very enthousiastic about it. I think

Unfortunatly, the doc is not totally up-to-date and mostly does not reflect wat
*could* be done with Overflow.

> it's structures are to small, it's more suiteble for realtime processing as it is for
> application intergration. Using the Overflow core will requere a lot of
> configurartion before it can do something usefull, even when on got a lot 'subnets'.
> Also its components wont be manageble, they will be in the system in too great
> numbers for a security\authentication system to handle. 2ndly, as I memtioned before
> in some emails, I think nodes of different 'types' wont be as  powerfull as a general
> object type, even when this general type has more cpu\memory overhead. Modern C

Don't confuse node types and data types. Node types just means what the node
does (FFT, OR, IF, exec, ...). You need them (Though if you want to run an app,
you can use the "Exec" node). As for the data types, they all derive from a base
object class, so every node have Objects as inputs and outputs. We use real-time
type identification in nodes to make sure that the inputs are the right type
(eg. the FFT node expects its input to be a vector, otherwise it'll throw an
exception). Out Object type is very similar to Glib's Object, except that we use
some C++ features to do the reference counting and casting automatically
(instead of gtk_object_ref or GTK_OBJECT()).

> programming libraries, like Glib, will greatly remove this overhead anyway. And 3thly
> I dont really like the way Overflow enforces high speed and high configuarability;
> these features should mainly remain inside the wrapped applications themselfs. GMS
> (and also Loci) want to accomplish 'Application Intergration", they dont want to be
> "Doing the work themselfs".
> Having the structures of Overflow inside nodes does make sense, this way it offers a
> methode for writing nodes\plugins in a 'scripting' language. 4thly the gms core is
> gonna be very fast: too, it already executes binairy code instead of some form of
> scripting and I'm gonnan optimize the dataflow prosessor a lot.  So there's no need
> to chose  the "overflow way' for speed reasons.

When I say fast, I mean that Overflow is capable of running ~50,000 simple nodes
per second - and that's barely enough for what I do (we're working on making it
faster). Just a fork or opening a socket is orders of magnitude too slow.

Also considering remote execution, it's a nice thing to have, but it's not
something you're going to use all the time. I wonder what a simple RSH node (~15
minutes to code) in Overflow could do...

As for the name, vsh (although it exists - but dead) would be nice... Besides
that, the reason we chose Overflow is that ObjectFlow, gFlow, VisualFlow, ...
were all taken.

	Jean-Marc


From bizzaro at geoserve.net  Tue Mar 14 16:56:26 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Messaging/Loci alliance mailing list
References: <38CDAD1A.78B3BC52@geoserve.net> <38CDDBEA.566BEB9F@redpoll.pharmacy.ualberta.ca> <38CE565F.FFC31F55@geoserve.net> <38CE5BFD.92A90DD5@casema.net> <38CE5FF1.AE10258A@geoserve.net> <38CE646B.8E1D946C@redpoll.pharmacy.ualberta.ca>
Message-ID: <38CEB58A.A051F122@geoserve.net>

Gary Van Domselaar wrote:
> 
> Its a little ambiguously (gay ;-).  bonobo is associated with
> application integration (as in component software integration).  We are
> dealing with data connectivity brokering, so perhaps to keep some kind
> of data reference in the title?  Petty bickering I know, but first
> impressions are lasting ones!

Then how about...

    Application Integration and Development Environment
                         AIDE

?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 17:11:36 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
References: <38CE50AC.EC48D4A@geoserve.net> <38CE82ED.39CCB98@hermes.usherb.ca>
Message-ID: <38CEB918.BA6C03DA@geoserve.net>

Jean-Marc Valin wrote:
> 
> Also considering remote execution, it's a nice thing to have, but it's not
> something you're going to use all the time.

You might not need remote execution for AI/robotics, but it is critical for
computational biology.

I think rather than some sort of homogenization of our projects, we'd be
better off distinguishing application integration and application
development.  GMS/Loci can handle the first, and Overflow can handle the
second.  This is item 1) from before, which you said is trivial.  But
considering issues of authentication/security that Jarl brought up, issues of
Internet-awareness that I brought up, and issues of speed/performance that you
brought up, it may be an insurmountable task trying to get Overflow to do all
of this.  In any case, making Overflow's core 'the' core would leave very
little left of the GMS project...and I think that's why Jarl is feeling a bit
jilted.

> As for the name, vsh (although it exists - but dead) would be nice... Besides
> that, the reason we chose Overflow is that ObjectFlow, gFlow, VisualFlow, ...
> were all taken.

We could use...

    GNU VSH

like 'New VSH'.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 14 17:11:57 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <200003141702.MAA110152@archa12.cc.uga.edu>
Message-ID: <38CEB92D.8172D2A3@casema.net>

>
>     I'd be more than happy to help in any way I can, which includes
> both doing GUI/front end stuff in python and helping with the GMS
> stuff in C. I'll probably be pretty sad in C at first, since I only
> have C++ experience, and all I know about C is from staring at other
> people's code, but I'm willing to work on it :)

Good attitude, and C isn't hard. I've commented the gms code heavily, but
feel free to ask, it's gonna be hard to get the global picture I guess.

bye,
jarl



From jarl at casema.net  Tue Mar 14 17:32:47 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Loci/GMS collaboration
References: <38CE50AC.EC48D4A@geoserve.net> <38CE82ED.39CCB98@hermes.usherb.ca>
Message-ID: <38CEBE0F.2A6CF97E@casema.net>

> Unfortunatly, the doc is not totally up-to-date and mostly does not reflect wat
> *could* be done with Overflow.

I know, I've some Cellulair Automata knowledge and overflow seems to be based upon these.
A program based upon the Turing set, iterations and selection can do everything. Wether
this building blocks are fitted for a production environment is something else. I'm not
denying Overflow's way of implementing nodes, but I'm pointing at things outside a
scientific setup. A desktop system must be secure and stable too.

The only limitation I'm proposing is: Only use subnets, do not use seperate Overflow
nodes.
The overhead this will give is very slim, only 4 dataflow processing cicles. (Sensoring,
piping, subnet-execution and visualising), and believe me: the DFP is gonna be very fast,
I'll write it in assembler I that's what is takes to please you :-)
The wrapping will mostly take place at the authentication moment, which is gonna be once
at load time for a node. After registration nodes are trusted and will be executed without
delay.

>
> Don't confuse node types and data types. Node types just means what the node
> does (FFT, OR, IF, exec, ...). You need them (Though if you want to run an app,
> you can use the "Exec" node). As for the data types, they all derive from a base
> object class, so every node have Objects as inputs and outputs. We use real-time
> type identification in nodes to make sure that the inputs are the right type
> (eg. the FFT node expects its input to be a vector, otherwise it'll throw an
> exception). Out Object type is very similar to Glib's Object, except that we use
> some C++ features to do the reference counting and casting automatically
> (instead of gtk_object_ref or GTK_OBJECT()).

I see. It's probably best that I firstly have a detailed look at the overflow code before
I'll have anymore comments on it.

> When I say fast, I mean that Overflow is capable of running ~50,000 simple nodes
> per second - and that's barely enough for what I do (we're working on making it
> faster). Just a fork or opening a socket is orders of magnitude too slow.

If I may ask: where do you need it for?
maybe it's just too much out of reach to combine with gms's objectives after all :(

bye,
jarl



From bizzaro at geoserve.net  Tue Mar 14 17:48:26 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <200003141702.MAA110152@archa12.cc.uga.edu> <38CEB92D.8172D2A3@casema.net>
Message-ID: <38CEC1BA.6DC78FF6@geoserve.net>

jarl van katwijk wrote:
> 
> >
> >     I'd be more than happy to help in any way I can, which includes
> > both doing GUI/front end stuff in python and helping with the GMS
> > stuff in C. I'll probably be pretty sad in C at first, since I only
> > have C++ experience, and all I know about C is from staring at other
> > people's code, but I'm willing to work on it :)
> 
> Good attitude, and C isn't hard. I've commented the gms code heavily, but
> feel free to ask, it's gonna be hard to get the global picture I guess.

Brad, providing we all agree that the core will be based on GMS and coded in
C, you will want to give GMS much of the functionality we designed for Loci:

  * XML command stream between GUI (front) and core (middle) ??? (unless we
use CORBA)
  * Network (workflow diagram) script construction (in XML database?)
  * Network script manipulation
  * Directory services (like DNS - know what is where)
  * Request/comply and send/receive communication between cores

We would need to integrate this with what GMS already has (if it does not
already have some of these).  Some of the plugin and security/authentication
issues are things Loci has not considered.

If we agreed on this, I'd be happy.  And I think if Overflow needs little work
to be integrated into this system, perhaps there are some GMS/Loci problems
Jean-Marc and Dominic could address and solve through contributed code. 
Overflow is probably the most advanced (computer science-wise) of the systems,
so there is much that can be gleaned and used in a GMS-based core.  And
sharing technologies would help to tightly integrate our projects.

Thoughts?

Jarl, if Overflow were to be used as a node in a GMS/Loci core, would that
solve the authentication/security issues for you?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 14 17:54:20 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
References: <38CE50AC.EC48D4A@geoserve.net> <38CE82ED.39CCB98@hermes.usherb.ca> <38CEB918.BA6C03DA@geoserve.net>
Message-ID: <38CEC31B.B2BEEA04@casema.net>

"J.W. Bizzaro" wrote:

> Jean-Marc Valin wrote:
> >
> > Also considering remote execution, it's a nice thing to have, but it's not
> > something you're going to use all the time.
>
> You might not need remote execution for AI/robotics, but it is critical for
> computational biology.
>
> I think rather than some sort of homogenization of our projects, we'd be
> better off distinguishing application integration and application
> development.  GMS/Loci can handle the first, and Overflow can handle the
> second.  This is item 1) from before, which you said is trivial.  But
> considering issues of authentication/security that Jarl brought up, issues of
> Internet-awareness that I brought up, and issues of speed/performance that you
> brought up, it may be an insurmountable task trying to get Overflow to do all
> of this.  In any case, making Overflow's core 'the' core would leave very
> little left of the GMS project...and I think that's why Jarl is feeling a bit
> jilted.
>

hehe, just a little bit :)  But mostly I'm defending my idears..
Also when the Overflow project aint going to collaborate, I'll still watch them
very regulairly and I surtainly will think a lot more about how to do wrapping of
sub nets of overflow-type nodes inside a gms\loci node. Because it add new
features without breaking the security \ locationing \ etc.




From bizzaro at geoserve.net  Tue Mar 14 18:01:40 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Loci/GMS collaboration
References: <38CE50AC.EC48D4A@geoserve.net> <38CE82ED.39CCB98@hermes.usherb.ca> <38CEBE0F.2A6CF97E@casema.net>
Message-ID: <38CEC4D4.3C05BC0C@geoserve.net>

jarl van katwijk wrote:
> 
> If I may ask: where do you need it for?
> maybe it's just too much out of reach to combine with gms's objectives after all :(

We have to start thinking of Overflow and GMS/Loci as two different
applications that can be used together.  Overflow is for APPLICATION
DEVELOPMENT and GMS/Loci are for APPLICATION INTEGRATION.

And if we look at Overflow as being an APPLICATION too, we can run Overflow
WITHIN GMS/Loci.

I think we could consider any unspecified application running under GMS/Loci
as a blank slate for an Overflow application to be developed.  Imagine
Overflow running in a windowlet within GMS/Loci, as depicted in this
screenshot:

  http://bioinformatics.org/loci/documentation/screenshots/latest.gif

We can make the workspaces look and work almost exactly alike so that you
wouldn't know Overflow is a separate application!  I can do that!

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 18:09:55 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
References: <38CE50AC.EC48D4A@geoserve.net> <38CE82ED.39CCB98@hermes.usherb.ca> <38CEB918.BA6C03DA@geoserve.net> <38CEC31B.B2BEEA04@casema.net>
Message-ID: <38CEC6C3.4EE64EED@geoserve.net>

jarl van katwijk wrote:
> 
> hehe, just a little bit :)  But mostly I'm defending my idears..
> Also when the Overflow project aint going to collaborate, I'll still watch them
> very regulairly and I surtainly will think a lot more about how to do wrapping of
> sub nets of overflow-type nodes inside a gms\loci node. Because it add new
> features without breaking the security \ locationing \ etc.

I don't believe Jean-Marc wants to bulldoze your code and ideas.  I think he
just saw Overflow has possibly being able to handle the role of application
integrator.  Again, we should consider GMS and Overflow as separate
applications, but applications that share a lot of technology.  You even
admitted that there are some Overflow features that you like and would want to
have for GMS.

Is there anything wrong with the idea that Overflow and GMS can run together
yet be perfectly usable without each other?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 14 18:12:17 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <200003141702.MAA110152@archa12.cc.uga.edu> <38CEB92D.8172D2A3@casema.net> <38CEC1BA.6DC78FF6@geoserve.net>
Message-ID: <38CEC751.63959AA1@casema.net>

>
> Brad, providing we all agree that the core will be based on GMS and coded in
> C, you will want to give GMS much of the functionality we designed for Loci:
>
>   * XML command stream between GUI (front) and core (middle) ??? (unless we
> use CORBA)

Use corba to transport the XML data? This supports having multiple 'clients'
attached to the core (DFP, or dataflow prosessor) even remotely across a network.
 I suggest that communication between cores takes place by extending the pipeline
functionality by a sharing ability.
All other points are already started or are fine to me. XML is the mayor item gms
is missing.


> If we agreed on this, I'd be happy.  And I think if Overflow needs little work
> to be integrated into this system, perhaps there are some GMS/Loci problems
> Jean-Marc and Dominic could address and solve through contributed code.
> Overflow is probably the most advanced (computer science-wise) of the systems,
> so there is much that can be gleaned and used in a GMS-based core.  And
> sharing technologies would help to tightly integrate our projects.

Agree completely.
But first everybody should  have a good look at each others code\situation. I
might be better to do some more work seperately before any intergration attempt
is done.
I want to complete verson 0.4.0 which will take about 1-2 weeks.

bye,
jarl



From jarl at casema.net  Tue Mar 14 18:16:46 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Overflow/Loci/GMS collaboration
References: <38CE50AC.EC48D4A@geoserve.net> <38CE82ED.39CCB98@hermes.usherb.ca> <38CEBE0F.2A6CF97E@casema.net> <38CEC4D4.3C05BC0C@geoserve.net>
Message-ID: <38CEC85E.7C216F92@casema.net>

> >
> > If I may ask: where do you need it for?
> > maybe it's just too much out of reach to combine with gms's objectives after all :(
>
> We have to start thinking of Overflow and GMS/Loci as two different
> applications that can be used together.  Overflow is for APPLICATION
> DEVELOPMENT and GMS/Loci are for APPLICATION INTEGRATION.
>

got your point

>
> We can make the workspaces look and work almost exactly alike so that you
> wouldn't know Overflow is a separate application!  I can do that!
>

ok
I could technically solve this by dynamically loading a overflow binairy into the
'procedure' space of a messaging-object.





From bizzaro at geoserve.net  Tue Mar 14 18:23:01 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net>
	 <38CDBEC7.6C48A3C7@hermes.usherb.ca> <38CDD93B.762E508F@geoserve.net> <38CDE41F.5A263F64@hermes.usherb.ca>
Message-ID: <38CEC9D5.1F43ED68@geoserve.net>

Jean-Marc Valin wrote:
> 
> > > 1) Wrap all Overflow as a Loki/GMS node
> > > 2) Use Overflow's core and implement the missing functionnalities.
>
> OK, let's talk about 2) (I consider 1 to be trivial).

I think you would be surprised that if we consider 1), many interoperability
issues will arise.

Plus, I don't think Jarl is willing to even consider 2)  :-)

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From chapmanb at arches.uga.edu  Tue Mar 14 18:19:09 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
In-Reply-To: <38CEC1BA.6DC78FF6@geoserve.net>
Message-ID: <Pine.A41.4.10.10003141804510.109636-100000@arch14.cc.uga.edu>

> jarl van katwijk wrote:
> > Good attitude, and C isn't hard. I've commented the gms code heavily, but
> > feel free to ask, it's gonna be hard to get the global picture I guess.

It took me a while to get the hang of Loci's big picture, so I expect I'll
have similar issues with GMS (and Overflow) but it'll come eventually! I'm
really excited to try a lot of the things your propose, especially the
neural net/genetic algorithm idea for generating a best overall flow path
though a diagram. Very cool.

J.W. Bizzaro wrote:
> Brad, providing we all agree that the core will be based on GMS and coded in
> C, you will want to give GMS much of the functionality we designed for Loci:
> 
>   * XML command stream between GUI (front) and core (middle) ??? (unless we
> use CORBA)
>   * Network (workflow diagram) script construction (in XML database?)
>   * Network script manipulation
>   * Directory services (like DNS - know what is where)
>   * Request/comply and send/receive communication between cores

What I'd like to think about doing is keeping the small middle that we
already have in python, and then building the core off of this using
CORBA. Some of the reasons I would like to try this are:

1. Reuse of code: I'm not too pumped to be reprogramming the same stuff in
C (Python to C converters excluded).

2. Ease of building fronts: While the GMS core has a better suited design
for implementing data connectivity etc, it's CORBA idl is huge, and would
(IMHO) discourage people from developing fronts because they would need to
implement a large interface. This may also be a problem if we need to
adjust the middle, as the prospect of breaking a whole bunch of fronts is
really messy. While I think it will be a challenge to map the "simple"
streaming XML dialog communication between the front to the middle that we
currently have to GMS's messaging object based core, I hope it might be
possible. In addition, if we are going to be using XML, python has *a lot*
of great tools for working with it, and so we might be able to use tools
from both python and C for the core. How does this sound? Possible? Am I
smoking crack?

Brad




From jarl at casema.net  Tue Mar 14 18:24:08 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38CD4D12.874E3807@geoserve.net> <38CD52F6.7462D936@hermes.usherb.ca> <38CDB5F8.90F6A49@geoserve.net>
		 <38CDBEC7.6C48A3C7@hermes.usherb.ca> <38CDD93B.762E508F@geoserve.net> <38CDE41F.5A263F64@hermes.usherb.ca> <38CEC9D5.1F43ED68@geoserve.net>
Message-ID: <38CECA18.A0AD9671@casema.net>

>
> Plus, I don't think Jarl is willing to even consider 2)  :-)

also 2) will cost me a lot more time :)




From chapmanb at arches.uga.edu  Tue Mar 14 18:27:28 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
In-Reply-To: <38CEC31B.B2BEEA04@casema.net>
Message-ID: <Pine.A41.4.10.10003141821380.109636-100000@arch14.cc.uga.edu>

On Tue, 14 Mar 2000, jarl van katwijk wrote:

> hehe, just a little bit :)  But mostly I'm defending my idears..
> Also when the Overflow project aint going to collaborate, I'll still watch them
> very regulairly and I surtainly will think a lot more about how to do wrapping of
> sub nets of overflow-type nodes inside a gms\loci node. Because it add new
> features without breaking the security \ locationing \ etc.

I would personally really like to see Overflow, GMS and Loci all coming
together. For my personal use, I would really be interested in having a
mechanism for wrapping "smaller" objects than just programs. While you may
not often want to pipe these small objects over internet connections, you
may way to combine a whole bunch of local small objects, and pipe the
output to another object on a different machine. This is where a
combination of all three programs really makes a lot of sense to me.
	For those on the list interested in bioinformatics stuff, I'm
specifically thinking of the bioperl/biojava/biopython projects which have
a combined idl specifying a general set of functions for all three
projects. Wrapping these functions seems like something Overflow is
ideally suited to do, while GMS and Loci don't address such small data
structures.
	For such a powerful system as we all are proposing, we really need
to include both small and large structures, or I think we'll be sorry
later. Another 2 cents.

Brad



From bizzaro at geoserve.net  Tue Mar 14 18:58:41 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:22 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <200003141702.MAA110152@archa12.cc.uga.edu> <38CEB92D.8172D2A3@casema.net> <38CEC1BA.6DC78FF6@geoserve.net> <38CEC751.63959AA1@casema.net>
Message-ID: <38CED231.65D65D23@geoserve.net>

jarl van katwijk wrote:
> 
> Use corba to transport the XML data?

We could, but Brad has some good points about continuing the use of Loci's
command stream for core-to-GUI communication.  This API allows...

  * Fronts to be made in any language, without the use of an IDL
  * Multiple fronts to read and respond to the core SIMULTANEOUSLY
  * The command stream to be saved and then recalled at a later time
    (and have an 'instant-replay' feature!)
  * The front(s) and core to be separated via simple Internet socket

And I don't think GMS's API for core-to-GUI communication will have to be
changed.  Brad can probably plug Loci's mechanism for core-to-front
communication directly into GMS.  We'll see.

> This supports having multiple 'clients'
> attached to the core (DFP, or dataflow prosessor) even remotely across a network.

Can one client replicate the actions of another client that is operated by the
user?  For example, the user connects 2 lines in one client and a second
client does the same exact thing ON ITS OWN.

You see, this is important to Loci, because we want to have a console-based
client that can control the GUI client.  Fo instance, the user could type...

    "connect node 3728 to node 4578"

And the GUI would do it.  Better yet, with speech recognition (maybe Jean-Marc
can help with this), a user (perhaps a blind user) can speak...

    "connect node 3728 to node 4578"

And the GUI would do it!  How does this sound to you?

>  I suggest that communication between cores takes place by extending the pipeline
> functionality by a sharing ability.

Hmmm.  I like the idea in Loci of self-referencing (recursive) cores.  Could
we do that?

> All other points are already started or are fine to me. XML is the mayor item gms
> is missing.

Well, you're lucky to have Brad help then.  He's a whiz when it comes to XML.

BTW, Overflow and Loci take VERY similar approaches to the use of XML.  It
would be nice if we all standardized...maybe even just for the sake of
neatness and comprehension.

> But first everybody should  have a good look at each others code\situation. I
> might be better to do some more work seperately before any intergration attempt
> is done. I want to complete verson 0.4.0 which will take about 1-2 weeks.

Yeah, I think we need some time to look at each other's ideas, plus a
cooling-down period :-)  I want to re-package and post some of Loci's more
important ideas in that time too.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 19:08:28 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
References: <Pine.A41.4.10.10003141821380.109636-100000@arch14.cc.uga.edu>
Message-ID: <38CED47C.9A897FCC@geoserve.net>

Brad Chapman wrote:
> 
> I would personally really like to see Overflow, GMS and Loci all coming
> together. For my personal use, I would really be interested in having a
> mechanism for wrapping "smaller" objects than just programs. While you may
> not often want to pipe these small objects over internet connections, you
> may way to combine a whole bunch of local small objects, and pipe the
> output to another object on a different machine.

Yes!  This is exactly what we could do by making Overflow an APPLICATION
DEVELOPMENT NODE!  Overflow even has a nice way to generate command-lines for
those 'unspecified' or 'unknown' applications that we come across (we can use
Overflow to give us simple connectivity).

Or, hey, how about this: The plugin/wrapper needed to run a specific
application under GMS can be constructed or specified using Overflow?!

> This is where a
> combination of all three programs really makes a lot of sense to me.
>         For those on the list interested in bioinformatics stuff, I'm
> specifically thinking of the bioperl/biojava/biopython projects which have
> a combined idl specifying a general set of functions for all three
> projects. Wrapping these functions seems like something Overflow is
> ideally suited to do, while GMS and Loci don't address such small data
> structures.

Right!

>         For such a powerful system as we all are proposing, we really need
> to include both small and large structures, or I think we'll be sorry
> later. Another 2 cents.

Exactly!  Gee I really like this kid :-)

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 19:13:20 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <Pine.A41.4.10.10003141804510.109636-100000@arch14.cc.uga.edu>
Message-ID: <38CED5A0.EC51982B@geoserve.net>

Brad Chapman wrote:
> 
> What I'd like to think about doing is keeping the small middle that we
> already have in python, and then building the core off of this using
> CORBA. Some of the reasons I would like to try this are:
> 
> 1. Reuse of code: I'm not too pumped to be reprogramming the same stuff in
> C (Python to C converters excluded).

For now.  We'll have to see how much Jarl likes or dislikes a heterogeneous
programming environment.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 19:16:44 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
References: <Pine.A41.4.10.10003141821380.109636-100000@arch14.cc.uga.edu> <38CED47C.9A897FCC@geoserve.net>
Message-ID: <38CED66C.5CBBFE41@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> Or, hey, how about this: The plugin/wrapper needed to run a specific
> application under GMS can be constructed or specified using Overflow?!

Then every node would run through Overflow.  GMS could provide the
authentication/security before handing the dataflow to Overflow.  And we
wouldn't get a performance hit, because Overflow is so fast.

How does this sound?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 19:31:10 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
References: <Pine.A41.4.10.10003141821380.109636-100000@arch14.cc.uga.edu> <38CED47C.9A897FCC@geoserve.net> <38CED66C.5CBBFE41@geoserve.net>
Message-ID: <38CED9CE.89AFED83@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> Then every node would run through Overflow.  GMS could provide the
> authentication/security before handing the dataflow to Overflow.  And we
> wouldn't get a performance hit, because Overflow is so fast.

>From a Loci GUI point of view, every GMS-node would contain a minimum of 1
Overflow-node.  If no interface/widget is specified by the system, the
GMS-node's windowlet would contain an Overflow-workspace with exactly 1
Overflow-node.

If an interface/widget is specified (via GLADE XML or Bonobo), the
Overflow-workspace can be 'replaced' by that interface/widget.  This is a Loci
GUI idea we've had for a while: The user can switch back and forth between
workspace and interface/widget.  And the heirarchy of nodes in a network
determines the placement of widgets in the interface.  This may be hard to
grasp, so I'll post some more about it later.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 14 20:59:33 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: [Fwd: [Pipet Devel] Re: Overflow/Loci/GMS collaboration]
References: <Pine.A41.4.10.10003141821380.109636-100000@arch14.cc.uga.edu> <38CED47C.9A897FCC@geoserve.net> <38CED66C.5CBBFE41@geoserve.net> <38CED9CE.89AFED83@geoserve.net>
Message-ID: <38CEEE85.B27E2EA@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> If no interface/widget is specified by the system, the
> GMS-node's windowlet would contain an Overflow-workspace with exactly 1
> Overflow-node.

Let me reword that:

If a GMS-node does NOT represent an existing application, the GMS-node's
windowlet would contain an empty Overflow-workspace.

If a GMS-node DOES represent an existing application, the GMS-node's windowlet
would contain an Overflow-workspace with a network of Overflow-nodes visible.

If an interface/widget is NOT specified by the network of Overflow-nodes, the
GMS-node's windowlet would contain an Overflow-workspace with a network of
Overflow-nodes visible.

If an interface/widget IS specified by the network of Overflow-nodes, the
GMS-node's windowlet would show that interface/widget (and be able to switch
between network and interface/widget views).

...

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Wed Mar 15 02:39:13 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Overflow/Loci/GMS
References: <200003141702.MAA110152@archa12.cc.uga.edu> <38CEB92D.8172D2A3@casema.net> <38CEC1BA.6DC78FF6@geoserve.net> <38CEC751.63959AA1@casema.net> <38CED231.65D65D23@geoserve.net>
Message-ID: <38CF3E20.5723B348@casema.net>

>
> > This supports having multiple 'clients'
> > attached to the core (DFP, or dataflow prosessor) even remotely across a network.
>
> Can one client replicate the actions of another client that is operated by the
> user?  For example, the user connects 2 lines in one client and a second
> client does the same exact thing ON ITS OWN.
>

Race exceptions, they should be handled. Also when some day there's support added for
shareing between instances.

> can help with this), a user (perhaps a blind user) can speak...
>
>     "connect node 3728 to node 4578"
>
> And the GUI would do it!  How does this sound to you?

Very very nice :)


>
> Hmmm.  I like the idea in Loci of self-referencing (recursive) cores.  Could
> we do that?

Self referencing nodes?





From jarl at casema.net  Wed Mar 15 02:42:47 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <Pine.A41.4.10.10003141804510.109636-100000@arch14.cc.uga.edu> <38CED5A0.EC51982B@geoserve.net>
Message-ID: <38CF3EF7.99F72734@casema.net>

> > What I'd like to think about doing is keeping the small middle that we
> > already have in python, and then building the core off of this using
> > CORBA. Some of the reasons I would like to try this are:
> >
> > 1. Reuse of code: I'm not too pumped to be reprogramming the same stuff in
> > C (Python to C converters excluded).
>
> For now.  We'll have to see how much Jarl likes or dislikes a heterogeneous
> programming environment.
>

That's depending on the technical possibilities mostly. I'll try to get
familiair with python-C hooks, I cant judge the possibilities right now. Anyone
knows more about this?



From bizzaro at geoserve.net  Wed Mar 15 06:44:00 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <Pine.A41.4.10.10003141804510.109636-100000@arch14.cc.uga.edu> <38CED5A0.EC51982B@geoserve.net> <38CF3EF7.99F72734@casema.net>
Message-ID: <38CF7780.83623634@geoserve.net>

jarl van katwijk wrote:
> 
> > > What I'd like to think about doing is keeping the small middle that we
> > > already have in python, and then building the core off of this using
> > > CORBA. Some of the reasons I would like to try this are:
[...]
> That's depending on the technical possibilities mostly. I'll try to get
> familiair with python-C hooks, I cant judge the possibilities right now. Anyone
> knows more about this?

In any case, what we need to do is have Overflow and GMS use THE SAME
core-to-GUI API.  Or at least something very similar.  That just makes it
easier for people working on the GUI, since we really need 2 GUIs that work
alike: one for Overflow and one for GMS.  This should be one 'point of
agreement'.  We need to make a whole list of these.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jturner at raba.com  Wed Mar 15 07:34:04 2000
From: jturner at raba.com (Jeff Turner)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] orbit/python bindings (was Re: [Pipet Devel] GMS) 
Message-ID: <38CF833B.8E8D50E1@raba.com>

> ...
> BTW, do you know of anyone working on Python bindings to Bonobo?  I
saw Miguel
> de Icaza speek last week and give a demo of Bonobo.  He showed an 'SVG
Bonobo
> component'.  I think this would be great to use for visuals, but we
need the
> Python bindings to do it.
> ...

the following message came across the bonobo mailing list yesterday,
and shows how you can use omniORBPy and a cvs version of bonobo to
script gnumeric (the gnome spreadsheet)


------- Forwarded Message

Date: Tue, 14 Mar 2000 21:10:46 +0100
From: Jon K Hellan <hellan@acm.org>
To: gnome-components-list@gnome.org
Subject: Scripting with Python and omniORB

Hi,

I've made some experiments with Gnome CORBA objects and omniORB for
Python. omniORB is fast, even with Python, as you kan see in a
separate article.

omniidl is the idl compiler of omniORB. To compile Gnome IDL and
produce Python stubs and skeletons, you need the following flags:

   omniidl -bpython -Wbinline <idlfile>

I used the latest omniORBpy with the official omniORB 2.8.0
release. According to the announcement, omniORB 3.0 should now also
work.

You need the latest Bonobo from CVS: We used some constructs which
weren't standard, and omniORB is strict about checking. The IDL in CVS
is now compatible with omniORB.

The Gnorba authentication mechanism isn't portable, so in order to
experiment with a "foreign" ORB, you have to disable authentication. I
hacked gnome_ORBit_request_validate in gnome-libs/libgnorba/orbitgtk.c
to always return ORBIT_MESSAGE_ALLOW_ALL. This is something you only
want to do behind a firewall where you can trust everybody.

Here's a small script to walk the naming tree, and a script to
exercise Gnumeric. You have to write the IOR of the gnome name service
to /tmp/naming.ref before trying these scripts. And you must change
the file name given to factory.read() in the Gnumeric script.

Jon K??re

- --

- --
#!/usr/bin/env python

# nsbrowse.py
#
# Author: Jon K Hellan <hellan@acm.org>
#
# Do what you wish with this script. No warranties - if it breaks, you
get
# keep both pieces.

"""Script to walk the naming tree of a CORBA name service.
Uses omniORBpy. Can locate the Gnome Name Service."""

# Standard/built-in modules.
import sys

# omniORB modules.
from omniORB import CORBA
import CosNaming

## gnames find the Gnome Name Service in the same way as gnome apps do.
#try:
#    import gnames
#except:
#    print """Did not find the "gnames" extension module.
#Use orb.resolve_initial_references or read the IOR from a file."""

def name_to_string(name):
    """Convert CosNaming Name to a list of (id, kind) tuples."""
    res = []
    for i in range(len(name)):
        binding_name = name[i]
        res = res + [(binding_name.id, binding_name.kind)]
    return `res`

def walk(sofar, nameservice, visitor):
    """Walk the naming tree. Call visitor at object nodes."""
    (bl, bi) = nameservice.list(10000)
    print "%d bindings at naming context %s" % (len(bl),
name_to_string(sofar))
    print "binding iterator: " + `bi`

    for i in range(len(bl)):
        binding = bl[i]
        try:
            obj = nameservice.resolve(binding.binding_name)
        except CosNaming.NamingContext.NotFound:
            print "Could not find %s" % name_to_string(sofar
                                                       +
binding.binding_name)
            continue

        if binding.binding_type is CosNaming.ncontext:
            walk(sofar + binding.binding_name, obj, visitor)
        else:
            visitor(sofar, binding, obj)

def walk_print(nameservice):
    """Walk the naming tree. Print object nodes."""
    def printit(sofar, binding, obj):
        """Print object node."""
        print name_to_string(sofar + binding.binding_name)
        print obj

    walk([], nameservice, printit)

# Initialise the ORB.
orb = CORBA.ORB_init(sys.argv, CORBA.ORB_ID)

# Locate the name service.
# This is how it's supposed to work:
#nameservice=orb.resolve_initial_references("NameService")
# This is the last resort - have the name server write IOR to file.
ior = open("/tmp/naming.ref").read()
# This works with the Gnome name service and the 'gnames' extension
module.
#ior = gnames.get_ior()
nameservice = orb.string_to_object(ior)

# Breaks with normal libgnorba:
walk_print(nameservice)

- --
#!/usr/bin/env python

# gntest.py
#
# Author: Jon K Hellan <hellan@acm.org>
#
# Do what you wish with this script. No warranties - if it breaks, you
get
# keep both pieces.

"""Script to explore the CORBA interface of Gnumeric.
Uses omniORBpy.

Gnome uses an authentication mechanism for CORBA interactions which only

works with the ORBit ORB. To use other ORBs, you have to disable
authentication. Change gnome_ORBit_request_validate in
gnome-libs/libgnorba/orbitgtk.c to always return
ORBIT_MESSAGE_ALLOW_ALL.
Needless to say, you should only do this if trust *everybody* who can
access your system.
"""
# Standard/built-in modules.
import sys

# omniORB modules.
from omniORB import CORBA
import CosNaming
import GNOME

## gnames find the Gnome Name Service in the same way as gnome apps do.
#try:
#    import gnames
#except:
#    print """Did not find the "gnames" extension module.
#Use orb.resolve_initial_references or read the IOR from a file."""

def name_to_string(name):
    """Convert CosNaming Name to a list of (id, kind) tuples."""
    res = []
    for i in range(len(name)):
        binding_name = name[i]
        res = res + [(binding_name.id, binding_name.kind)]
    return `res`

# Initialise the ORB.
orb = CORBA.ORB_init(sys.argv, CORBA.ORB_ID)

# Locate the name service.
# This is how it's supposed to work:
#nameservice=orb.resolve_initial_references("NameService")
# This is the last resort - have the name server write IOR to file.
ior = open("/tmp/naming.ref").read()
## This works with the Gnome name service and the 'gnames' extension
module.
#ior = gnames.get_ior()
nameservice = orb.string_to_object(ior)

def resolve_goad_object(nameservice, id):
    """Resolve a name with the standard Gnome prefix."""
    nc = CosNaming.NameComponent
    prefix = [nc('GNOME', 'subcontext'), nc('Servers', 'subcontext')]
    name   = prefix + [nc(id, 'object')]
    return nameservice.resolve(name)

# Breaks with normal libgnorba:
#walk_print(nameservice)
try:
    factory = resolve_goad_object(nameservice,

'GOADID:GNOME:Gnumeric:WorkbookFactory:1.0')
except CosNaming.NamingContext.NotFound:
    print 'Gnumeric not found'
    sys.exit()
#Other exceptions ...

if not factory._is_a('IDL:GNOME/Gnumeric/WorkbookFactory:1.0'):
    raise 'This is not a Gnumeric WorkbookFactory'

newbook = factory.read("/home/jk/tmp/test.gnumeric")
sc=newbook.sheet_current()

sc.cursor_set(4, 4, 1, 1, 5, 5)

sc.make_cell_visible(4, 500)
sc.cursor_move(3, 500)
sc.cursor_move(1, 1)
sc.make_cell_visible(1, 1)
sc.make_cell_visible(0, 0)

v=GNOME.Gnumeric.Value(GNOME.Gnumeric.VALUE_FLOAT, 1)
sc.cell_set_value(1, 1, v)
v=GNOME.Gnumeric.Value(GNOME.Gnumeric.VALUE_BOOLEAN, 1)
sc.cell_set_value(1, 3, v)
v=GNOME.Gnumeric.Value(GNOME.Gnumeric.VALUE_FLOAT, 3.1415926)
sc.cell_set_value(1, 4, v)


- --
        FAQ: Frequently-Asked Questions at http://www.gnome.org/gnomefaq

         To unsubscribe: mail gnome-components-list-request@gnome.org
with
                       "unsubscribe" as the Subject.

------- End of Forwarded Message





From bizzaro at geoserve.net  Wed Mar 15 07:50:09 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
Message-ID: <38CF8701.B8729187@geoserve.net>

Greetings.

I thought we should try to come to an agreement about a name for the
collaborative project, and then get a mailing list set up ASAP.

Jean-Marc suggested the name VSH or 'Visual Shell'.  I was able find a very
old and now dead file manager for UNIX that went by the same name.  But
Jean-Marc says that does not matter to him.

Personally, I like the name, because I always thought of Loci as a graphical
(GSH is taken) or visual shell.  (I still love the name Loci, but we're no
more than 1/3rd the collaborative now and we have to make compromises.)  But I
think we should, at least in announcements and descriptions, say that this is
'the new VSH'.  That may pique people's curiousity too.

So, we're missing the vote of the remaining 1/3rd of the collaborative: Jarl.

I'd also like to get opinions from famous Locians like Gary, Brad, Justin,
David, Tim, Humberto, Jennifer, Alan, etc.  (Loci has 40 people on the mailing
list, BTW, which is something I am proud of, but perhaps attributable to Loci
having a list for more than a year.)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From dlapointe at mediaone.net  Wed Mar 15 07:49:13 2000
From: dlapointe at mediaone.net (David Lapointe)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
In-Reply-To: <38CF8701.B8729187@geoserve.net>
References: <38CF8701.B8729187@geoserve.net>
Message-ID: <00031507501000.00538@gnomen>

How about Visgnu?


-- 
 .david
 David Lapointe
If you are not confused 
you're not paying attention. -- Will Durst


From jarl at casema.net  Wed Mar 15 08:06:39 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net>
Message-ID: <38CF8ADF.737FC1EF@casema.net>

"J.W. Bizzaro" wrote:

> Greetings.
>
> I thought we should try to come to an agreement about a name for the
> collaborative project, and then get a mailing list set up ASAP.
>
> Jean-Marc suggested the name VSH or 'Visual Shell'.  I was able find a very
> old and now dead file manager for UNIX that went by the same name.  But
> Jean-Marc says that does not matter to him.
>
> Personally, I like the name, because I always thought of Loci as a graphical
> (GSH is taken) or visual shell.  (I still love the name Loci, but we're no
> more than 1/3rd the collaborative now and we have to make compromises.)  But I
> think we should, at least in announcements and descriptions, say that this is
> 'the new VSH'.  That may pique people's curiousity too.
>
> So, we're missing the vote of the remaining 1/3rd of the collaborative: Jarl.

VSH is nice, but I would like to see 'it' running without being noticed too..
just doing it's work. I like to have a 'A' added, Automation.
Or does VASH again has some unwanted side effects like 'GIA' ? :)



From chapmanb at arches.uga.edu  Wed Mar 15 10:43:00 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
Message-ID: <200003151545.KAA125764@archa12.cc.uga.edu>


Jarl wrote:
> That's depending on the technical possibilities mostly. I'll try to 
get
> familiair with python-C hooks, I cant judge the possibilities right 
now.
> Anyone
> knows more about this?

When I was proposing this, I was thinking of the small middle which we 
currently have talking to Overflow and GMS through corba (ie. GMS 
could keep the same structure it currently has, but would talk to the 
middle that we currently have instead of directly to the front.
    Of course, it would also be possible to wrap the C and C++ classes 
so they can be called from python. There is a tutorial about this on 
the python documentation web site (http://www.python.org/doc/) called 
"Extending and Embedding." The C API which is used to connect outside 
classes with the python core code is also documented there ("Python/C 
API"). There are also C++ classes that sit on top of these C classes 
which make wrapping C++ possible.
    There is also a tool to do these wrappings semi-automatically 
called SWIG (http://www.swig.org/). I've heard both good and bad 
things about this, but it is also a possibility is we use the method 
of directly embedding the C/C++ classes to be callable from a python 
middle.

Brad



From bizzaro at geoserve.net  Wed Mar 15 23:59:46 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
Message-ID: <38D06A42.75098B53@geoserve.net>

Brad Chapman wrote:
> 
> When I was proposing this, I was thinking of the small middle which we
> currently have talking to Overflow and GMS through corba (ie. GMS
> could keep the same structure it currently has, but would talk to the
> middle that we currently have instead of directly to the front.

When you think about it, EVERYTHING must be passed through GMS first.  Since
Overflow will be (if Jean-Marc agrees) a node in GMS, the GUI must talk only
to GMS.  GMS would then talk to Overflow.  The GUI simply cannot bypass GMS
and talk to Overflow.

I know I said we need to make 2 GUIs.  We still do, but (and this is a Loci
design) the Overflow GUI is only a REPRESENTATION of Overflow.  Remember that
we'll use XML to REPRESENT the actual applications being integrated.  So, the
Overflow GUI will puke out XML.

It would probably be best then to make GMS communicate directly with Overflow,
either via the API GMS uses for all applications or via special API.  Any
ideas?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 16 00:26:47 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Introduction to the GConf library
Message-ID: <38D07097.D8BA61EE@geoserve.net>

http://developer.gnome.org/feature/current/index.html

Jeff


From bizzaro at geoserve.net  Thu Mar 16 00:32:59 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net>
Message-ID: <38D0720B.8CE08448@geoserve.net>

Jarl van Katwijk wrote:
> 
> VSH is nice, but I would like to see 'it' running without being noticed too..
> just doing it's work.

That's a good point.  Much of the collaborative will be non-graphical, so is
it really accurate calling it a 'visual shell'?  What do you think, Jean-Marc?

> I like to have a 'A' added, Automation.

Visual Automation SHell?

> Or does VASH again has some unwanted side effects like 'GIA' ? :)

You mean like 'GAI'?  I don't see anything wrong.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 16 00:45:42 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
Message-ID: <38D07506.F7CA4E00@geoserve.net>

How about...

  Iconic

or

  Iconic Shell

or

  Glyph

or

  Glyphic

or

  Glyphic Shell

?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 16 00:49:39 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net> <38D07506.F7CA4E00@geoserve.net>
Message-ID: <38D075F3.6BA13F10@geoserve.net>

Or how about...

  RAIDE

Rapid Application Integration and Development Environment

?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 16 00:54:09 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net> <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net>
Message-ID: <38D07701.56BB0AEA@geoserve.net>

Or...

    Gadget

?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Thu Mar 16 02:27:14 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Re: Overflow/Loci/GMS collaboration
References: <38D06A42.75098B53@geoserve.net>
Message-ID: <38D08CD2.F90F58A6@casema.net>

>
> It would probably be best then to make GMS communicate directly with Overflow,
> either via the API GMS uses for all applications or via special API.  Any
> ideas?
>

Hi,

I dont know if that was already clear, but gms nodes have this pointer  :
gint   (*procedure)(messaging_object *MO);

Normally this procedure does 1) take care of the processing ability of the node
or 2) it wraps an external application.

2) will kill the speed Overflow introduces, but 1) can be done, we could load
overflow as a dynamic library (every application
can be turned into such easaly) and have the 'procedure pointer calling a 'entry'
function in Overflow. By 'entry' I main a function
that does NOT initialise but only executes a subnet. This way the wrapping has
very little overhead.

Futhermore there can be made an api which lets the GUI interact directly with
this subnet in order to have the full power of the small\detailed nodes Overflow
has.

bye,
jarl



From jarl at casema.net  Thu Mar 16 02:29:14 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Introduction to the GConf library
References: <38D07097.D8BA61EE@geoserve.net>
Message-ID: <38D08D4A.ABCBEFF3@casema.net>

> http://developer.gnome.org/feature/current/index.html
>
>

Very interesting.. the 'wrapping of external application'
code should definitly make somehow use of Gconf.



From jarl at casema.net  Thu Mar 16 02:30:28 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
Message-ID: <38D08D94.EFD4A342@casema.net>

> That's a good point.  Much of the collaborative will be non-graphical, so is
> it really accurate calling it a 'visual shell'?  What do you think, Jean-Marc?
>
> > I like to have a 'A' added, Automation.
>
> Visual Automation SHell?

Everyone pleased by VASH? I am...



From jarl at casema.net  Thu Mar 16 02:32:01 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net> <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net>
Message-ID: <38D08DF1.61849A22@casema.net>

>
>   RAIDE
>
> Rapid Application Integration and Development Environment
>

Even better as vash..
I vote for RAIDE

bye,
jarl



From jarl at casema.net  Thu Mar 16 03:04:56 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Scheme
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net> <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D08DF1.61849A22@casema.net>
Message-ID: <38D095A8.75805508@casema.net>

I tried to collect all proposals made..

maybe it's best we edit this dia document to keep everybody
informed about the idears?
(dia :  http://www.lysator.liu.se/~alla/dia/)

bye,
jarl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lociGMSoverflow.dia
Type: application/octet-stream
Size: 1224 bytes
Desc: not available
Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000316/c44da45e/lociGMSoverflow.obj
From jarl at casema.net  Thu Mar 16 08:59:15 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] Good info source
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net> <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D08DF1.61849A22@casema.net>
Message-ID: <38D0E8B3.F2086912@casema.net>

check this site : http://www.fipa.org





From bizzaro at geoserve.net  Thu Mar 16 10:07:43 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net> <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net>
Message-ID: <38D0F8BF.46D5C167@geoserve.net>

"J.W. Bizzaro" wrote:
> 
>   RAIDE
> 
> Rapid Application Integration and Development Environment

    Blockaide
    Block-aide

  Block Application Integration and Development Environment

?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Thu Mar 16 12:17:40 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:23 2006
Subject: [Pipet Devel] vsh?
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
 <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D07701.56BB0AEA@geoserve.net>
Message-ID: <38D11734.C4E93A4E@hermes.usherb.ca>

Hi,

I still think there's nothing that keeps Overflow from doing distributed stuff,
and I think I've found a way to do it (simply by creating two new node types).
Now the problem is that I've never used sockets before. Is anyone interested in
helping out. This would actually serve two goals at once: See it we can get
networking in Overflow, and have someone in Loci (or GMS) to get some
"experience" with Overflow. Anyone interrested?

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Thu Mar 16 12:36:35 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Scheme
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net> <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D08DF1.61849A22@casema.net> <38D095A8.75805508@casema.net>
Message-ID: <38D11BA3.821D885D@geoserve.net>

jarl van katwijk wrote:
> 
> I tried to collect all proposals made..

I converted it png, so you won't need Dia.  Attached.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lociGMSoverflow.png
Type: image/png
Size: 46605 bytes
Desc: not available
Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000316/af1e8cc5/lociGMSoverflow.png
From bizzaro at geoserve.net  Thu Mar 16 12:47:46 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] vsh? (resent)
Message-ID: <38D11E42.C0147D88@geoserve.net>

"J.W. Bizzaro" wrote:
> 
>   RAIDE
> 
> Rapid Application Integration and Development Environment

    Blockaide
    Block-aide

  Block Application Integration and Development Environment

?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 16 12:53:13 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] which handles distribution (was: vsh?)
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
	 <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D07701.56BB0AEA@geoserve.net> <38D11734.C4E93A4E@hermes.usherb.ca>
Message-ID: <38D11F89.870A2578@geoserve.net>

Jean-Marc Valin wrote:
> 
> I still think there's nothing that keeps Overflow from doing distributed stuff,
> and I think I've found a way to do it (simply by creating two new node types).
> Now the problem is that I've never used sockets before. Is anyone interested in
> helping out. This would actually serve two goals at once: See it we can get
> networking in Overflow, and have someone in Loci (or GMS) to get some
> "experience" with Overflow. Anyone interrested?

This is a VERY important question:

  If Overflow handles application distribution and integration,
  what will happen to GMS and all of its code???  This is what
  GMS is designed to do.

We CANNOT thow GMS away.  Jarl simply wouldn't collaborate if we did.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 16 13:03:27 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
		 <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D07701.56BB0AEA@geoserve.net> <38D11734.C4E93A4E@hermes.usherb.ca> <38D11F89.870A2578@geoserve.net>
Message-ID: <38D121EF.C063F057@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> We CANNOT thow GMS away.  Jarl simply wouldn't collaborate if we did.

No one is questioning if Overflow is capable of distributing applications. 
The issue is much more political than technical.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Thu Mar 16 13:03:02 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
 <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D07701.56BB0AEA@geoserve.net>
 <38D11734.C4E93A4E@hermes.usherb.ca> <38D11F89.870A2578@geoserve.net>
Message-ID: <38D121D6.576BCCB2@hermes.usherb.ca>

> > I still think there's nothing that keeps Overflow from doing distributed stuff,
> > and I think I've found a way to do it (simply by creating two new node types).
> > Now the problem is that I've never used sockets before. Is anyone interested in
> > helping out. This would actually serve two goals at once: See it we can get
> > networking in Overflow, and have someone in Loci (or GMS) to get some
> > "experience" with Overflow. Anyone interrested?
> 
> This is a VERY important question:
> 
>   If Overflow handles application distribution and integration,
>   what will happen to GMS and all of its code???  This is what
>   GMS is designed to do.
> 
> We CANNOT thow GMS away.  Jarl simply wouldn't collaborate if we did.
> 

I'm not talking about throwing GMS away... I'm just saying that we could
probably paste the GMS code into Overflow nodes and have all the GMS
functionnality into Overflow. So far, I haven't found anything that can't be
inserted into an Overflow node (GMS might prove me wrong, but I don't think).
The suggestion of writing network code was more as a proof of concept... I don't
want to rewrite all GMS.

As for the name... why not dfsh (data-flow shell) or fsh (flow shell)? I think
RAIDE sounds a bit more like RAID... plus "raide" means "stiff" in french.

	Jean-Marc


-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jean-marc.valin at hermes.usherb.ca  Thu Mar 16 13:05:35 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
 <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D07701.56BB0AEA@geoserve.net>
 <38D11734.C4E93A4E@hermes.usherb.ca> <38D11F89.870A2578@geoserve.net> <38D121EF.C063F057@geoserve.net>
Message-ID: <38D1226F.A35AE74E@hermes.usherb.ca>

> > We CANNOT thow GMS away.  Jarl simply wouldn't collaborate if we did.
> 
> No one is questioning if Overflow is capable of distributing applications.
> The issue is much more political than technical.

My argument is not that much that I want Overflow to "take over everything", but
the fact that I think it would work better if we can have only one application,
instead of one for complex stuff and one for fast stuff. ...and I think it would
be feasable in Overflow.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jarl at casema.net  Thu Mar 16 13:14:48 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] which handles distribution (was: vsh?)
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
		 <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D07701.56BB0AEA@geoserve.net> <38D11734.C4E93A4E@hermes.usherb.ca> <38D11F89.870A2578@geoserve.net>
Message-ID: <38D12498.BFCA5EB3@casema.net>

"J.W. Bizzaro" wrote:

> Jean-Marc Valin wrote:
> >
> > I still think there's nothing that keeps Overflow from doing distributed stuff,
> > and I think I've found a way to do it (simply by creating two new node types).
> > Now the problem is that I've never used sockets before. Is anyone interested in
> > helping out. This would actually serve two goals at once: See it we can get
> > networking in Overflow, and have someone in Loci (or GMS) to get some
> > "experience" with Overflow. Anyone interrested?
>
> This is a VERY important question:
>
>   If Overflow handles application distribution and integration,
>   what will happen to GMS and all of its code???  This is what
>   GMS is designed to do.
>
> We CANNOT thow GMS away.  Jarl simply wouldn't collaborate if we did.
>

We do not have to intergrate the projects now.. we can wait, learn, try & test...
I'm gonna do what I scretched in the diagram, if it isn't the  way other project(s)
want it to be done, that's fine to me. I'll still be here anyway :)

And so I'm be collaborating by discussion anyhow.. it's only about the coding.

bye,
jarl



From jarl at casema.net  Thu Mar 16 13:18:12 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
	 <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D07701.56BB0AEA@geoserve.net>
	 <38D11734.C4E93A4E@hermes.usherb.ca> <38D11F89.870A2578@geoserve.net> <38D121EF.C063F057@geoserve.net> <38D1226F.A35AE74E@hermes.usherb.ca>
Message-ID: <38D12564.B6A7BDD1@casema.net>

> > > We CANNOT thow GMS away.  Jarl simply wouldn't collaborate if we did.
> >
> > No one is questioning if Overflow is capable of distributing applications.
> > The issue is much more political than technical.
>
> My argument is not that much that I want Overflow to "take over everything", but
> the fact that I think it would work better if we can have only one application,
> instead of one for complex stuff and one for fast stuff. ...and I think it would
> be feasable in Overflow.
>

Why not (I thing it's what Jeff said before) : come to a common GUI-core API, so
Overflow and GMS will continue, and user can deside which core fullfills their
needs?
Or will this GUI be impossible to build?






From bizzaro at geoserve.net  Thu Mar 16 13:25:43 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <38CF8701.B8729187@geoserve.net> <38CF8ADF.737FC1EF@casema.net> <38D0720B.8CE08448@geoserve.net>
	 <38D07506.F7CA4E00@geoserve.net> <38D075F3.6BA13F10@geoserve.net> <38D07701.56BB0AEA@geoserve.net>
	 <38D11734.C4E93A4E@hermes.usherb.ca> <38D11F89.870A2578@geoserve.net> <38D121D6.576BCCB2@hermes.usherb.ca>
Message-ID: <38D12727.79E5C582@geoserve.net>

Jean-Marc Valin wrote:
> 
> I'm not talking about throwing GMS away... I'm just saying that we could
> probably paste the GMS code into Overflow nodes and have all the GMS
> functionnality into Overflow. So far, I haven't found anything that can't be
> inserted into an Overflow node (GMS might prove me wrong, but I don't think).
> The suggestion of writing network code was more as a proof of concept... I don't
> want to rewrite all GMS.

Okay, now let's see what Jarl has to say.

I'm fairly impartial about this issue of which controls distribution, since I
can get the GUI to work either way.

> As for the name... why not dfsh (data-flow shell) or fsh (flow shell)? I think

I thought fsh sounded alright, but then I found that the name is being used:

    http://www.lysator.liu.se/fsh/

What do you think of Blockaide?

> RAIDE sounds a bit more like RAID... plus "raide" means "stiff" in french.

Oh man.  It's good to have some french-speaking people on board :-)

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 16 13:30:00 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Jean-Marc and Dominic are not on this list
Message-ID: <38D12828.2BD3E8E2@geoserve.net>

Guys,

Jean-Marc and Dominic are not on this list.  Please CC messages to them.

    Jean-Marc Valin <valj01@gel.usherb.ca>
    Dominic Letourneau <doumdi@yahoo.com>

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From justin at ukans.edu  Fri Mar 17 00:45:57 2000
From: justin at ukans.edu (Justin Bradford)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Ideas for network distributed objects
Message-ID: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu>

http://www.levien.com/athshe/prelim-design.html

The above was written by Raph Levien, author of 
many things, including libart, Gill, and gdome.

Anyway, way back when I first got here, there was
lots of talk about how we'd access objects over
the network. I was proposing a DOM-like interface
over the network.

gdome, part of Gnome, was a DOM implementation
accessible through CORBA. Unfortunately, that's
really slow and there are lots of problems with
it.

So back when I was looking at these things, I
talked to Raph about gdome and we discussed the
possibility of a light-weight wire protocol to
do DOM like things over the network efficiently.

The link above describes Raph's thoughts/design
ideas on such a system.

Now, here is why I think this is extremely useful
for Loci (or whatever we call it now ;)

Bioinformatic/computational data can be very 
large, and it would be bad to have to transfer 
big blocks of data when you just need a bit of 
it.

Instead, imagine a simple API to a tree-like
structure. You could quickly browse through the
structure until you found the right nodes, 
lists, and/or blobs of data. Then you just get
the data you want. Even substrings of binary
blobs.

Also, it discusses the implementation of locking
and transaction for these distributed objects,
which could be also be very useful for things
like distributed computation and simulation.

Anyway, this Athshe describes my original vision
for distributed communication/object interaction
in Loci. I'm curious what people think, and if 
such a system would still be useful for Loci's
current direction.


Justin




From chapmanb at arches.uga.edu  Fri Mar 17 08:53:13 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
Message-ID: <200003171355.IAA102600@archa10.cc.uga.edu>

Jean-Marc Valin wrote:
>>I still think there's nothing that keeps Overflow from doing 
distributed 
>> stuff,and I think I've found a way to do it (simply by creating two 
new node 
> > types).Now the problem is that I've never used sockets before. Is 
anyone
> > interested in helping out. This would actually serve two goals at 
once: See
> > it we can get networking in Overflow, and have someone in Loci (or 
GMS) to
> > get some "experience" with Overflow. Anyone interrested?

Whatever we end up trying to do, I'd definately be willing to chip in. 
I can't promise much since I just started learning sockets myself (in 
python) and am still fighting to compile Overflow (and learn 
autoconf/make), but I'll definately be willing to give it a try.

Jeff wrote:
> This is a VERY important question:
> 
>   If Overflow handles application distribution and integration,
>   what will happen to GMS and all of its code???  This is what
>   GMS is designed to do.
> 
> We CANNOT thow GMS away.  Jarl simply wouldn't collaborate if we did.

Jean-Marc Valin wrote:
> I'm not talking about throwing GMS away... I'm just saying that we 
could
> probably paste the GMS code into Overflow nodes and have all the GMS
> functionnality into Overflow. So far, I haven't found anything that 
can't be
> inserted into an Overflow node (GMS might prove me wrong, but I 
don't think).
> The suggestion of writing network code was more as a proof of 
concept... I 
> don't want to rewrite all GMS.

Jarl wrote:
> Why not (I think it's what Jeff said before) : come to a common 
GUI-core
> API, so Overflow and GMS will continue, and users can deside which 
core
> fullfills their needs?

I think it's important that Overflow and GMS aren't completely 
dependent on each other (so you could use GMS for "big" programs 
with distributed systems and use Overflow for "small" libraries on a 
local scale). However, I know for my needs that I would like to use 
both "small" and "big" together, and so I'd like to see Overflow and 
GMS interoperate, so that you don't have to choose one core over 
another. Would it be difficult to define a "distributed API" for GMS 
that Overflow could plug into (when distribution over multiple 
computer was necessary)? Then Overflow could define a "small library 
API" that GMS could plug into to run libraries? Does this make any 
sense?

Brad








From jarl at casema.net  Fri Mar 17 10:27:42 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu>
Message-ID: <38D24EEE.F47F0C77@casema.net>

>
> I think it's important that Overflow and GMS aren't completely
> dependent on each other (so you could use GMS for "big" programs
> with distributed systems and use Overflow for "small" libraries on a
> local scale). However, I know for my needs that I would like to use
> both "small" and "big" together, and so I'd like to see Overflow and
> GMS interoperate, so that you don't have to choose one core over
> another. Would it be difficult to define a "distributed API" for GMS
> that Overflow could plug into (when distribution over multiple
> computer was necessary)? Then Overflow could define a "small library
> API" that GMS could plug into to run libraries? Does this make any
> sense?
>

Yes surtainly does. I've proposed this before, but the overflow people
want to keep the processing speed
and dont want any wrapping. I like to see code like overflow intergrated
into gms, but this wont be started
soon because of the mayor priorities like XML and other that need to be
coded.

bye,
jarl



From bizzaro at geoserve.net  Fri Mar 17 12:16:34 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] new collaborative mailing list is ready!
Message-ID: <38D26872.717EC504@geoserve.net>

Okay guys,

We're going to move general discussions about this collaborative to its own
list.

The list is already made.  It's on the existing server, and the name is VSH. 
Well, VSH is the only name we could all half-heartedly agree on.  And since
this list is on a server soon to be outmoded, we can always change the name
when we move to the new server.  Another reason for not using the new server
is that it's not completely set up yet.  Makes sense.

Go to this Web page to subscribe:

  http://bioinformatics.org/mailman/listinfo/pipet-devel

Mail is to be addressed to...

  pipet-devel@bioinformatics.org

And at last, I won't have to forward 10-20 messages a day to Jean-Marc and
Dominic :-)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 17 12:55:54 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Introduction to CORBA, Part 2
Message-ID: <38D271A9.3DABC54B@geoserve.net>

http://www.linuxworld.com/linuxworld/lw-2000-03/lw-03-corba2.html

Jeff


From bizzaro at geoserve.net  Fri Mar 17 13:38:37 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu>
Message-ID: <38D27BAD.ADD4BA3E@geoserve.net>

Brad Chapman wrote:
>
> I think it's important that Overflow and GMS aren't completely
> dependent on each other (so you could use GMS for "big" programs
> with distributed systems and use Overflow for "small" libraries on a
> local scale).

I agree that there should be some separation of functionality, but to the
point where it would still be possible (with code modification) to get one
application to execute without the others being present.  IOW, I still don't
want to see GMS or Overflow dismantled, dissolved and absorbed.

BUT, if our projects are to merge, none of them will any longer be stand-alone
applications.  Of course.

> However, I know for my needs that I would like to use
> both "small" and "big" together, and so I'd like to see Overflow and
> GMS interoperate, so that you don't have to choose one core over
> another.

Right.

> Would it be difficult to define a "distributed API" for GMS
> that Overflow could plug into (when distribution over multiple
> computer was necessary)? Then Overflow could define a "small library
> API" that GMS could plug into to run libraries? Does this make any
> sense?

Yeah, I think this has to be done.  Neither Overflow nor GMS can be agnostic
about the other.  But, for Loci's sake, we don't want to be flip-flopping on
how user-to-core-to-application/object communication will occur.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 17 14:08:53 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net>
Message-ID: <38D282C5.656BDCD9@geoserve.net>

Jarl van Katwijk wrote:
> 
> Yes surtainly does. I've proposed this before, but the overflow people
> want to keep the processing speed
> and dont want any wrapping.

This whole 'speed' issue has me confused.  If the small Overflow objects
remain 'networked' on a sigle host (none are sent through the Internet), how
can GMS affect their speed???

And if small Overflow objects are 'networked' across the Internet, IT'S THE
INTERNET THAT WILL SLOW THEM DOWN, NOT A GMS WRAPPER.  Isn't that obvious to
everone else?

> I like to see code like overflow intergrated
> into gms, but this wont be started
> soon because of the mayor priorities like XML and other that need to be
> coded.

Code *LIKE* Overflow's, fine.

You know, it's looking more to me as though both Jarl's and Jean-Marc's ideas
for connecting/merging GMS and Overflow are not incompatible.  It seems
they're two sides of the same coin.  Overflow will need GMS for
Overflow-to-Overflow (across the Internet) communication, and GMS will need
Overflow for application construction (and I think wrapping).

The point of contention here is this: WHICH COMMUNICATES *DIRECTLY* WITH THE
GUI???

As I mentioned before, I'd like to see the GUI communicate with GMS, GMS
communicate with Overflow across the Internet, and Overflow handle application
wrapping/binding and development.  So, each and every GMS node is an instance
of Overflow on the Internet, which may in many cases connect to foreign
applications running on that host.

                  +------------+
                  |  Loci GUI  |
                  +------------+
                        |
                        |
                        |
                        |
                       \|/
                  +-----------+
                  |    GMS    |
                  +-----------+
                     /     \
                    /       \
         INTERNET  /         \  INTERNET
                  /           \
                 /             \
                |               |
               \|/             \|/
          +-----------+   +-----------+
          |    GMS    |   |    GMS    |
          +-----------+   +-----------+
                |               |
                |               |
               \|/             \|/
          +-----------+   +-----------+
          | Overflow  |   | Overflow  |
          +-----------+   +-----------+
                |               |
                |               |
               \|/             \|/
        +-------------+   +-------------+
        | Application |   | Application |
        +-------------+   +-------------+

Comments?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Fri Mar 17 14:57:47 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net> <38D282C5.656BDCD9@geoserve.net>
Message-ID: <38D28E3B.ADD8AB66@hermes.usherb.ca>

> > Yes surtainly does. I've proposed this before, but the overflow people
> > want to keep the processing speed
> > and dont want any wrapping.

I have nothing against wrapping in itself, nor do I think speed matters anymore
when you have the parts distributed across a network. What bothers me the most
is that your program would have to be written in "two different languages": one
GMS part, and one Overflow part, while I think it would be feasable to put the
GMS code (not rewrite it) into Overflow. Now, maybe the best thing would be to
do both: Adapt Overflow so that it can be used as a GMS node, but also add
network stuff to Overflow. The kind of task you want to perform will then tell
you which to use.

> 
> This whole 'speed' issue has me confused.  If the small Overflow objects
> remain 'networked' on a sigle host (none are sent through the Internet), how
> can GMS affect their speed???
> 
> And if small Overflow objects are 'networked' across the Internet, IT'S THE
> INTERNET THAT WILL SLOW THEM DOWN, NOT A GMS WRAPPER.  Isn't that obvious to
> everone else?

True, my concern with speed was only that Overflow nodes could not be put in GMS
individually, but that's probably obvious to everybody. Sorry, I guess I have
not expresed myself correctly on speed. 


> You know, it's looking more to me as though both Jarl's and Jean-Marc's ideas
> for connecting/merging GMS and Overflow are not incompatible.  It seems
> they're two sides of the same coin.  Overflow will need GMS for
> Overflow-to-Overflow (across the Internet) communication, and GMS will need
> Overflow for application construction (and I think wrapping).
> 
> The point of contention here is this: WHICH COMMUNICATES *DIRECTLY* WITH THE
> GUI???

I'm afraid we might need two GUIs (though they might be bonobized to look like
one), because GMS and Overflow will probably have different requirements. This
is what I'd find annoying in wrapping Overflow in GMS. I might be wrong on that
though...

About Overflow integration in GMS... I'd like to point out that, except for the
GUI, Overflow is more a library than a program. It is easy to integrate into any
program. I'd like to know, is GMS build a bit the same?

OK, so this summerized my thoughts:
1) I don't mind if Overflow is integrated in GMS (mainly it's not a lot of work)
and I can help those who want to do it (It would probably be as simple as
writing a couple new nodes). 
2) I would like to try the opposite so for most simple "networked applications",
just Overflow would be fine.

As a last, about communicating objects between different parts, Overflow already
has an Object hierarchy (GTK-like, but cleaner) that supports
serialization/de-serialization, so most of the stuff would already be done. Does
GMS has something like that (in which case, we'd need to have compatibility
between the two)?

I hope it clarifies a bit what I think.

Regards,
	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jean-marc.valin at hermes.usherb.ca  Fri Mar 17 15:03:35 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:24 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net> <38D282C5.656BDCD9@geoserve.net>
Message-ID: <38D28F97.D174716B@hermes.usherb.ca>

> > Yes surtainly does. I've proposed this before, but the overflow people
> > want to keep the processing speed
> > and dont want any wrapping.

I have nothing against wrapping in itself, nor do I think speed matters anymore
when you have the parts distributed across a network. What bothers me the most
is that your program would have to be written in "two different languages": one
GMS part, and one Overflow part, while I think it would be feasable to put the
GMS code (not rewrite it) into Overflow. Now, maybe the best thing would be to
do both: Adapt Overflow so that it can be used as a GMS node, but also add
network stuff to Overflow. The kind of task you want to perform will then tell
you which to use.

> 
> This whole 'speed' issue has me confused.  If the small Overflow objects
> remain 'networked' on a sigle host (none are sent through the Internet), how
> can GMS affect their speed???
> 
> And if small Overflow objects are 'networked' across the Internet, IT'S THE
> INTERNET THAT WILL SLOW THEM DOWN, NOT A GMS WRAPPER.  Isn't that obvious to
> everone else?

True, my concern with speed was only that Overflow nodes could not be put in GMS
individually, but that's probably obvious to everybody. Sorry, I guess I have
not expresed myself correctly on speed. 


> You know, it's looking more to me as though both Jarl's and Jean-Marc's ideas
> for connecting/merging GMS and Overflow are not incompatible.  It seems
> they're two sides of the same coin.  Overflow will need GMS for
> Overflow-to-Overflow (across the Internet) communication, and GMS will need
> Overflow for application construction (and I think wrapping).
> 
> The point of contention here is this: WHICH COMMUNICATES *DIRECTLY* WITH THE
> GUI???

I'm afraid we might need two GUIs (though they might be bonobized to look like
one), because GMS and Overflow will probably have different requirements. This
is what I'd find annoying in wrapping Overflow in GMS. I might be wrong on that
though...

About Overflow integration in GMS... I'd like to point out that, except for the
GUI, Overflow is more a library than a program. It is easy to integrate into any
program. I'd like to know, is GMS build a bit the same?

OK, so this summerized my thoughts:
1) I don't mind if Overflow is integrated in GMS (mainly it's not a lot of work)
and I can help those who want to do it (It would probably be as simple as
writing a couple new nodes). 
2) I would like to try the opposite so for most simple "networked applications",
just Overflow would be fine.

As a last, about communicating objects between different parts, Overflow already
has an Object hierarchy (GTK-like, but cleaner) that supports
serialization/de-serialization, so most of the stuff would already be done. Does
GMS has something like that (in which case, we'd need to have compatibility
between the two)?

I hope it clarifies a bit what I think.

Regards,
	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jarl at casema.net  Fri Mar 17 15:41:27 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net> <38D282C5.656BDCD9@geoserve.net>
Message-ID: <38D29877.EB1252F8@casema.net>

>
> This whole 'speed' issue has me confused.  If the small Overflow objects
> remain 'networked' on a sigle host (none are sent through the Internet), how
> can GMS affect their speed???

Every data flow will go thru
a sensor ->
    -> the pipenline ->
        -> collector (can contain a application, a internal procedure or a
overflow subnet) ->
            -> visual (GUI or whatever).
This currently goes pretty slow. Lots of IIOP traffic and memory access.  I'm
currently coding
a benchmark plugin.. more on this later.

NOTE that the collector can also read it's own data from disk, open local
sockets, etc
BUT it will be authenticated\controlled\bla-bla-bla.


>
>
> And if small Overflow objects are 'networked' across the Internet, IT'S THE
> INTERNET THAT WILL SLOW THEM DOWN, NOT A GMS WRAPPER.  Isn't that obvious to
> everone else?

Hehe.. I think the concept of just using the overflow library for subnets caused
some confusion :)

>
> As I mentioned before, I'd like to see the GUI communicate with GMS, GMS
> communicate with Overflow across the Internet, and Overflow handle application
> wrapping/binding and development.  So, each and every GMS node is an instance
> of Overflow on the Internet, which may in many cases connect to foreign
> applications running on that host.
>
>                   +------------+
>                   |  Loci GUI  |
>                   +------------+
>                         |
>                         |
>                         |
>                         |
>                        \|/
>                   +-----------+
>                   |    GMS    |
>                   +-----------+
>                      /     \
>                     /       \
>          INTERNET  /         \  INTERNET
>                   /           \
>                  /             \
>                 |               |
>                \|/             \|/
>           +-----------+   +-----------+
>           |    GMS    |   |    GMS    |
>           +-----------+   +-----------+
>                 |               |
>                 |               |
>                \|/             \|/
>           +-----------+   +-----------+
>           | Overflow  |   | Overflow  |
>           +-----------+   +-----------+
>                 |               |
>                 |               |
>                \|/             \|/
>         +-------------+   +-------------+
>         | Application |   | Application |
>         +-------------+   +-------------+
>

Fine. I will erase 'Application wrapping' of my TODO..
why was there so much email needed for this concept 8)




From bizzaro at geoserve.net  Tue Mar 21 12:11:25 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] vsh web page
Message-ID: <38D7AD3D.4CFAA892@geoserve.net>

I made a simple Web page for 'VSH' (if that is to be the name):

    http://bioinformatics.org/vsh/

with screenshots and links to the respective projects.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 21 11:57:53 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu>
Message-ID: <38D7AA11.3796AA@geoserve.net>

Justin Bradford wrote:
> 
> http://www.levien.com/athshe/prelim-design.html
[...]
> So back when I was looking at these things, I
> talked to Raph about gdome and we discussed the
> possibility of a light-weight wire protocol to
> do DOM like things over the network efficiently.
> 
> The link above describes Raph's thoughts/design
> ideas on such a system.

I'd like to hear comments from Jarl, Dominic, Jean-Marc and Brad on this.

> Now, here is why I think this is extremely useful
> for Loci (or whatever we call it now ;)
> 
> Bioinformatic/computational data can be very
> large, and it would be bad to have to transfer
> big blocks of data when you just need a bit of
> it.

Right.

> Instead, imagine a simple API to a tree-like
> structure. You could quickly browse through the
> structure until you found the right nodes,
> lists, and/or blobs of data. Then you just get
> the data you want. Even substrings of binary
> blobs.

...but using DOM instead of XML.

> Also, it discusses the implementation of locking
> and transaction for these distributed objects,
> which could be also be very useful for things
> like distributed computation and simulation.

Yes.  At a lower level than what we've been considering, these things are
important.  Now that we will be working with low-level objects via Overflow,
we should look into this.

> Anyway, this Athshe describes my original vision
> for distributed communication/object interaction
> in Loci. I'm curious what people think, and if
> such a system would still be useful for Loci's
> current direction.

Currently, we have 3 different ideas about how distribution can be handled:
one for each project.  Our challenge is to bring these ideas together.

Athshe is (yet) another way to implement object distribution and, having
little experience with DOM, is perhaps a bit out of my area of expertise.  So,
I'll defer to comment.  I want to hear what Jarl, Dominic, Jean-Marc and Brad
have to say.

Justin, if you want to see Athshe or an Athshe-like system implemented in VSH,
you're more than welcome to help code it :-)  This is a GNU project afterall.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 21 12:11:10 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] barCode
Message-ID: <38D7AD2E.8BAFE876@geoserve.net>

developerWorks wrote an article about the barCode event:

  http://www-4.ibm.com/software/developer/library/barcode/

Jeff


From bizzaro at geoserve.net  Tue Mar 21 15:06:39 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] CHANGES
References: <200003182206.RAA175796@archa11.cc.uga.edu>
Message-ID: <38D7D64F.C5059364@geoserve.net>

3/21/00 Brad
* Fixed deletion of loci so it now occurs through the middle. So now a 
shift-1 click on a loci in the workspace will delete it.

3/14/00 Brad
* Tons o' changes. Middle and front are now separate (yay!) with limited 
functionality. Specifically, DB table views do not work, and loci 
deletion (via shift-button1) does not work. Also, only the gadfly database 
works currently. Hopefully these problems will be fixed soon.
* Finished making the name changes so now at least everything being used in 
the middle is following the new case formatting style.
* Added a doc directory with some information.

20000313 J.W. Bizzaro <jeff@bioinformatics.org>
* Added 'Getting Started' to README.
* Made a couple changes to get loci-gnome working as an independent
executable.
* Made menu modules for workspace and connectors...to be implemented later.
* Separated windowlet creation into its own module.
* Started separating connector creation into own module.

(from a message off of the list)
Brad Chapman wrote:
> 
> Do you have any idears for the problems I mention?

On 3/14/00 you mention some DB problems and the locus deletion problem.  I see
you got the locus deletion working again.  BTW, I will make that work from a
popup menu; I'm not so sure shift-button1 click is obvious.

As for databases, can you provide small samples (I think I mentioned this
before) and default values to go in the database selection form?  I haven't
been able to get this to work.  Samples should probably go under...

    back/private/?

And of course, it would be nice to eventually be able to DnD databases onto
the workspace rather than fill out a form.

About the doc directory, is this something that cannot go in the loci-doc
module?  Do you think documentation should come with the main distribution?

Also Brad, take a look at your 2/7/00 entry in CHANGES.  Did you intend for
all of that to go into the file?  It looks like an accidental text paste.

Finally, should I make a tarball out of the current module?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 21 15:11:10 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] getting started with Loci
Message-ID: <38D7D75E.E3EB9A20@geoserve.net>

This is in Loci's README file:

--------------------------------------------------------

GETTING STARTED WITH LOCI

There are four basic objects you will be working with in Loci:

    (1) Workspace
    (2) Locus
    (3) Connector
    (4) Windowlet

When you first start Loci with the Gnome interface (default start), you will
see an empty 'workspace' (for now).

To place a 'locus' on the workspace, press the right mouse button, and you
will get a pop-up menu:

                    +------------+
                    | Locus    > |
                    | Add      > |
                    | Session  > |
                    | Help     > |
                    +------------+

Choosing the sub-menu Add, you will find a list of loci that can be placed on
the workspace.  Choose a locus.

                        +------+
                        |      |
                   O----|      |---->O
                        |      |
                        +------+
                       [Locus One]

Each locus (represented by an icon) will appear on the workspace with lines
radiating from a central icon.  These lines are 'connectors' and will be used
to connect one locus to another.  The lines with an arrow pointing away from
the icon are 'output connectors'.  Lines without an arrow are 'input
connectors'.

            +------+                +------+
            |      |                |      |
       O----|      |---->O     O----|      |---->O
            |      |                |      |
            +------+                +------+
           [Locus One]             [Locus Two]

You will also notice that there is a red dot at the end of the connectors.
The colors indicate the connection status: red for unconnected, yellow for
connection in progress, and green for connected.  It's simple.

Connecting loci is simple too.  Just drag-and-drop the dot from one connector
onto the dot of another connector.  (Note, you can only connect connectors
that have been unconnected, plus output to input and visa versa.)

            +------+                +------+
            |      |                |      |
       O----|      |------->O-------|      |---->O
            |      |                |      |
            +------+                +------+
           [Locus One]             [Locus Two]

You can examine the GUI for each locus by double-clicking on the center icon.
This will lower a 'windowlet', which is akin to an X window but always stays
on the workspace and attached to its locus.

                        +------+
                        |      |
                   O----|      |---->O
                        |      |
                        +------+
                       [Locus One]
                +----------------------+
                |                      |
                |                      |
                |                      |
                |                      |
                |                      |
                |                      |
                |                      |
                +----------------------+

The windowlet will contain all sorts of goodies related to the function of
the locus.  Double-clicking on the center icon again will raise the
windowlet.

--------------------------------------------------------

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 21 15:35:57 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] G-CHIP (Graphical, Command-Line, Hyperlinked Interface Program)
Message-ID: <38D7DD2D.573B01FA@geoserve.net>

There are some very interesting ideas in this 'paper':

  http://www.geocities.com/SiliconValley/Bay/5371/gchip.html

Jeff


From bizzaro at geoserve.net  Tue Mar 21 16:49:37 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] graphical console
Message-ID: <38D7EE71.A69CFF77@geoserve.net>

I like this mockup of a 'graphical console' from the G-CHIP page (attached). 
A Loci/VSH widget like this could be connected as a viewer/visual for the 'ls'
command.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: g-chip.gif
Type: image/gif
Size: 1652 bytes
Desc: not available
Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000321/9f735090/g-chip.gif
From chapmanb at arches.uga.edu  Tue Mar 21 18:28:04 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] CHANGES
Message-ID: <200003212330.SAA213184@archa12.cc.uga.edu>

> Brad Chapman wrote:
>> 
>> Do you have any idears for the problems I mention?
> 
> On 3/14/00 you mention some DB problems and the locus deletion 
problem.  

I kind o' went through the problems I was having in my 'Code updates' 
message from March 14th (on the list). It was right in the middle of 
all the initial Overflow/GMS craziness, so was probably overlooked 
(boo hoo, can you hear my weeping at being ignored? :) BTW, the 
problems are more "theoretical" than actual (ie. I hid them in 
the implementation for right now), so no one should really need to get 
anything working, like DBs, to offer suggestions.

> I see
> you got the locus deletion working again.  BTW, I will make that 
work from a
> popup menu; I'm not so sure shift-button1 click is obvious.

Sounds fine to me, the shift-click was just meant so I could test it 
out. Now that we have the middle and front separated I can add things 
without worrying so much about implementing them "properly" in the GUI 
and can let your magic hand figure this stuff out. As an example, I'm 
going to start adding support for disconnecting loci next and I don't 
have to worry about how the GUI will implement this eventually (yay!). 
Hopefully things will work a little quicker this way as we can work in 
tandem on things. 

> As for databases, can you provide small samples (I think I mentioned 
this
> before) and default values to go in the database selection form?  I 
haven't
> been able to get this to work.  Samples should probably go under...

Did you have a chance to try out the gadfly stuff from my "Gadfly 
info" messaging on Feb 9th? What problems did you have? This should 
give you a small db to get started.

> And of course, it would be nice to eventually be able to DnD 
databases onto
> the workspace rather than fill out a form.

Yuppers. Once we can save loci (hopefully soon!) this should be less 
of a problem.

> About the doc directory, is this something that cannot go in the 
loci-doc
> module?  Do you think documentation should come with the main 
distribution?

I was just treating this as more of "developer's docs" rather than 
formal documentation. I just thought this might be a good place to 
stash the streaming XML communication protocol (which I change a lot 
right now) and maybe API docs, which will also change a lot. For 
"real" distributions (ie. distributions with numbers :-) we can just 
remove this.

> Also Brad, take a look at your 2/7/00 entry in CHANGES.  Did you 
intend for
> all of that to go into the file?  It looks like an accidental text 
paste.

No really, that big hunk of 4DOM code is important to the changes. 
Really :) Ooopsies, I'll take that out.
 
> Finally, should I make a tarball out of the current module?

Shore, that would be spectacular. Then Jarl, Jean-Marc and Dominic can 
have a chance to look at our current stuff. I think the last snapshot 
is still the one with the really slow communication between front and 
middle (since they were sharing one process). Plus we now have the 
formal front and middle separation, which gives a better idea of the 
direction stuff is heading...

Brad




From jarl at casema.net  Wed Mar 22 04:01:47 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] vsh web page
References: <38D7AD3D.4CFAA892@geoserve.net>
Message-ID: <38D88BFB.9C2EE7B@casema.net>

"J.W. Bizzaro" wrote:

> I made a simple Web page for 'VSH' (if that is to be the name):
>
>     http://bioinformatics.org/vsh/

Nice.
Though I'm not enthousiastic about the name VSH, lets use it for
now :)



From bizzaro at geoserve.net  Wed Mar 22 11:47:31 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] loci snapshot; 20000321
Message-ID: <38D8F923.11788234@geoserve.net>

A new snapshot of Loci is available on the Web:

  http://bioinformatics.org/loci/download/snapshots/loci-20000321.tar.gz

The GUI and core have been completely separated, and there are some
substantial improvements in performance.  Thanks to Brad.  It's worth the
download.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 22 17:48:16 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] gui protocol revisited
Message-ID: <38D94DB0.3353F6C3@geoserve.net>

I think the GUI-to-core *API* that Brad and I have been developing for Loci is
very nice.  It's a simple dialog of request, approval, and pointing to data. 
But what I am questioning is the *protocol* used, which is a standard TCP/IP
socket.

Commands from Loci's XML-based API are streamed across a socket.  Here is a
short example where the GUI (front) requests the addition of a new workspace
(subnet), and the core (middle) performs the addition and approves a change in
the GUI to reflect what the core has done:

<front>
  <add_workspace>
    <workspace id='' >
    </workspace>
  </add_workspace>
</front>
<middle>
  <success>
    <workspace id='workspace1' >
    </workspace>
  </success>
</middle>

So, once again, this API is XML-based and ran through a socket.  My question
is: Should we use CORBA instead?  What are the advantages/disadvantages?  My
MAJOR interest is that we have multiple UIs that can be attached to (and
controlled by) a single core, can run concurrently in the same session, and
can be controlled by one-another (via the front-request, middle-approve API). 
Can we do that with CORBA?

Thoughts?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Thu Mar 23 15:37:42 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net> <38D282C5.656BDCD9@geoserve.net> <38D28F97.D174716B@hermes.usherb.ca>
Message-ID: <38DA8096.D4BE18D7@casema.net>

Hi Jean,

There's a lot o'discussion going on in the vsh mailinglist,

and you're not commenting on them so I'm asking if you could

give us your opinion is on a few postings.

thnx,

jarl





From chapmanb at arches.uga.edu  Fri Mar 24 09:31:16 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] xml database stuff
Message-ID: <200003241434.JAA83976@archa12.cc.uga.edu>

Jeff wrote:
> Also Brad, recall how Loci was going to use an XML database (XDBM) 
to store
> the workflow diagram (network and subnetworks).  

    Yeah, we need to think about a more secure way to store the 
temporary (and permanent) XML then in directories. I've grown pretty 
cold on the XDBM python bindings because they are proposing 
implementing a full DOM interface in C on top of the current XDBM 
before wrapping it. This sounds to me like a ton of work that I don't 
have time for right now :-< 
    I have been thinking of an intermediate solution, which is 
basically the "write a Loci specific database" idea which I hated 
before. 
    I've been looking at the shelve module and python and thought of a 
way to do this hopefully rather quickly:

1. Convert the current storage ids (ie. 
'workspace1/workspace2/viewer1.xml') 
into unique ids and store this mapping in a dictionary (random_id 
-> current_id).

2. Shelve the xml according to their random ids.

3. Provide lots o' helper functions to make this work smoothly with 
the current set up.

What do you all think? Good idea? Bad idea?

Brad



From chapmanb at arches.uga.edu  Fri Mar 24 09:45:35 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] Front to middle communication
Message-ID: <200003241448.JAA156374@archa10.cc.uga.edu>

Jeff wrote:
> I was just thinking that if we're using CORBA at all anywhere, why 
not use it
> everywhere?

Wow! Did you really write this? CORBA everywhere, yay! :-)

    I think if we want to consider CORBA communication between the 
front GUI and the middle the most important thing is to consider how 
easy it will be for other people to implement front ends. Will CORBA 
be more difficult than sockets? 
    We can define a small CORBA interface that will mirror what we 
have now with sockets, and this could provide us some additional 
flexibility in terms of passing data (and might help deal with the 
problem I mentioned earlier about how to pass large amount of data 
from databases or files to the front). 
    I don't really want to make this change right yet unless we have a 
big problem we can't overcome with since sockets are working and we'll 
have millions of other changes to make to fit in with the 'vsh' 
project collaboration.
    What do you all think?

Brad
    



From jarl at casema.net  Fri Mar 24 09:59:16 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] xml database stuff
References: <200003241434.JAA83976@archa12.cc.uga.edu>
Message-ID: <38DB82C4.B124EB13@casema.net>

> before wrapping it. This sounds to me like a ton of work that I don't
> have time for right now :-<

No need to work fast, stick to working good :)

> 1. Convert the current storage ids (ie.
> 'workspace1/workspace2/viewer1.xml')
> into unique ids and store this mapping in a dictionary (random_id
> -> current_id).
> 2. Shelve the xml according to their random ids.
> 3. Provide lots o' helper functions to make this work smoothly with
> the current set up.
> What do you all think? Good idea? Bad idea?
>

I dont get what you are trying the archive here..
If it is 'storing the XML representation', why not pipe the file through
a PGP
library?




From chapmanb at arches.uga.edu  Fri Mar 24 10:04:35 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] Moving forward in the middle
Message-ID: <200003241507.KAA102964@archa11.cc.uga.edu>

Hey Jeff (and anyone else who is interested);
    I just wanted to let you know my coding plans for this upcoming 
weekend because I'd like to try and coordinate front development along 
with what I'm doing so we can get it implemented and test it out. 
    As I mentioned, I've pulled myself out of the GUI comletely so 
that I won't be interfering there any more :-) As a result this has 
led me to start thinking about how to implement stuff xml-wise and not 
gui-wise, so I could use your input.
    Specifically what I'm blathering about is that I've just about 
(keeping my fingers crossed here!) got a generalized loci 
disconnection communication protocol finished (and partly implemented 
in the GUI, but only during deletion of loci, and I have not thought 
about providing a way for generalized disconnecting of loci). This 
weekend I'm hoping to tackle persistance/saving of loci in the 
back/private directory and, if I am really lucky, start thinking about 
how to try to whip up a connection diagram into a good XML 
representation (of course, we have to work the basis of this out with 
the other vsh guys). Do you have ideas about how these things should 
work in the GUI? I'll need the GUI to test things out and make sure 
I'm implementing them properly, and just thought we could try and 
coordinate stuff and get this all working beautifully as soon as we 
can.
    What do you think? What are your plans?

Brad



From chapmanb at arches.uga.edu  Fri Mar 24 10:29:45 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] xml database stuff
Message-ID: <200003241532.KAA99760@archa12.cc.uga.edu>

>> 1. Convert the current storage ids (ie.
>> 'workspace1/workspace2/viewer1.xml')
>> into unique ids and store this mapping in a dictionary (random_id
>> -> current_id).
>> 2. Shelve the xml according to their random ids.
>> 3. Provide lots o' helper functions to make this work smoothly with
>> the current set up.
>> What do you all think? Good idea? Bad idea?
>>
Jarl wrote: 
> I dont get what you are trying the archive here..

I was just suggesting a way to store the xml representation, as you 
thought. I should have been more clear in this regard. Sorry. 
    I think we have two options:
    
    1. Shelve the DOM (will this work, can it be pickled? I haven't 
thought
        about this? Hmm...)
    2. Shelve the XML string. This will probably require extra steps 
of DOM to
        string conversion, which is bad.

> If it is 'storing the XML representation', why not pipe the file 
through
> a PGP
> library?

Can you elaborate? (sorry, I'm slow :-) What is a PGP library? Or just 
point me to a place to look up more info...

Brad





From bizzaro at geoserve.net  Fri Mar 24 16:53:43 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] Front to middle communication
References: <200003241448.JAA156374@archa10.cc.uga.edu>
Message-ID: <38DBE3E7.6DE03B9C@geoserve.net>

Brad Chapman wrote:
> 
> Wow! Did you really write this? CORBA everywhere, yay! :-)

:-)  Well, with Loci, we didn't have CORBA anywhere.  So, keeping it all
CONSISTENT with sockets was a worthy cause.  My preference is for consistency
(and simplicity) rather than for sockets.

>     I think if we want to consider CORBA communication between the
> front GUI and the middle the most important thing is to consider how
> easy it will be for other people to implement front ends. Will CORBA
> be more difficult than sockets?

Ah, an ancient riddle, my young Locian.  Perhaps the spirit of the Loci will
show you the way ;-)

>     We can define a small CORBA interface that will mirror what we
> have now with sockets, and this could provide us some additional
> flexibility in terms of passing data (and might help deal with the
> problem I mentioned earlier about how to pass large amount of data
> from databases or files to the front).

I think it's something to work for, providing we can still have multiple
fronts control the middle and each other.

>     I don't really want to make this change right yet unless we have a
> big problem we can't overcome with since sockets are working and we'll
> have millions of other changes to make to fit in with the 'vsh'
> project collaboration.
>     What do you all think?

Why don't we put it on the back burner for now.  You're correct that it's
something we can do later.  The merger is more important right now.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 17:11:15 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:25 2006
Subject: [Pipet Devel] Moving forward in the middle
References: <200003241507.KAA102964@archa11.cc.uga.edu>
Message-ID: <38DBE803.E66D87E0@geoserve.net>

Brad Chapman wrote:
> 
>     As I mentioned, I've pulled myself out of the GUI comletely so
> that I won't be interfering there any more :-) As a result this has
> led me to start thinking about how to implement stuff xml-wise and not
> gui-wise

Great.  Oh, the wonders of abstraction!

>     Specifically what I'm blathering about is that I've just about
> (keeping my fingers crossed here!) got a generalized loci
> disconnection communication protocol finished (and partly implemented
> in the GUI, but only during deletion of loci, and I have not thought
> about providing a way for generalized disconnecting of loci). This
> weekend I'm hoping to tackle persistance/saving of loci in the
> back/private directory and, if I am really lucky, start thinking about
> how to try to whip up a connection diagram into a good XML
> representation (of course, we have to work the basis of this out with
> the other vsh guys).

Right.  How does Overflow 'store' its network?

> Do you have ideas about how these things should
> work in the GUI? I'll need the GUI to test things out and make sure
> I'm implementing them properly, and just thought we could try and
> coordinate stuff and get this all working beautifully as soon as we
> can.
>     What do you think? What are your plans?

Sure, I'll be here :-)

As for the way the GUI will handle locus/node disconnection, it will be done
through a popup menu on a button3 click.  The popup menu will actually be
different for each of the 4 main GUI objects:

  (1) Workspace
  (2) Locus/node
  (3) Connector
  (4) Windowlet

Clicking button3 on a connector will bring up a menu for that connector:

    +---------------+
    | Connector  >  +---------------+
    | Session    >  | Details...    |
    | Help       >  | Add         > |
    +---------------| Disconnect    |
                    +---------------+

(Note that it doesn't work this way now, but this is my plan.)

And choosing 'Disconnect' will....disconnect the connector.  The dot color
will turn from green to yellow, during the wait for approval, and then to red,
showing that it is disconnected.

What else do you need to know?  Do you want me to get this working this
weekend?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 17:42:48 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Zip Up: New Linux Distribution Speaks to Users
Message-ID: <38DBEF68.3CC4468F@geoserve.net>

More along the line of 'speech control of software' in Linux:

http://www.linuxmall.com/news/features/000322zipspeak

Jeff


From bizzaro at geoserve.net  Fri Mar 24 22:37:20 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
	 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
Message-ID: <38DC3470.D68AF122@geoserve.net>

Jean-Marc Valin wrote:
> 
> OK, I'll try to summerize the way XML is used in Overflow with an example:
> 
> <?xml version="1.0"?>
> <Document>
> 
> A document is a file and can contain many networks (it's the equivalent of a .c)

Then this is something like Loci's 'workspace' or 'workflow diagram', or
Jarl's 'dataflow structure'.

>   <Network type="subnet" name="MAIN">
> 
> A network if the C equivalent of a function. Each document must contain a MAIN
> network.
> 
>     <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
>       <Parameter name="VALUE" type="subnet_param" value="IN"/>
>     </Node>
>
> This is a node definition. The name is arbitrary,

The name is akin to Loci's identification number, which is really a URI to the
XML representation of the node.

> while the type corresponds to an Overflow node class.

Great, this is just like Loci's locus types/classes.

> x and y are for the GUI.

I think for VSH we should have the GUI store that information.  If we are to
have more than one interface available for VSH, x and y will not always be
relevant.  This is what we were going to do with Loci.

>  The parameters have to correspond to what the node expects.

This is like a mime-type for link I/O?  This is also something Loci was going
to define.

>     <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
>       <Parameter name="FRAME_SIZE" type="int" value="256"/>
>     </Node>
>     <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
>       <Parameter name="COMMAND" type="string" value="cat"/>
>     </Node>
>     <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
>       <Parameter name="VALUE" type="string" value="frames"/>
>     </Node>

I'm not sure what you mean by value=256, cat, or frames.

>     <Node name="node8" type="OutputStream" x="43.000000" y="190.000000"/>
>     <Node name="node9" type="Save" x="243.000000" y="119.000000"/>
>     <Link from="node2" output="OUTPUT" to="node5" input="INPUT"/>
>     <Link from="node1" output="VALUE" to="node2" input="INPUT"/>
>     <Link from="node6" output="VALUE" to="node8" input="INPUT"/>
>     <Link from="node5" output="TRAIN_IN" to="node9" input="OBJECT"/>
>     <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>
> 
> Links define which node is connected to which. Input and output names need to be
> defined since nodes can have more than one input/output.

Excellent, just like Loci.  Although Loci embedded <link> (what Loci calls
<input> and <output> 'connectors') in with the node information.  This is how
Loci defines a node (loci == node, although 'locus' is singular):

  <loci type='document' >
    <widget>file_select</widget>
    <input xml:link='simple'  href='baselocus.xml' >
    </input>
    <output xml:link='simple'  href='workspace1/viewer1.xml' >
    </output>
  </loci>

(Brad, shouldn't we also have locus 'id=' in here?)

In this example, when we say 'type=', we mean the same thing you do with
'type='.  But our type 'document' is a node that stores data: a file.  It's
not where Loci sets up a network.  What Overflow calls 'document', Loci calls
'workspace'.

<widget> is the name of the interface component to be used.  We're working on
using more XML here, and the 'widget' representation in XML can be located
anywhere on the Internet.

<input> and <output> define the links (connectors) for this particular node. 
href points to the XML representation of the node it is connected to. 
xml:link is from an XML document linking tool that Brad has been using.

Loci also gives the node an ID (not shown above) that is the location/URI of
the node's XML representation.  Here the GUI of Loci requests that a node
called 'viewer1' gets linked to a node called 'document1':

  <connect>
    <input id='workspace1/viewer1.xml' >
    </input>
    <output id='workspace1/document1.xml' >
    </output>
  </connect>

Other information related to a node (e.g., library URI and version number) can
be kept between <loci> and </loci>

>     <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>
> 
> NetOutput means which node (and which output of that node) will be used as the
> output of the whole network. The same applies to NetInput and NetCondition.

What is the actual output?  The screen, a file?  Or is this something the user
defines?

>   </Network>
>   <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>

In this case, the whole 'document' gets a 'parameter' tag?

When I first came across Overflow (when you posted the announcement on
gnotices), I noted to Brad how remarkably similar some of its features are to
Loci's.  This is particularly true for the XML representation of a network.

This is something GMS (which uses no XML) needs to implement as well.  And
since Overflow and Loci are so similar, we should choose one definition.  Can
you comment on what you like or dislike about Loci's use of XML for network
representation?  Let's hammer this out right away.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Fri Mar 24 23:20:57 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
 <38DC3470.D68AF122@geoserve.net>
Message-ID: <38DC3EA9.F84B3EDF@hermes.usherb.ca>

"J.W. Bizzaro" a ?crit :

> > OK, I'll try to summerize the way XML is used in Overflow with an example:
> >
> > <?xml version="1.0"?>
> > <Document>
> >
> > A document is a file and can contain many networks (it's the equivalent of a .c)
> 
> Then this is something like Loci's 'workspace' or 'workflow diagram', or
> Jarl's 'dataflow structure'.

Something I forgot to say about documents... if this document is in the
VFLOW_PATH, then the whole document can be used as a node in another document.

> 
> >   <Network type="subnet" name="MAIN">
> >
> > A network if the C equivalent of a function. Each document must contain a MAIN
> > network.

BTW, the network type can be three things: "subnet", "iterator" or
"threadedIterator". The "threadedIterator" is very experimental, so I will not
talk about it. A "subnet" is the C equivalent of a function, while the
"iterator" is the equivalent (not exact) of a loop. Actually an iterator is the
equivalent of a function that looks like

void foo(...)
{
   while(condition)
   {
      do something
   }
}

> >
> >     <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
> >       <Parameter name="VALUE" type="subnet_param" value="IN"/>
> >     </Node>
> >
> > This is a node definition. The name is arbitrary,
> 
> The name is akin to Loci's identification number, which is really a URI to the
> XML representation of the node.
> 
> > while the type corresponds to an Overflow node class.
> 
> Great, this is just like Loci's locus types/classes.
> 

Just to make sure everyone understand, the definition is the equivalent of a
function call, not a function prototype/implementation.

> > x and y are for the GUI.
> 
> I think for VSH we should have the GUI store that information.  If we are to
> have more than one interface available for VSH, x and y will not always be
> relevant.  This is what we were going to do with Loci.
> 

I didn't like that either. It was just simpler to have everything in one file
(since this is the only info that was GUI-dependent)

> >  The parameters have to correspond to what the node expects.
> 
> This is like a mime-type for link I/O?  This is also something Loci was going
> to define.

I'm not sure what you're talking about...

> 
> >     <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
> >       <Parameter name="FRAME_SIZE" type="int" value="256"/>
> >     </Node>
> >     <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
> >       <Parameter name="COMMAND" type="string" value="cat"/>
> >     </Node>
> >     <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
> >       <Parameter name="VALUE" type="string" value="frames"/>
> >     </Node>
> 
> I'm not sure what you mean by value=256, cat, or frames.
> 

Here's the C equivalent:

void foo(int aNumber);

main()
{
   foo(256);
}

This would translate to:

<Node name="node1" type="foo">
   <Parameter name="aNumber" type="int" value="256"/>
</Node>

Also about parameters... Here is a macro that appears on top of each file that
implements a node:

NODE_INFO(FIR,"Signal:DSP", "INPUT:FILTER", "OUTPUT",
"LENGTH:CONTINUOUS:NONCAUSAL")

Here, FIR is the name of the class (the node's "type"), "Signal:DSP" is just a
classification for the menus, "INPUT" and "FILTER" are the names of the inputs
and "OUTPUT" is the name of the output. This node accepts 3 parameters
(arguments): LENGTH, CONTINUOUS and NONCAUSAL.

When the node is loaded (its library is dynamically linked), a function puts
that info in a node "database", which the GUI uses to know what node are
available (and takes what arguments).

> >     <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>
> >
> > Links define which node is connected to which. Input and output names need to be
> > defined since nodes can have more than one input/output.
> 
> Excellent, just like Loci.  Although Loci embedded <link> (what Loci calls
> <input> and <output> 'connectors') in with the node information.  This is how
> Loci defines a node (loci == node, although 'locus' is singular):
> 
>   <loci type='document' >
>     <widget>file_select</widget>
>     <input xml:link='simple'  href='baselocus.xml' >
>     </input>
>     <output xml:link='simple'  href='workspace1/viewer1.xml' >
>     </output>
>   </loci>

We used to have links as a property of the nodes (before we switched to XML).
The reason we changed is that is that a Link in the GUI is an actual object, so
it was just simpler.

> (Brad, shouldn't we also have locus 'id=' in here?)
> 
> In this example, when we say 'type=', we mean the same thing you do with
> 'type='.  But our type 'document' is a node that stores data: a file.  It's
> not where Loci sets up a network.  What Overflow calls 'document', Loci calls
> 'workspace'.
> 
> <widget> is the name of the interface component to be used.  We're working on
> using more XML here, and the 'widget' representation in XML can be located
> anywhere on the Internet.
> 
> <input> and <output> define the links (connectors) for this particular node.
> href points to the XML representation of the node it is connected to.
> xml:link is from an XML document linking tool that Brad has been using.
> 
> Loci also gives the node an ID (not shown above) that is the location/URI of
> the node's XML representation.  Here the GUI of Loci requests that a node
> called 'viewer1' gets linked to a node called 'document1':
> 
>   <connect>
>     <input id='workspace1/viewer1.xml' >
>     </input>
>     <output id='workspace1/document1.xml' >
>     </output>
>   </connect>
> 
> Other information related to a node (e.g., library URI and version number) can
> be kept between <loci> and </loci>
> 
> >     <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>
> >
> > NetOutput means which node (and which output of that node) will be used as the
> > output of the whole network. The same applies to NetInput and NetCondition.
> 
> What is the actual output?  The screen, a file?  Or is this something the user
> defines?

The output is what getOutput() returns... you can do whatever you like with it.
batchflow (text mode tool) writes it to stdout, vflow (the GUI tool) writes it
in a subwindow. If you want to save it in a file, all you need to do is to have
the last node be a "Save" node. ...or the last node can be a soundcard output...
In most cases, the output of the MAIN network is a nil object.

> 
> >   </Network>
> >   <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>
> 
> In this case, the whole 'document' gets a 'parameter' tag?
> 
> When I first came across Overflow (when you posted the announcement on
> gnotices), I noted to Brad how remarkably similar some of its features are to
> Loci's.  This is particularly true for the XML representation of a network.
> 
> This is something GMS (which uses no XML) needs to implement as well.  And
> since Overflow and Loci are so similar, we should choose one definition.  Can
> you comment on what you like or dislike about Loci's use of XML for network
> representation?  Let's hammer this out right away.

I'm not that much a fan of XML... The only reason I used it was that it saved me
so much time (instead of writing a parser myself). So Anything that does the job
is OK for me.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca



From bizzaro at geoserve.net  Fri Mar 24 23:50:14 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
	 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
	 <38DC3470.D68AF122@geoserve.net> <38DC3EA9.F84B3EDF@hermes.usherb.ca>
Message-ID: <38DC4586.178AE513@geoserve.net>

Jean-Marc Valin wrote:
> 
> Something I forgot to say about documents... if this document is in the
> VFLOW_PATH, then the whole document can be used as a node in another document.

Right.  Loci can embed workspaces in workspaces too.

> BTW, the network type can be three things: "subnet", "iterator" or
> "threadedIterator". The "threadedIterator" is very experimental, so I will not
> talk about it. A "subnet" is the C equivalent of a function, while the
> "iterator" is the equivalent (not exact) of a loop. Actually an iterator is the
> equivalent of a function that looks like
> 
> void foo(...)
> {
>    while(condition)
>    {
>       do something
>    }
> }

Understood.

> > Great, this is just like Loci's locus types/classes.
> >
> 
> Just to make sure everyone understand, the definition is the equivalent of a
> function call, not a function prototype/implementation.

Okay, I was showing you some actual locus implementation in XML.  Loci doesn't
keep workspace information in a single flat file like Overflow does.  Loci
(right now) uses the local filesystem to help organize the structure.  I'll
let you, Jarl and Brad work out how VSH will do it.

> > This is like a mime-type for link I/O?  This is also something Loci was going
> > to define.
> 
> I'm not sure what you're talking about...

I mean, when Overflow defines a link, does it also define the type of data
that is passed through the link?

> > I'm not sure what you mean by value=256, cat, or frames.
> >
> 
> Here's the C equivalent:
> 
> void foo(int aNumber);
> 
> main()
> {
>    foo(256);
> }
> 
> This would translate to:
> 
> <Node name="node1" type="foo">
>    <Parameter name="aNumber" type="int" value="256"/>
> </Node>

Gotcha, these are akin to function parameters in C.

> When the node is loaded (its library is dynamically linked), a function puts
> that info in a node "database", which the GUI uses to know what node are
> available (and takes what arguments).

How are you making this 'database'?  Is it an external db like mySQL or your
own creation?

> We used to have links as a property of the nodes (before we switched to XML).
> The reason we changed is that is that a Link in the GUI is an actual object, so
> it was just simpler.

That's a good point.  Loci also treats connectors as objects.  But they are
always a subclass of loci (nodes).

> > What is the actual output?  The screen, a file?  Or is this something the user
> > defines?
> 
> The output is what getOutput() returns... you can do whatever you like with it.
> batchflow (text mode tool) writes it to stdout, vflow (the GUI tool) writes it
> in a subwindow. If you want to save it in a file, all you need to do is to have
> the last node be a "Save" node. ...or the last node can be a soundcard output...
> In most cases, the output of the MAIN network is a nil object.

Excellent.  In Loci, we were going to have unconnected output connectors
(links) be 'outputs' of this type.  If these outputs are left unconnected by
the user, Loci would send the data to the parent of the workspace (document). 
And if there was no parent (the workspace is the highest-level workspace),
Loci would send the data to stdout.

> I'm not that much a fan of XML... The only reason I used it was that it saved me
> so much time (instead of writing a parser myself). So Anything that does the job
> is OK for me.

I like it.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Sat Mar 25 00:02:06 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
	 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
Message-ID: <38DC484E.781D6528@geoserve.net>

Jean-Marc Valin wrote:
> 
> OK, I'll try to summerize the way XML is used in Overflow with an example:

Brad and Jarl, perhaps GMS can reuse much of Overflow's code for handling XML
and network construction?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Sat Mar 25 00:08:02 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
 <38DC3470.D68AF122@geoserve.net> <38DC3EA9.F84B3EDF@hermes.usherb.ca> <38DC4586.178AE513@geoserve.net>
Message-ID: <38DC49B2.CD741462@hermes.usherb.ca>

> > > Great, this is just like Loci's locus types/classes.
> > >
> >
> > Just to make sure everyone understand, the definition is the equivalent of a
> > function call, not a function prototype/implementation.
> 
> Okay, I was showing you some actual locus implementation in XML.  Loci doesn't
> keep workspace information in a single flat file like Overflow does.  Loci
> (right now) uses the local filesystem to help organize the structure.  I'll
> let you, Jarl and Brad work out how VSH will do it.
> 
> > > This is like a mime-type for link I/O?  This is also something Loci was going
> > > to define.
> >
> > I'm not sure what you're talking about...
> 
> I mean, when Overflow defines a link, does it also define the type of data
> that is passed through the link?

Well, that's not necessary since all Overflow objects derive from the Object
class. Of course, some nodes require their input to be a certain type, so they
will check the type. However some nodes, like "save", don't care about the type,
since all objects know how to save themselves. Also some nodes, like "load"
don't even know what type they will return (it depends on what's in the file
their loading). BTW, the virtual getOutput() method defined in the base Node
class returns a type ObjectRef, which is a reference-counted pointer (aka
"smart-pointer") to an Object (the base class). For those who know the language,
our object system is a lot inspired by Smalltalk.

> > When the node is loaded (its library is dynamically linked), a function puts
> > that info in a node "database", which the GUI uses to know what node are
> > available (and takes what arguments).
> 
> How are you making this 'database'?  Is it an external db like mySQL or your
> own creation?

It's not a database in that sense, it's just a structure (I use an STL map) that
maps a node name to a set of information for the GUI (inputs, outputs,
parameters, ...).

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Sat Mar 25 01:05:30 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Jungle Monkey
Message-ID: <38DC572A.D7852AB1@geoserve.net>

I think this is something like Napster:

  http://www.eecs.umich.edu/~dhelder/misc/jm/

which could be used to make distributed VSH nodes easily accessible across the
Internet.

Jeff


From jarl at casema.net  Sat Mar 25 06:04:33 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
		 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
		 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
		 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca> <38DC3470.D68AF122@geoserve.net>
Message-ID: <38DC9D41.99AED1B2@casema.net>

>
> > A document is a file and can contain many networks (it's the equivalent of a .c)
>
> Then this is something like Loci's 'workspace' or 'workflow diagram', or
> Jarl's 'dataflow structure'.

Yes.

> > This is a node definition. The name is arbitrary,
>
> The name is akin to Loci's identification number, which is really a URI to the
> XML representation of the node.

GMS uses dname : description name, or 'human readable name',
and name : an id number. Every instance of gms gets a id number too, so the
'Loci URI' is equalavent to gms's 'instance id + name', a 64bit number.

> > while the type corresponds to an Overflow node class.
>
> Great, this is just like Loci's locus types/classes.

This is something gms wont touch, class 'types' wont exsist in this layer. They will
simply be handled as a node and passed to overflow.

> > x and y are for the GUI.
>
> I think for VSH we should have the GUI store that information.  If we are to
> have more than one interface available for VSH, x and y will not always be
> relevant.  This is what we were going to do with Loci.

Seems logical. There's no need to store x\y into the processing layer. But it wont
harm the GUI too, it can just ignore those values. Or is there a special reason to
have them inside the nodes?

> When I first came across Overflow (when you posted the announcement on
> gnotices), I noted to Brad how remarkably similar some of its features are to
> Loci's.  This is particularly true for the XML representation of a network.
>
> This is something GMS (which uses no XML) needs to implement as well.  And
> since Overflow and Loci are so similar, we should choose one definition.  Can
> you comment on what you like or dislike about Loci's use of XML for network
> representation?  Let's hammer this out right away.

OK, the overflow & loci people should define this definition. GMS will only use a few

lines where the gui and overflow layer will use every single line. Make it whatever
you
see fits best.




From gvd at redpoll.pharmacy.ualberta.ca  Sat Mar 25 14:32:13 2000
From: gvd at redpoll.pharmacy.ualberta.ca (Gary Van Domselaar)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Front to middle communication
References: <200003241448.JAA156374@archa10.cc.uga.edu>
Message-ID: <38DD143D.B2EEF023@redpoll.pharmacy.ualberta.ca>

Brad Chapman wrote:
> 
> Jeff wrote:
> > I was just thinking that if we're using CORBA at all anywhere, why
> not use it
> > everywhere?
> 
> Wow! Did you really write this? CORBA everywhere, yay! :-)
> 
>     I think if we want to consider CORBA communication between the
> front GUI and the middle the most important thing is to consider how
> easy it will be for other people to implement front ends. Will CORBA
> be more difficult than sockets?

I'm inclined to think that CORBA would be a barrier for other people who
might want to implement their own front end.  If the front-end need only
an XML stream from the loci middel ('If' being the operative word here),
then I'm not sure what we would stand to gain from a GUI-Loci-middle
CORBA communication layer.

>     We can define a small CORBA interface that will mirror what we
> have now with sockets, and this could provide us some additional
> flexibility in terms of passing data (and might help deal with the
> problem I mentioned earlier about how to pass large amount of data
> from databases or files to the front).
>     I don't really want to make this change right yet unless we have a
> big problem we can't overcome with since sockets are working and we'll
> have millions of other changes to make to fit in with the 'vsh'
> project collaboration.
>     What do you all think?

I say stay the course, unless someone can put forward a good argument to
use CORBA.

g.
-- 
                                  Gary Van Domselaar
                               gary@bioinformatics.org
                          http://www.bioinformatics.org/~gary
                ----------------------------------------------------
                           bioinformatics.org: The Open Lab
                            http://www.bioinformatics.org/


From gvd at redpoll.pharmacy.ualberta.ca  Sat Mar 25 14:55:19 2000
From: gvd at redpoll.pharmacy.ualberta.ca (Gary Van Domselaar)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] xml database stuff
References: <200003241434.JAA83976@archa12.cc.uga.edu>
Message-ID: <38DD19A7.6FEFFBA9@redpoll.pharmacy.ualberta.ca>

Brad Chapman wrote:

>     I've been looking at the shelve module and python and thought of a
> way to do this hopefully rather quickly:
> 
> 1. Convert the current storage ids (ie.
> 'workspace1/workspace2/viewer1.xml')
> into unique ids and store this mapping in a dictionary (random_id
> -> current_id).
> 
> 2. Shelve the xml according to their random ids.
> 
> 3. Provide lots o' helper functions to make this work smoothly with
> the current set up.
> 
> What do you all think? Good idea? Bad idea?

I don't know much about python's shelve module, but from what you
describe it seems pretty obvious.  It sounds to me like a good idea, but
probably not as good as storing in a real DBMS (for sophisticated
querying etc.).  If you still want to use XDBM, once it has python
bindings, then I suggest you implement XML storage in a fashion that
will facilitate its storage in XDBM, and not worry too much right now
about issues of security, elegance, sophistication, etc.  That will come
when we store our XML in a professionally designed DBMS. 

g.
-- 
                                  Gary Van Domselaar
                               gary@bioinformatics.org
                          http://www.bioinformatics.org/~gary
                ----------------------------------------------------
                           bioinformatics.org: The Open Lab
                            http://www.bioinformatics.org/


From bizzaro at geoserve.net  Sat Mar 25 22:04:16 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Painting by Numbers with SVG
Message-ID: <38DD7E30.938AA44D@geoserve.net>

http://www.xml.com/pub/2000/03/15/deviant/index.html

Jeff


From bizzaro at geoserve.net  Sat Mar 25 22:06:36 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] XML compression
Message-ID: <38DD7EBC.15FA77B7@geoserve.net>

http://www.xml.com/pub/2000/03/22/deviant/index.html

Jeff


From chapmanb at arches.uga.edu  Sun Mar 26 18:37:24 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] hOpla
Message-ID: <200003262340.SAA111256@archa12.cc.uga.edu>

> 
> What do you think of this, Brad?
> 
>  http://hopla.linuxbe.org/en/

    Well, I guess it seems to be a set of C classes to map XML -> 
relational database (PostgreSQL). It is kind of hard for me to assess 
it since there isn't much documentation and the comments in the code 
are written in French (and I only speak about 3 words of French :-)
    The thing we really need to look at is how easily we can query and 
adjust XML in a database. There is a lot of DOM manipulation and SAX 
parsing in the current code to change the XML and get info from in, 
and if we moved into a relational database scheme such as this it 
would all have to be rewritten. This might be inevitable, but if we 
have to do it, I'd rather only do it once, so we should really have a 
darn good system ready to move to before we do it. 
    I've been looking more into the shelve module (the proposal I shot 
out before), but I don't really think now that it will work. 
Apparently the files it creates are really large, and the database it 
uses to store things has a relatively low storage limit (this is just 
based on reading the comp.lang.python archives, not really on any 
actual knowledge of mine). 
    I've been thinking a lot back to the WHAX 
(http://www.seas.upenn.edu/~liefke/whax.html) research that Jennifer 
clued us in to a while back, and this just seems like the ideal system 
for what we are doing. We could maintain a huge database of all the 
xml--pull out specific documents and then query/update them using 
standard python xml tools (DOM and XML parsers) and then update them. 
It seems like we could almost have a really weak (well, not true to 
their beautiful vision) version of this if my shelve idea would work, 
or if we could think of another good way to store things in an 
analagous manner. 
    Right now I'm kind of stumped on this. Does anyone else have any 
brilliant ideas?

Brad



From jarl at casema.net  Mon Mar 27 01:24:54 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] hOpla
References: <200003262340.SAA111256@archa12.cc.uga.edu>
Message-ID: <38DEFEB6.D853CC95@casema.net>


> It seems like we could almost have a really weak (well, not true to
> their beautiful vision) version of this if my shelve idea would work,
> or if we could think of another good way to store things in an
> analagous manner.
>     Right now I'm kind of stumped on this. Does anyone else have any
> brilliant ideas?

What is the problem, when you can store the XML documents by key = uriT
into a sql (mysql) database?
I've the feeling the Loci people have though of this some more, what are the
(practical) problems?



From bizzaro at geoserve.net  Mon Mar 27 23:49:34 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] overflow<->loci translation and 3 basic gui objects
Message-ID: <38E039DE.BFA7B00B@geoserve.net>

It's an understatement to say that VSH is a complex system.  But from the
user's point of view, it should be simple.  So, my motto for the development
of the VSH GUI is K.I.S.: Keep It Simple!

That said, I want define 3 basic objects for the GUI.  I started out with 4
under Loci:

  (1) Workspace
  (2) Locus (node)
  (3) Connector
  (4) Windowlet

But thinking about this more recently, I believe (1) and (4) are the same. 
Afterall, workspaces are always in windowlets (windows) and windows always
contain workspaces.

So, how about this.  The 3 basic GUI objects:

  (1) Window
  (2) Locus (node)
  (3) Connector

(Note: a 'network' is a collection of nodes and probably shouldn't be
considered a separate object by the GUI.)

This way we have wonderful nesting and recursion: Windows contain nodes, nodes
contain connectors and windows, windows contain nodes, etc., etc.

Okay, now this brings me to how objects are named in both Overflow and Loci. 
As we have seen, both applications are remarkably similar, varying (so it
seems) mostly by how things are named.  Here is a quick translation between
the two:

  Overflow             Loci
  --------             --------
  Document             Windowlet/workspace
  Network/subnet       Workflow diagram
  Node                 Locus
  Link                 Connector

We need to standardize on these names.  I can settle with using Overflow's
terminology in every case except for 'document'.  Since VSH will be dealing
with large text files often called 'documents', I think that name is
confusing.  So, I propose we (including Overflow) change this to something
else.

Since the Loci equivalent to 'document' is 'windowlet' (and 'workspace'), and
since I mention above those two can be considered the same object, I propose
'document' be changed to 'window'.

Also, I'm not sure whether I like 'link' or 'connector' better.  What do you
guys like?  We could also consider 'line' and 'branch'.

So, maybe VSH could use the following (basic and collective) object names:

  Overflow             Loci                   VSH
  --------             --------               ---------
  Document             Windowlet/workspace    Window
  Network/subnet       Workflow diagram       Network
  Node                 Locus                  Node
  Link                 Connector              Link (or connector)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 00:13:27 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net>
Message-ID: <38E03F77.6751C58F@geoserve.net>

"J.W. Bizzaro" wrote:
>
> So, how about this.  The 3 basic GUI objects:
> 
>   (1) Window
>   (2) Locus (node)
>   (3) Connector

Also, in GUI terms, the node is an icon.

You know, if we really want to go wacky, the core can define 3 basic objects:

  (1) Network
  (2) Node
  (3) Link

Networks contain nodes, nodes contain links and networks, networks contain
nodes, nodes contain links and networks, networks contain nodes, nodes contain
links and networks, networks contain nodes, nodes contain links and networks,
networks contain nodes, nodes contain links and networks, networks contain
nodes, nodes contain links and networks, networks contain nodes, hahaha!

So, in Overflow, 'document' can be replaced with 'node', no?

<?xml version="1.0"?>
<Node>
  <Network type="subnet" name="MAIN">
    <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
      <Parameter name="VALUE" type="subnet_param" value="IN"/>
    </Node>
    <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
      <Parameter name="FRAME_SIZE" type="int" value="256"/>
    </Node>
    <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
      <Parameter name="COMMAND" type="string" value="cat"/>
    </Node>
    <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
      <Parameter name="VALUE" type="string" value="frames"/>
    </Node>
    <Node name="node8" type="OutputStream" x="43.000000" y="190.000000"/>
    <Node name="node9" type="Save" x="243.000000" y="119.000000"/>
    <Link from="node2" output="OUTPUT" to="node5" input="INPUT"/>
    <Link from="node1" output="VALUE" to="node2" input="INPUT"/>
    <Link from="node6" output="VALUE" to="node8" input="INPUT"/>
    <Link from="node5" output="TRAIN_IN" to="node9" input="OBJECT"/>
    <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>
    <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>
  </Network>
  <Network type="iterator" name="MAIN_LOOP">
    <Node name="node1" type="AudioStream" x="-526.000000" y="93.000000">
      <Parameter name="LENGTH" type="subnet_param" value="FRAME_SIZE"/>
      <Parameter name="ENCODING" type="string" value="LIN16"/>
      <Parameter name="STREAM_TYPE" type="string" value="FILE"/>
    </Node>
    <Node name="node2" type="NotDone" x="-344.000000" y="45.000000"/>
    <Node name="node10" type="FrameAccumulator" x="-336.000000"
y="131.000000">
      <Parameter name="LENGTH" type="int" value="256"/>
      <Parameter name="CACHEALL" type="int" value="1"/>
    </Node>
    <Link from="node1" output="OUTPUT" to="node2" input="INPUT"/>
    <Link from="node1" output="OUTPUT" to="node10" input="INPUT"/>
    <NetInput name="INPUT" node="node1" terminal="INPUT"/>
    <NetCondition name="CONDITION" node="node2" terminal="OUTPUT"/>
    <NetOutput name="TRAIN_IN" node="node10" terminal="OUTPUT"/>
  </Network>
  <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>
</Node>

Why not?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 00:12:56 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net>
Message-ID: <38E03F58.BBBD4343@hermes.usherb.ca>

> It's an understatement to say that VSH is a complex system.  But from the
> user's point of view, it should be simple.  So, my motto for the development
> of the VSH GUI is K.I.S.: Keep It Simple!

Well, maybe we should have more than one GUI, or at least a GUI with a couple
"modes". There are many applications that require a different interface:
 -a GUI equivalent to something like "ls -l | awk '{print $2}' | sed 's/aa/bb/'
| sort -u > somewhere" (a la Loci)
 -a system to run complex distributed tasks (a la GMS)
 -a signal processing framework a la Matlab/Simulink/Lab View (a la Overflow)

I don't see how the same interface can handle all those correctly.

> 
> That said, I want define 3 basic objects for the GUI.  I started out with 4
> under Loci:
> 
>   (1) Workspace
>   (2) Locus (node)
>   (3) Connector
>   (4) Windowlet
> 
> But thinking about this more recently, I believe (1) and (4) are the same.
> Afterall, workspaces are always in windowlets (windows) and windows always
> contain workspaces.
> 
> So, how about this.  The 3 basic GUI objects:
> 
>   (1) Window
>   (2) Locus (node)
>   (3) Connector
> 
> (Note: a 'network' is a collection of nodes and probably shouldn't be
> considered a separate object by the GUI.)

Well, a network would be the equivalent of a "group" in a vector drawing app.

> 
> This way we have wonderful nesting and recursion: Windows contain nodes, nodes
> contain connectors and windows, windows contain nodes, etc., etc.
> 
> Okay, now this brings me to how objects are named in both Overflow and Loci.
> As we have seen, both applications are remarkably similar, varying (so it
> seems) mostly by how things are named.  Here is a quick translation between
> the two:
> 
>   Overflow             Loci
>   --------             --------
>   Document             Windowlet/workspace
>   Network/subnet       Workflow diagram
>   Node                 Locus
>   Link                 Connector

I prefer workspace de document (which is too generic). I prefer Node though, a
month ago I had never heard the word Loci/Locus (sorry, engligh is my second
language). I don't mind Link of Connector (link is shorter, bout that's all)...
pipe, maybe?..

As for Network... we have to be careful... I Overflow (though it's not
implemented exactly that way), you can see "Network" as a base class and
"subnet", "iterator" and "threadedIterator" (possibly more later) as subclasses
of Network.


> 
> We need to standardize on these names.  I can settle with using Overflow's
> terminology in every case except for 'document'.  Since VSH will be dealing
> with large text files often called 'documents', I think that name is
> confusing.  So, I propose we (including Overflow) change this to something
> else.
> 
> Since the Loci equivalent to 'document' is 'windowlet' (and 'workspace'), and
> since I mention above those two can be considered the same object, I propose
> 'document' be changed to 'window'.
> 
> Also, I'm not sure whether I like 'link' or 'connector' better.  What do you
> guys like?  We could also consider 'line' and 'branch'.
> 
> So, maybe VSH could use the following (basic and collective) object names:
> 
>   Overflow             Loci                   VSH
>   --------             --------               ---------
>   Document             Windowlet/workspace    Window
>   Network/subnet       Workflow diagram       Network
>   Node                 Locus                  Node
>   Link                 Connector              Link (or connector)
> 

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 00:29:32 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net>
Message-ID: <38E0433C.2B97AB08@hermes.usherb.ca>

> So, in Overflow, 'document' can be replaced with 'node', no?

No. You're confusing two things here: in C we's say "function call" vs "function
implementation". The <Node> tag refers to a function call (a node
instantiation), while the <Network> or <Document> can be viewed as a function
implementation (node definition) from other nodes. 


> 
> <?xml version="1.0"?>
> <Node>
>   <Network type="subnet" name="MAIN">
>     <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
>       <Parameter name="VALUE" type="subnet_param" value="IN"/>
>     </Node>
>     <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
>       <Parameter name="FRAME_SIZE" type="int" value="256"/>
>     </Node>
>     <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
>       <Parameter name="COMMAND" type="string" value="cat"/>
>     </Node>
>     <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
>       <Parameter name="VALUE" type="string" value="frames"/>
>     </Node>
>     <Node name="node8" type="OutputStream" x="43.000000" y="190.000000"/>
>     <Node name="node9" type="Save" x="243.000000" y="119.000000"/>
>     <Link from="node2" output="OUTPUT" to="node5" input="INPUT"/>
>     <Link from="node1" output="VALUE" to="node2" input="INPUT"/>
>     <Link from="node6" output="VALUE" to="node8" input="INPUT"/>
>     <Link from="node5" output="TRAIN_IN" to="node9" input="OBJECT"/>
>     <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>
>     <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>
>   </Network>
>   <Network type="iterator" name="MAIN_LOOP">
>     <Node name="node1" type="AudioStream" x="-526.000000" y="93.000000">
>       <Parameter name="LENGTH" type="subnet_param" value="FRAME_SIZE"/>
>       <Parameter name="ENCODING" type="string" value="LIN16"/>
>       <Parameter name="STREAM_TYPE" type="string" value="FILE"/>
>     </Node>
>     <Node name="node2" type="NotDone" x="-344.000000" y="45.000000"/>
>     <Node name="node10" type="FrameAccumulator" x="-336.000000"
> y="131.000000">
>       <Parameter name="LENGTH" type="int" value="256"/>
>       <Parameter name="CACHEALL" type="int" value="1"/>
>     </Node>
>     <Link from="node1" output="OUTPUT" to="node2" input="INPUT"/>
>     <Link from="node1" output="OUTPUT" to="node10" input="INPUT"/>
>     <NetInput name="INPUT" node="node1" terminal="INPUT"/>
>     <NetCondition name="CONDITION" node="node2" terminal="OUTPUT"/>
>     <NetOutput name="TRAIN_IN" node="node10" terminal="OUTPUT"/>
>   </Network>
>   <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>
> </Node>
> 
> Why not?
> 
> Jeff
> --
>                       +----------------------------------+
>                       |           J.W. Bizzaro           |
>                       |                                  |
>                       | http://bioinformatics.org/~jeff/ |
>                       |                                  |
>                       |        BIOINFORMATICS.ORG        |
>                       |           The Open Lab           |
>                       |                                  |
>                       |    http://bioinformatics.org/    |
>                       +----------------------------------+
> 
> _______________________________________________
> pipet-devel maillist  -  pipet-devel@bioinformatics.org
> http://bioinformatics.org/mailman/listinfo/pipet-devel

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Tue Mar 28 01:13:25 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca>
Message-ID: <38E04D85.811D236C@geoserve.net>

Jean-Marc Valin wrote:
> 
> Well, maybe we should have more than one GUI, or at least a GUI with a couple
> "modes". There are many applications that require a different interface:
>  -a GUI equivalent to something like "ls -l | awk '{print $2}' | sed 's/aa/bb/'
> | sort -u > somewhere" (a la Loci)
>  -a system to run complex distributed tasks (a la GMS)
>  -a signal processing framework a la Matlab/Simulink/Lab View (a la Overflow)
> 
> I don't see how the same interface can handle all those correctly.

Ahhh, but this is where we use XML to define GUIs!  As I mentioned earlier,
XML documents can be arranged according to the arrangement of nodes in a
network.  Then the XML can be translated into a Gtk interface using something
like libglade.

What I haven't brought up yet is that 'windows' can switch between views.  At
one moment, a window can show a network of nodes.  The next moment, this
window can be switched to show Gtk widgets arranged as I mention above.

> I prefer workspace de document (which is too generic). I prefer Node though, a
> month ago I had never heard the word Loci/Locus (sorry, engligh is my second
> language).

Loci/locus is actually a Latin word, meaning location :-)

> I don't mind Link of Connector (link is shorter, bout that's all)...
> pipe, maybe?..

Pipe may be confused with Unix pipes, which are local.  Link is okay, and is
reminiscent of HTML links.

> As for Network... we have to be careful... I Overflow (though it's not
> implemented exactly that way), you can see "Network" as a base class and
> "subnet", "iterator" and "threadedIterator" (possibly more later) as subclasses
> of Network.

So, what are nodes a subclass of?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 01:17:05 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
Message-ID: <38E04E61.B2634CC5@geoserve.net>

Jean-Marc Valin wrote:
> 
> No. You're confusing two things here: in C we's say "function call" vs "function
> implementation". The <Node> tag refers to a function call (a node
> instantiation), while the <Network> or <Document> can be viewed as a function
> implementation (node definition) from other nodes.

You can't say the <Node> tag refers to BOTH function calls AND
implementations, to make it more recursive?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 01:18:52 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E04D85.811D236C@geoserve.net>
Message-ID: <38E04ECC.7B84B42D@hermes.usherb.ca>

> So, what are nodes a subclass of?

The Node class doesn't have any parent, though it might inherit from Object if
we need to (probably). ...and Network inherits from Node.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 01:22:11 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
 <38E04E61.B2634CC5@geoserve.net>
Message-ID: <38E04F93.C45AC9FE@hermes.usherb.ca>

> > No. You're confusing two things here: in C we's say "function call" vs "function
> > implementation". The <Node> tag refers to a function call (a node
> > instantiation), while the <Network> or <Document> can be viewed as a function
> > implementation (node definition) from other nodes.
> 
> You can't say the <Node> tag refers to BOTH function calls AND
> implementations, to make it more recursive?

If you can find a way to do that in C, you have found how to do it with
Overflow. If not, forget it :-). I don't see how, it's really two different
things... If I had to replace Document by something existing, I'd choose Network
(which, remember, derives from Node!).

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jarl at casema.net  Tue Mar 28 00:22:21 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca>
Message-ID: <38E0418D.A96D3D63@casema.net>

>
> Well, maybe we should have more than one GUI, or at least a GUI with a couple
> "modes". There are many applications that require a different interface:
>  -a GUI equivalent to something like "ls -l | awk '{print $2}' | sed 's/aa/bb/'
> | sort -u > somewhere" (a la Loci)
>  -a system to run complex distributed tasks (a la GMS)
>  -a signal processing framework a la Matlab/Simulink/Lab View (a la Overflow)
>
> I don't see how the same interface can handle all those correctly.

Have different modes like you suggest:
- normal mode: editing nodes and 'relations'
- access mode: editing the security\accessabilities (of the brokering level)
- detail mode: editing the PL (overflow) nodes

Maybe learn from how 3Dstudio works.





From bizzaro at geoserve.net  Tue Mar 28 01:35:11 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui 
 objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E04D85.811D236C@geoserve.net> <38E04ECC.7B84B42D@hermes.usherb.ca>
Message-ID: <38E0529F.4CEED298@geoserve.net>

Jean-Marc Valin wrote:
> 
> > So, what are nodes a subclass of?
> 
> The Node class doesn't have any parent, though it might inherit from Object if
> we need to (probably). ...and Network inherits from Node.

Maybe I should say 'subset' rather than 'subclass', so that I'm not confusing
OO terminology with ordinary logic.

This is what I'm trying to saying in logical terms:

  The Overflow way:

      'Documents' contain networks
      Networks are of types subnet and iterator
      Networks contain nodes and links

  But can't we say...

      Nodes contain links and networks
      Networks are of types subnet and iterator
      Networks contain nodes
      Nodes contain links and networks
      ...


Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 01:48:39 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E0418D.A96D3D63@casema.net>
Message-ID: <38E055C7.ECE117C0@geoserve.net>

jarl van katwijk wrote:
> 
> Have different modes like you suggest:
> - normal mode: editing nodes and 'relations'
> - access mode: editing the security\accessabilities (of the brokering level)
> - detail mode: editing the PL (overflow) nodes

I see 3 modes/views:

  (1) Network normal view: Nodes and relations are visible at
      full size (icons and subwindows shown)
  (2) Network world view: Nodes and relations are visible and
      fit entirely in the window (icons and subwindows not shown)
  (3) GUI view: Gtk widgets from all nodes are visible

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 28 00:54:40 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E0418D.A96D3D63@casema.net> <38E055C7.ECE117C0@geoserve.net>
Message-ID: <38E04920.78735DA3@casema.net>

>   (1) Network normal view: Nodes and relations are visible at
>       full size (icons and subwindows shown)
>   (2) Network world view: Nodes and relations are visible and
>       fit entirely in the window (icons and subwindows not shown)
>   (3) GUI view: Gtk widgets from all nodes are visible

The "GTk widget" from a node?
A node has a gtk-widget or is a gtk-widget 'created' by the gui to visualise it?



From bizzaro at geoserve.net  Tue Mar 28 01:58:45 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E0418D.A96D3D63@casema.net> <38E055C7.ECE117C0@geoserve.net>
Message-ID: <38E05825.851881C1@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> > Have different modes like you suggest:
> > - normal mode: editing nodes and 'relations'
> > - access mode: editing the security\accessabilities (of the brokering level)
> > - detail mode: editing the PL (overflow) nodes

I actually want to try and hide these 'modes' from the user.  They should be
indistinguishable from each other.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 02:01:00 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
	 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca>
Message-ID: <38E058AC.26F3FA21@geoserve.net>

Jean-Marc Valin wrote:
> 
> If you can find a way to do that in C, you have found how to do it with
> Overflow. If not, forget it :-). I don't see how, it's really two different
> things... If I had to replace Document by something existing, I'd choose Network
> (which, remember, derives from Node!).

I didn't realize Overflow had that limitation.  Perhaps then the GMS layer can
provide the recursion?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 28 01:06:55 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:26 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui 
 objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
		 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E058AC.26F3FA21@geoserve.net>
Message-ID: <38E04BFF.85220082@casema.net>

> >
> > If you can find a way to do that in C, you have found how to do it with
> > Overflow. If not, forget it :-). I don't see how, it's really two different
> > things... If I had to replace Document by something existing, I'd choose Network
> > (which, remember, derives from Node!).
>
> I didn't realize Overflow had that limitation.  Perhaps then the GMS layer can
> provide the recursion?

Uhh.. sorry, dont get it what this 'call\implementation' is about?
Somebody could explain?



From bizzaro at geoserve.net  Tue Mar 28 02:22:42 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E0418D.A96D3D63@casema.net> <38E055C7.ECE117C0@geoserve.net> <38E04920.78735DA3@casema.net>
Message-ID: <38E05DC2.BB77F60D@geoserve.net>

jarl van katwijk wrote:
> 
> >   (1) Network normal view: Nodes and relations are visible at
> >       full size (icons and subwindows shown)
> >   (2) Network world view: Nodes and relations are visible and
> >       fit entirely in the window (icons and subwindows not shown)
> >   (3) GUI view: Gtk widgets from all nodes are visible
> 
> The "GTk widget" from a node?
> A node has a gtk-widget or is a gtk-widget 'created' by the gui to visualise it?

Each node has user-settable parameters (see what pops up when you double-click
on an Overflow node).  As I mentioned before, these parameters are
'represented by XML' tags and translated into Gtk widgets so that the
parameters can be set.  This XML makes up the 'XML representation of a GUI'.

The XML can then be arranged according to the arrangement of nodes in a
network.  The result is a 'composite' GUI.

This seems to be difficult for everyone to grasp.  I'm not sure anyone on the
Loci list understood me either, except maybe Gary.  Gary, can you explain this
differently?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 02:27:16 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
			 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E058AC.26F3FA21@geoserve.net> <38E04BFF.85220082@casema.net>
Message-ID: <38E05ED4.2DA647A5@geoserve.net>

jarl van katwijk wrote:
> 
> > > If you can find a way to do that in C, you have found how to do it with
> > > Overflow. If not, forget it :-). I don't see how, it's really two different
> > > things... If I had to replace Document by something existing, I'd choose Network
> > > (which, remember, derives from Node!).
> >
> > I didn't realize Overflow had that limitation.  Perhaps then the GMS layer can
> > provide the recursion?
> 
> Uhh.. sorry, dont get it what this 'call\implementation' is about?
> Somebody could explain?

In procedural languages you define a procedure with its parameters.  This is
what Jean-Marc means by 'implementation'.  Then somewhere else in the program
you give the procedure 'call'.

But Jean-Marc, you can call a procedure from within its implementation, giving
you some recursion, right?  Can Overflow do that?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 02:36:41 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
	 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca>
Message-ID: <38E06109.7F52F78E@geoserve.net>

Jean-Marc Valin wrote:
> 
> If you can find a way to do that in C, you have found how to do it with
> Overflow. If not, forget it :-). I don't see how, it's really two different
> things... If I had to replace Document by something existing, I'd choose Network
> (which, remember, derives from Node!).

Then how about this:

<?xml version="1.0"?>
<GMSNode>
  <OverflowNetwork type="subnet" name="MAIN">
    <OverflowNode name="node1" type="Constant" x="-294.000000" y="101.000000">
      <OverflowParameter name="VALUE" type="subnet_param" value="IN"/>
    </OverflowNode>
    ...
    <OverflowLink from="node2" output="OUTPUT" to="node5" input="INPUT"/>
    ...
  </OverflowNetwork>
  <OverflowParameter name="IN" type="string" value="/data/ntt/french/*/*"/>
</GMSNode>

Or what would these be called in the newspeak?  BLNode, PLNode?

There will be SOME differences between the 2 types of networks, so we probably
shouldn't use the same words/tags for both.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 28 02:01:01 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
				 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E058AC.26F3FA21@geoserve.net> <38E04BFF.85220082@casema.net> <38E05ED4.2DA647A5@geoserve.net>
Message-ID: <38E058AD.C1F874FF@casema.net>

"J.W. Bizzaro" wrote:

> jarl van katwijk wrote:
> >
> > > > If you can find a way to do that in C, you have found how to do it with
> > > > Overflow. If not, forget it :-). I don't see how, it's really two different
> > > > things... If I had to replace Document by something existing, I'd choose Network
> > > > (which, remember, derives from Node!).
> > >
> > > I didn't realize Overflow had that limitation.  Perhaps then the GMS layer can
> > > provide the recursion?
> >
> > Uhh.. sorry, dont get it what this 'call\implementation' is about?
> > Somebody could explain?
>
> In procedural languages you define a procedure with its parameters.  This is
> what Jean-Marc means by 'implementation'.  Then somewhere else in the program
> you give the procedure 'call'.
>

OK, I know that, but why it this an issue?




From jarl at casema.net  Tue Mar 28 02:08:28 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E0418D.A96D3D63@casema.net> <38E055C7.ECE117C0@geoserve.net> <38E04920.78735DA3@casema.net> <38E05DC2.BB77F60D@geoserve.net>
Message-ID: <38E05A6C.AF6C1EA@casema.net>

>
> This seems to be difficult for everyone to grasp.  I'm not sure anyone on the
> Loci list understood me either, except maybe Gary.  Gary, can you explain this
> differently?

I understand. I needed this explanation because nodes in GMS have a GtkPlug,
hardcoded Gtk-code that's 'plugged' into the 'sockets' the GUI provides.
Loci uses the 'other' methode.

bye,
jarl



From chapmanb at arches.uga.edu  Tue Mar 28 07:27:05 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Proposal for implementing permanent storage
Message-ID: <200003281229.HAA43112@archa12.cc.uga.edu>

Hello all;
    I've been working in the middle on adding capabilities for 
"permanent storage" of loci, so that they can be saved in one session 
and recalled in a different session. I think this will be really 
important to have working if we want to have good functionality with 
the new merger with GMS and Overflow. So, I've come up with a proposal 
for implementing permanent storage in the GUI and would like to toss 
it out for people's comments before I get going with it (since I'm not 
so hot at user interfaces). I think this is pretty much based on ideas 
that Jeff posted previously. So here goes.

1. The permanent storage locus will be a "special case" of the 
container locus that we currently have. That is, the locus will look 
like the current container locus with a list/tree view of the contents.

1.1 For anyone who looks at the code, I'm planning on implementing 
this as a new class since I'm not really very happy with how I did 
widgets/container.py. Then I'll revamp these classes so one either 
inherets from another or (more likely) so they both inheret from a 
base class.

1.2 Each element in a list will be represented by the name of the 
locus and a little pixmap according to it's type. Anyone know where to 
get a bunch o' little pixmaps I could use for this?

1.3 The permanent storage locus will have to be special in that it is 
either always present on the desktop, or can be called up through a 
menu function.

2. The permanent storage locus will be implemented like a "separate 
hard drive" from the rest of the workspace. So anything which moves 
into permanent storage will be copied in, and anything that moves out 
will be copied out.

3. loci will be moved out of permanent storage onto a workspace (so 
they can be used or manipulated) via drag and drop (like things work 
now for normal containers).

4. loci will be moved into containers via drag and drop from a locus 
in the workspace. Right now I'm planning to implement a dnd from a 
locus on the workspace using a button 2 click (the middle button for 3 
button mouse users and buttons 1 + 2 for 2 button users). Then we can 
explore further how to integrate dnd movement with the "regular local 
loci movement" and get how we'd work that straightened out.

5. I'm not positive the best way to view deeper levels of directories 
inside of the storage container. We could do this using:
    a. A GtkTree (like I had originally implemented for regular 
containers). I
_might_ be able to make this faster on a second try.
    b, Make a double click on a container in the directory open up a 
window/another loci that displays the contents on this directory.
    c. Other ideas?

6. We will probably also need to implement a "trash" container to go 
along with this.

    I think that's what I've got for now. Any suggestions or comments 
would be very welcome!

Brad



From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 08:53:06 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
				 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E058AC.26F3FA21@geoserve.net> <38E04BFF.85220082@casema.net> <38E05ED4.2DA647A5@geoserve.net>
Message-ID: <38E0B942.85996C69@hermes.usherb.ca>

"J.W. Bizzaro" a ?crit :
> 
> jarl van katwijk wrote:
> >
> > > > If you can find a way to do that in C, you have found how to do it with
> > > > Overflow. If not, forget it :-). I don't see how, it's really two different
> > > > things... If I had to replace Document by something existing, I'd choose Network
> > > > (which, remember, derives from Node!).
> > >
> > > I didn't realize Overflow had that limitation.  Perhaps then the GMS layer can
> > > provide the recursion?
> >
> > Uhh.. sorry, dont get it what this 'call\implementation' is about?
> > Somebody could explain?
> 
> In procedural languages you define a procedure with its parameters.  This is
> what Jean-Marc means by 'implementation'.  Then somewhere else in the program
> you give the procedure 'call'.
> 
> But Jean-Marc, you can call a procedure from within its implementation, giving
> you some recursion, right?  Can Overflow do that?

Yes, you can do that in Overflow. There are two ways to do it, but you don't
need to have <Node> mean two different things. You can see in TimeFilter.cc that
the getOutput method calls this->getOutput() so the output is the node is fed
back in the node. I'd call this implicit recursion. The other way is not fully
implemented yet. When it is, you'll be able to explicitly connect the output of
a node to its input (or the input of a node "upstream"). The overflow model
already supports that, but for now, the network initialization will fail.

Also, about node parameters. I think there's a little misunderstanding. The
<Parameter> tag in the <Node> doesn't say which parameters are available, but
which argument is passed to the node. The parameter "declarations" are in two
places:

-For built-in nodes (.cc files), the parameters are declared as an argument to
NODE_INFO on top of the file.
-For networks (that can be used as nodes), every time there's an argument of
type "subnet_param" for a node in that network, the network will have this
parameter. example:

<Network name="foo" type="subnet">
   <Node name="bar" type="SomeNode">
      <Parameter name="param1" type="Int" value="1"/>
      <Parameter name="param2" type="subnet_param" value="myParam"/>
   </Node>
</Network>

Here, when including network "foo" as a node, you will need to provide a
parameter named "myParam" that will be substituted as "param2" in node bar.

	Jean-Marc


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 09:04:11 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
		 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E06109.7F52F78E@geoserve.net>
Message-ID: <38E0BBDB.E94607F4@hermes.usherb.ca>

> <?xml version="1.0"?>
> <GMSNode>
>   <OverflowNetwork type="subnet" name="MAIN">
>     <OverflowNode name="node1" type="Constant" x="-294.000000" y="101.000000">
>       <OverflowParameter name="VALUE" type="subnet_param" value="IN"/>
>     </OverflowNode>
>     ...
>     <OverflowLink from="node2" output="OUTPUT" to="node5" input="INPUT"/>
>     ...
>   </OverflowNetwork>
>   <OverflowParameter name="IN" type="string" value="/data/ntt/french/*/*"/>
> </GMSNode>

You can't say GMSNode, since this could also be included as an Overflow Node.
BTW, the reason for "Document" is that XML requires ONE toplevel tag and since I
can have many networks in my file, I used Document, which only refers to the
fact that everything is in a file. But, hat about using the same type of things
that Pascal programs have: Nested functions. You'd have a toplevel network,
which would contain other networks...

<Network "name....>
   <Node ...>
   </Node>
   .
   .
   <Network ....>
      <Network...
      </Network>
   ..
   </Network>
</Network>

	JEan-Marc


From deanne at density.biop.umich.edu  Tue Mar 28 10:30:37 2000
From: deanne at density.biop.umich.edu (deanne@density.biop.umich.edu)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Greetings.
In-Reply-To: <200003281229.HAA43112@archa12.cc.uga.edu>
Message-ID: <Pine.LNX.4.05.10003281024390.11710-100000@density.biop.umich.edu>

Thought I'd just shoot a greeting out here.  I'm a graduate student
interested in becoming involved with the Loci project. I'm getting up to
speed now by reading the mailing list.  I've installed all the necessary
packages, so I'll lurk around and try to make heads or tails of what's
going on.

Deanne Taylor		        	     Biophysics Graduate Program
deanne@density.biop.umich.edu			University of Michigan
"Easy-open instructions inside box."			Ann Arbor 	



From stein at fmnh.org  Tue Mar 28 10:57:28 2000
From: stein at fmnh.org (Jennifer Steinbachs)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] BOND. The Distributed Object Multi-Agent System.
In-Reply-To: <Pine.LNX.4.05.10003281024390.11710-100000@density.biop.umich.edu>
Message-ID: <Pine.LNX.4.21.0003280955490.12644-100000@mail.fmnh.org>

Check out http://bond.cs.purdue.edu

A group of researchers at purdue university are constructing a virtual
laboratory for structural biology research.  Sounds very promising.

-jennifer
 
----------------------------------- 
J. Steinbachs, Ph.D.
Computational Biologist
Dept of Botany
The Field Museum
Chicago, IL 60605-2496

office: 312-665-7810
fax: 312-665-7158
http://cb.fmnh.org
-----------------------------------



From bizzaro at geoserve.net  Tue Mar 28 16:04:04 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Greetings.
References: <Pine.LNX.4.05.10003281024390.11710-100000@density.biop.umich.edu>
Message-ID: <38E11E44.58205120@geoserve.net>

Hi Deanne!

Welcome aboard.  If you are interested in the collaborative that Loci is now
part of, you may want to subscribe to the VSH list:

    http://bioinformatics.org/mailman/listinfo/pipet-devel/

The same goes for others on the pipet-devel list.  This (pipet-devel) list
may soon be split up into 2 lists: 1 for dicussions about the VSH GUI (which
is what Loci will become) and 1 for discussions about porting bioinformatics
applications to VSH.

Cheers.
Jeff

deanne@density.biop.umich.edu wrote:
> 
> Thought I'd just shoot a greeting out here.  I'm a graduate student
> interested in becoming involved with the Loci project. I'm getting up to
> speed now by reading the mailing list.  I've installed all the necessary
> packages, so I'll lurk around and try to make heads or tails of what's
> going on.


-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 16:43:37 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <200003282049.NAA32328@martin.pharmacy.ualberta.ca>
Message-ID: <38E12789.EC9F05F5@geoserve.net>

Gary Van Domselaar wrote:
> 
> The second concept concerns the connections/links made from one application/node/icon to
> another.
...
> The third concept concerns nesting of 'networks'.
...

Thanks Gary.  Basically, XML representations of the GUI are concatonated
(placed end-to-end) when the nodes are related by simple link:

    <NodeGUI 1>
    </NodeGUI 1>
    <NodeGUI 2>
    </NodeGUI 2>

But when the nodes are related by nesting (a node contains a network of
nodes), XML representations are nested:

    <NodeGUI 1>
        <NodeGUI 2>
        </NodeGUI 2>
    </NodeGUI 1>

And both formats can be translated directly to Gtk: The first format places
widgets/GUIs in adjacent cells in a table, and the second format places one
widget/GUI inside of another (NodeGUI 1 is the container for NodeGUI 2). 
(Remember, the widget/GUI appears in the window of the node that is parent to
the network.)

Now, you may be thinking 'Hey, this is going to produce one very big GUI, if
the parent has a network containing 100s of nodes!'  But the trick (and a very
neat one at that) is that the GUIs from individual nodes can be switched on
and off.  Recall that I mentioned you can switch between network and GUI views
in each window.  Well, switch a node's window view to network (and not GUI),
and its GUI won't appear in its parent's GUI.  And this can all be saved!  The
result is, you have custom-taylored GUIs for each node.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From chapmanb at arches.uga.edu  Tue Mar 28 17:38:33 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Greetings.
Message-ID: <200003282241.RAA121792@archa11.cc.uga.edu>

Hi Deanne; 
    I just went through the overwhealming task of getting up to speed 
with Loci a few months ago, so I know it can be quite a task! If you 
are interested in coding, I have quite a few small (and large) 
projects that I would be more than happy to direct your way, if you 
are interested :-) Of course, Jeff coordinates this project but it 
might be helpful to start with a few small things at first to get the 
feel for the code and all that (unless you are all ready to jump off 
on a big project!).
    As Jeff said, we are going to be merging with two other projects 
(GMS and Overflow) into one huge project tentatively called VSH. There 
are currently tons of coding choices, including programming in C, C++ 
or python, depending on what you have experience in or are interested 
in. The Loci code provides a beautiful GUI courtesy of Jeff's creative 
mind and a bunch of classes to represent the GUI as XML. The GMS and 
Overflow projects will eventually provide the processing power behind 
all of this, and we'll be communicating with them via a CORBA 
interface (which is nearly designed in it's first stage) using the 
XML descriptions of the GUI.
    If you are creative (unlike me!), there are millions of things to 
work on in the graphical user interface. We use the python bindings to 
Gnome/Gtk for the current GUI (although multiple GUIs are planned) and 
Jeff has a lot of great ideas for things that still need 
implementation. There is also lots to do with XML processing (we use 
the SAX interface for parsing XML and DOM for manipulating it) and 
other stuff.
    Sorry to go on--Loci is a big project and can be quite 
overwhealming at times, so I am very excited to hear someone is 
interested in pitching in. And, as a graduate of Michigan State, I 
won't even hold it against you that you go to U of M :-) Thanks again 
for writing--look forward to hearing more from you!
    
Brad





From mary18 at yifan.net  Thu Mar 30 06:46:57 2000
From: mary18 at yifan.net (mary18@yifan.net)
Date: Fri Feb 10 19:19:27 2006
Subject: [Pipet Devel] Email Advertising Works--Special Rates till Friday
References: 0A8A1AE4F
Message-ID: <c1bswaxiprp1zsi.300320000646@localhost>

PUT EMAIL MARKETING TO WORK FOR YOU...

Call NOW and receive 50,000
additional emails with your order
for only $100. Thats 40,000 FREE emails!!!

WE HAVE OPT-IN LISTS!!!!

see below for removal.

Special Ends Friday March 31, 2000

MLM'ers, We can build your downline.

Imagine having a product or idea and selling it
for only $10.

Now imagine sending an ad for your product or idea
to 25 million people!

If you only get a 1/10 of 1% response
you have just made $250,000!!

You hear about people getting rich off
the Internet everyday on TV,
now is the perfect time for you
to jump in on all the action.

FACT.
With the introduction of the Internet, one primary
KEY to conducting your business successfully is
creating massive exposure in a cost effective
manner.

FACT.
The experts agree that email marketing is one of
the most cost effective forms of promotion in
existence today.

Electronic mail has overtaken the telephone as the
primary means of business communication.(American
Management Association)

Of online users 41 percent check their email daily.

"A gold mine for those who can take advantage of
bulk email programs"- The New York Times

"Email is an incredible lead generation tool"
-Crains Magazine

"Blows away traditional Mailing"-Advertising Age

"It's truly arrived. Email is the killer app so
far in the online world"-Kate Delhagen, Forrester
Research Analyst

Why not let a professional company handle your
direct email marketing efforts for you?

*We will assist you in developing your entire
campaign!

*We can even create your ad or annoucement for
you!

*No responses? We resend at no cost!

For More Information CALL NOW-702-248-1043


For removal see below.

SPECIAL RATES
SPECIAL ENDS Friday March 31, 2000

Targeted Rates Upon Request.

BONUS!!!
Call In and receive 50,000 Extra Emails at
No Cost!

Call NOW - 702-248-1043








++++++++++++++++++++++++++++++++++++++++++++++++++
We are terribly sorry if you received this message
in error.
If you wish to be removed. Please, type "REMOVE"
in the subject line:   outnow@fiberia.com

++++++++++++++++++++++++++++++++++++++++++++++++++



From bizzaro at geoserve.net  Fri Mar 17 13:38:37 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu>
Message-ID: <38D27BAD.ADD4BA3E@geoserve.net>

Brad Chapman wrote:
>
> I think it's important that Overflow and GMS aren't completely
> dependent on each other (so you could use GMS for "big" programs
> with distributed systems and use Overflow for "small" libraries on a
> local scale).

I agree that there should be some separation of functionality, but to the
point where it would still be possible (with code modification) to get one
application to execute without the others being present.  IOW, I still don't
want to see GMS or Overflow dismantled, dissolved and absorbed.

BUT, if our projects are to merge, none of them will any longer be stand-alone
applications.  Of course.

> However, I know for my needs that I would like to use
> both "small" and "big" together, and so I'd like to see Overflow and
> GMS interoperate, so that you don't have to choose one core over
> another.

Right.

> Would it be difficult to define a "distributed API" for GMS
> that Overflow could plug into (when distribution over multiple
> computer was necessary)? Then Overflow could define a "small library
> API" that GMS could plug into to run libraries? Does this make any
> sense?

Yeah, I think this has to be done.  Neither Overflow nor GMS can be agnostic
about the other.  But, for Loci's sake, we don't want to be flip-flopping on
how user-to-core-to-application/object communication will occur.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 17 14:08:53 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net>
Message-ID: <38D282C5.656BDCD9@geoserve.net>

Jarl van Katwijk wrote:
> 
> Yes surtainly does. I've proposed this before, but the overflow people
> want to keep the processing speed
> and dont want any wrapping.

This whole 'speed' issue has me confused.  If the small Overflow objects
remain 'networked' on a sigle host (none are sent through the Internet), how
can GMS affect their speed???

And if small Overflow objects are 'networked' across the Internet, IT'S THE
INTERNET THAT WILL SLOW THEM DOWN, NOT A GMS WRAPPER.  Isn't that obvious to
everone else?

> I like to see code like overflow intergrated
> into gms, but this wont be started
> soon because of the mayor priorities like XML and other that need to be
> coded.

Code *LIKE* Overflow's, fine.

You know, it's looking more to me as though both Jarl's and Jean-Marc's ideas
for connecting/merging GMS and Overflow are not incompatible.  It seems
they're two sides of the same coin.  Overflow will need GMS for
Overflow-to-Overflow (across the Internet) communication, and GMS will need
Overflow for application construction (and I think wrapping).

The point of contention here is this: WHICH COMMUNICATES *DIRECTLY* WITH THE
GUI???

As I mentioned before, I'd like to see the GUI communicate with GMS, GMS
communicate with Overflow across the Internet, and Overflow handle application
wrapping/binding and development.  So, each and every GMS node is an instance
of Overflow on the Internet, which may in many cases connect to foreign
applications running on that host.

                  +------------+
                  |  Loci GUI  |
                  +------------+
                        |
                        |
                        |
                        |
                       \|/
                  +-----------+
                  |    GMS    |
                  +-----------+
                     /     \
                    /       \
         INTERNET  /         \  INTERNET
                  /           \
                 /             \
                |               |
               \|/             \|/
          +-----------+   +-----------+
          |    GMS    |   |    GMS    |
          +-----------+   +-----------+
                |               |
                |               |
               \|/             \|/
          +-----------+   +-----------+
          | Overflow  |   | Overflow  |
          +-----------+   +-----------+
                |               |
                |               |
               \|/             \|/
        +-------------+   +-------------+
        | Application |   | Application |
        +-------------+   +-------------+

Comments?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Fri Mar 17 14:57:47 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net> <38D282C5.656BDCD9@geoserve.net>
Message-ID: <38D28E3B.ADD8AB66@hermes.usherb.ca>

> > Yes surtainly does. I've proposed this before, but the overflow people
> > want to keep the processing speed
> > and dont want any wrapping.

I have nothing against wrapping in itself, nor do I think speed matters anymore
when you have the parts distributed across a network. What bothers me the most
is that your program would have to be written in "two different languages": one
GMS part, and one Overflow part, while I think it would be feasable to put the
GMS code (not rewrite it) into Overflow. Now, maybe the best thing would be to
do both: Adapt Overflow so that it can be used as a GMS node, but also add
network stuff to Overflow. The kind of task you want to perform will then tell
you which to use.

> 
> This whole 'speed' issue has me confused.  If the small Overflow objects
> remain 'networked' on a sigle host (none are sent through the Internet), how
> can GMS affect their speed???
> 
> And if small Overflow objects are 'networked' across the Internet, IT'S THE
> INTERNET THAT WILL SLOW THEM DOWN, NOT A GMS WRAPPER.  Isn't that obvious to
> everone else?

True, my concern with speed was only that Overflow nodes could not be put in GMS
individually, but that's probably obvious to everybody. Sorry, I guess I have
not expresed myself correctly on speed. 


> You know, it's looking more to me as though both Jarl's and Jean-Marc's ideas
> for connecting/merging GMS and Overflow are not incompatible.  It seems
> they're two sides of the same coin.  Overflow will need GMS for
> Overflow-to-Overflow (across the Internet) communication, and GMS will need
> Overflow for application construction (and I think wrapping).
> 
> The point of contention here is this: WHICH COMMUNICATES *DIRECTLY* WITH THE
> GUI???

I'm afraid we might need two GUIs (though they might be bonobized to look like
one), because GMS and Overflow will probably have different requirements. This
is what I'd find annoying in wrapping Overflow in GMS. I might be wrong on that
though...

About Overflow integration in GMS... I'd like to point out that, except for the
GUI, Overflow is more a library than a program. It is easy to integrate into any
program. I'd like to know, is GMS build a bit the same?

OK, so this summerized my thoughts:
1) I don't mind if Overflow is integrated in GMS (mainly it's not a lot of work)
and I can help those who want to do it (It would probably be as simple as
writing a couple new nodes). 
2) I would like to try the opposite so for most simple "networked applications",
just Overflow would be fine.

As a last, about communicating objects between different parts, Overflow already
has an Object hierarchy (GTK-like, but cleaner) that supports
serialization/de-serialization, so most of the stuff would already be done. Does
GMS has something like that (in which case, we'd need to have compatibility
between the two)?

I hope it clarifies a bit what I think.

Regards,
	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jean-marc.valin at hermes.usherb.ca  Fri Mar 17 15:03:35 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: which handles distribution (was: vsh?)
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net> <38D282C5.656BDCD9@geoserve.net>
Message-ID: <38D28F97.D174716B@hermes.usherb.ca>

> > Yes surtainly does. I've proposed this before, but the overflow people
> > want to keep the processing speed
> > and dont want any wrapping.

I have nothing against wrapping in itself, nor do I think speed matters anymore
when you have the parts distributed across a network. What bothers me the most
is that your program would have to be written in "two different languages": one
GMS part, and one Overflow part, while I think it would be feasable to put the
GMS code (not rewrite it) into Overflow. Now, maybe the best thing would be to
do both: Adapt Overflow so that it can be used as a GMS node, but also add
network stuff to Overflow. The kind of task you want to perform will then tell
you which to use.

> 
> This whole 'speed' issue has me confused.  If the small Overflow objects
> remain 'networked' on a sigle host (none are sent through the Internet), how
> can GMS affect their speed???
> 
> And if small Overflow objects are 'networked' across the Internet, IT'S THE
> INTERNET THAT WILL SLOW THEM DOWN, NOT A GMS WRAPPER.  Isn't that obvious to
> everone else?

True, my concern with speed was only that Overflow nodes could not be put in GMS
individually, but that's probably obvious to everybody. Sorry, I guess I have
not expresed myself correctly on speed. 


> You know, it's looking more to me as though both Jarl's and Jean-Marc's ideas
> for connecting/merging GMS and Overflow are not incompatible.  It seems
> they're two sides of the same coin.  Overflow will need GMS for
> Overflow-to-Overflow (across the Internet) communication, and GMS will need
> Overflow for application construction (and I think wrapping).
> 
> The point of contention here is this: WHICH COMMUNICATES *DIRECTLY* WITH THE
> GUI???

I'm afraid we might need two GUIs (though they might be bonobized to look like
one), because GMS and Overflow will probably have different requirements. This
is what I'd find annoying in wrapping Overflow in GMS. I might be wrong on that
though...

About Overflow integration in GMS... I'd like to point out that, except for the
GUI, Overflow is more a library than a program. It is easy to integrate into any
program. I'd like to know, is GMS build a bit the same?

OK, so this summerized my thoughts:
1) I don't mind if Overflow is integrated in GMS (mainly it's not a lot of work)
and I can help those who want to do it (It would probably be as simple as
writing a couple new nodes). 
2) I would like to try the opposite so for most simple "networked applications",
just Overflow would be fine.

As a last, about communicating objects between different parts, Overflow already
has an Object hierarchy (GTK-like, but cleaner) that supports
serialization/de-serialization, so most of the stuff would already be done. Does
GMS has something like that (in which case, we'd need to have compatibility
between the two)?

I hope it clarifies a bit what I think.

Regards,
	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jarl at casema.net  Fri Mar 17 16:01:42 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] Functional?
Message-ID: <38D29D36.39EC50C2@casema.net>




From jarl at casema.net  Fri Mar 17 16:01:29 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] vsh
References: <200003171355.IAA102600@archa10.cc.uga.edu> <38D24EEE.F47F0C77@casema.net> <38D282C5.656BDCD9@geoserve.net> <38D28E3B.ADD8AB66@hermes.usherb.ca>
Message-ID: <38D29D29.FD96E9DA@casema.net>

> is that your program would have to be written in "two different languages": one
> GMS part, and one Overflow part, while I think it would be feasable to put the
> GMS code (not rewrite it) into Overflow. Now, maybe the best thing would be to
> do both: Adapt Overflow so that it can be used as a GMS node, but also add
> network stuff to Overflow. The kind of task you want to perform will then tell
> you which to use.
>

This will be a slow process if you wanna do it right. Overflow & gms both want to
lose
functionality; lets slowly build toward whatever will turn out to be best. All we
need to
deside is if we want to build something together and need to tie up some ends at
first,
or if this is somehow impossible. It looks to me that everybody wants to do the
1st,
(like Jeffs diagram), so lets give it a try and evaluate after we got a pilot.

Right now just to translating projects into the other looks practically impossible
to me.


> True, my concern with speed was only that Overflow nodes could not be put in GMS
> individually, but that's probably obvious to everybody. Sorry, I guess I have
> not expresed myself correctly on speed.

OK, so Jean, what do you think about Jeffs diagram?


> I'm afraid we might need two GUIs (though they might be bonobized to look like
> one), because GMS and Overflow will probably have different requirements. This
> is what I'd find annoying in wrapping Overflow in GMS. I might be wrong on that
> though...

No, gms\overflow will figure out how does what, the gui just has 1 set of
functions.

>
> About Overflow integration in GMS... I'd like to point out that, except for the
> GUI, Overflow is more a library than a program. It is easy to integrate into any
> program. I'd like to know, is GMS build a bit the same?

The core uses corba, the plugins are libraries.

> OK, so this summerized my thoughts:
> 1) I don't mind if Overflow is integrated in GMS (mainly it's not a lot of work)
> and I can help those who want to do it (It would probably be as simple as
> writing a couple new nodes).
> 2) I would like to try the opposite so for most simple "networked applications",
> just Overflow would be fine.

OK, fine to me

>
> As a last, about communicating objects between different parts, Overflow already
> has an Object hierarchy (GTK-like, but cleaner) that supports
> serialization/de-serialization, so most of the stuff would already be done. Does
> GMS has something like that (in which case, we'd need to have compatibility
> between the two)?

Can you explain more about what you're aiming at? At how nodes are related?




From bizzaro at geoserve.net  Sat Mar 18 09:47:44 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
Message-ID: <38D39710.D0DC884A@geoserve.net>

Loci was called a 'connectivity broker' for a good reason: It wasn't going to
dictate the protocol to be used between nodes.  So, a way for Overflow to
communicate with other Overflows running across the Internet, is to let
Overflow establish its own line of communication.  Here is a flowchart
illustrating that:


                           +--------+
                  (HOST A) |  Loci  |
                           +--------+
                              | /|\
                              |  |
                        LOCAL |  | LOCAL
                              |  |
                             \|/ |
                           +--------+
                  (HOST A) |  GMS   |
                           +--------+
                              | /|\
                              |  |
                              |  |
                             /    \
                            /      \
                           /        \
               INTERNET   /          \   INTERNET
                         /            \
                        |              |
                        |              |
                       \|/             |
                  +--------+        +--------+
        (HOST B)  |  GMS   |        |  GMS   |  (HOST C)
                  +--------+        +--------+
                    |                     /|\
             LOCAL  |                      |   LOCAL
                    |                      |
                   /                        \
  +----------+/___/                          \____+----------+
  |          |\                                   |          |
  | Overflow |                                    | Overflow |
  |          |___________________________________\|          |
  +----------+        OVERFLOW'S PROTOCOL        /+----------+
    (HOST B)                                        (HOST C)


Now if Overflow wraps foreign applications, Overflow can directly control
node-to-node communication while GMS 'brokers' (one broker; 1st approach to
node-to-node communication):


                           +--------+
                  (HOST A) |  Loci  |
                           +--------+
                              | /|\
                              |  |
                        LOCAL |  | LOCAL
                              |  |
                             \|/ |
                           +--------+
                  (HOST A) |  GMS   |
                           +--------+
                              | /|\
                              |  |
                              |  |
                             /    \
                            /      \
                           /        \
               INTERNET   /          \   INTERNET
                         /            \
                        |              |
                        |              |
                       \|/             |
                  +--------+        +--------+
        (HOST B)  |  GMS   |        |  GMS   |  (HOST C)
                  +--------+        +--------+
                    |                     /|\
             LOCAL  |                      |   LOCAL
                    |                      |
    (HOST B)       /                        \       (HOST C)
  +----------+/___/                          \____+----------+
  |          |\                                   |          |
  | Overflow |                                    | Overflow |
  |          |___________________________________\|          |
  +----------+        OVERFLOW'S PROTOCOL        /+----------+
      | /|\                                           | /|\
      |  |   LOCAL                             LOCAL  |  |
     \|/ |                                           \|/ |
  +----------+                                    +----------+
  |   App.   |                                    |   App.   |
  +----------+                                    +----------+
    (HOST B)                                        (HOST C)


But in some circumstances (e.g., when the foreign app has its own protocol),
both Overflow and GMS can 'broker' (not directly control) node-to-node
communication (two brokers; 2nd approach):


                           +--------+
                  (HOST A) |  Loci  |
                           +--------+
                              | /|\
                              |  |
                        LOCAL |  | LOCAL
                              |  |
                             \|/ |
                           +--------+
                  (HOST A) |  GMS   |
                           +--------+
                              | /|\
                              |  |
                              |  |
                             /    \
                            /      \
                           /        \
               INTERNET   /          \   INTERNET
                         /            \
                        |              |
                        |              |
                       \|/             |
                  +--------+        +--------+
        (HOST B)  |  GMS   |        |  GMS   |  (HOST C)
                  +--------+        +--------+
                    |                     /|\
             LOCAL  |                      |   LOCAL
                    |                      |
    (HOST B)       /                        \       (HOST C)
  +----------+/___/                          \____+----------+
  |          |\                                   |          |
  | Overflow |                                    | Overflow |
  |          |                                    |          |
  +----------+                                    +----------+
      |                                                 /|\
      |  LOCAL                                    LOCAL  |
     \|/                                                 |
  +----------+                                    +----------+
  |   App.   |___________________________________\|   App.   |
  +----------+           ANY PROTOCOL            /+----------+
    (HOST B)                                        (HOST C)


And in other circumstances, GMS may directly control node-to-node
communication (no brokers; 3rd approach):


                           +--------+
                  (HOST A) |  Loci  |
                           +--------+
                              | /|\
                              |  |
                        LOCAL |  | LOCAL
                              |  |
                             \|/ |
                           +--------+
                  (HOST A) |  GMS   |
                           +--------+
                              | /|\
                              |  |
                              |  |
                             /    \
                            /      \
                           /        \
               INTERNET   /          \   INTERNET
                         /            \
                        |              |
                        |              |
                       \|/             |
                  +--------+        +--------+
        (HOST B)  |  GMS   |_______\|  GMS   |  (HOST C)
                  +--------+       /+--------+
                    |  /|\             |  /|\
             LOCAL  |   |              |   |   LOCAL
                    |   |              |   |
    (HOST B)       /   /               \    \       (HOST C)
  +----------+/___/   /                 \    \____+----------+
  |          |\      /                   \        |          |
  | Overflow |______/                     \______\| Overflow |
  |          |                                   /|          |
  +----------+                                    +----------+
      | /|\                                           | /|\
      |  |   LOCAL                             LOCAL  |  |
     \|/ |                                           \|/ |
  +----------+                                    +----------+
  |   App.   |                                    |   App.   |
  +----------+                                    +----------+
    (HOST B)                                        (HOST C)


I think we can make use of all 3 approaches and choose between none, one and
two brokers, depending on the circumstances.

Comments please.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Sat Mar 18 10:03:52 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net>
Message-ID: <38D39AD8.123A5BE1@geoserve.net>

"J.W. Bizzaro" wrote:

>                   +--------+        +--------+
>         (HOST B)  |  GMS   |_______\|  GMS   |  (HOST C)
>                   +--------+       /+--------+
                               GMS's
                              PROTOCOL
[cut]
>   +----------+/___/                          \____+----------+
>   |          |\                                   |          |
>   | Overflow |                                    | Overflow |
>   |          |___________________________________\|          |
>   +----------+        OVERFLOW'S PROTOCOL        /+----------+

Under what circumstances should 'VSH' switch between using Overflow's protocol
and GMS's?  Well, if we decide that GMS and Overflow will use DIFFERENT
protocols to communicate between themselves (i.e., Overflow-to-Overflow and
GMS-to-GMS), we can find these circumstances.

Perhaps Overflow can use a protocol better for small objects: like CORBA.  And
GMS could use a protocol better for moving large files: straight TCP/IP.

Hmmmm.  Me like.  How about you?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Sat Mar 18 19:13:32 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net> <38D39AD8.123A5BE1@geoserve.net>
Message-ID: <38D41BAC.9F3352C6@casema.net>

>

Hi all,

thnx Jeff for all the flowcharts.. you're really putting a lot of work into it.

> Under what circumstances should 'VSH' switch between using Overflow's protocol
> and GMS's?  Well, if we decide that GMS and Overflow will use DIFFERENT
> protocols to communicate between themselves (i.e., Overflow-to-Overflow and
> GMS-to-GMS), we can find these circumstances.

What does overflow-overflow comm. mean: 1) overflow NODE <-> NODE communication
or 2) overflow SUBNET <-> SUBNET communication?

>
> Perhaps Overflow can use a protocol better for small objects: like CORBA.  And
> GMS could use a protocol better for moving large files: straight TCP/IP.
>

GMS will get a specialised data transport protocal, I'm thinking about libgnet.
(http://www.eecs.umich.edu/~dhelder/misc/gnet/) It's a specialised network
library build on top of glib. So it's portable, fast and secure.

I think the 'overflow layer' should not use corba. That would be too slow and is
highly vurnerable to exploiting. It should have a simple communication mechanism,

corba would be overkill, also because the overflow nodes shouldn't transport data

into nodes on non-local systems. I like to see the overflow layer as nodes that
can do anything INSIDE THEIR SUBNET, they can not communicate trough the
overflow layer to another subnet, the dataflow must pass the gms layer.

I do not care about the numers of nodes, what they do, how or why, but they
should
do it all inside their subnet, or addressing space.
This is critical to me, so I want everybody to comment on this.





From jarl at casema.net  Sat Mar 18 19:24:41 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:34 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net>
Message-ID: <38D41E49.F8DB7232@casema.net>

> I think we can make use of all 3 approaches and choose between none, one and
> two brokers, depending on the circumstances.
>
> Comments please.
>

Chart 1 is not acceptable for security and management reasons,
chart 2 is fine, applications communication just the way they should have done
without any system wrapping them,
chart 3 is best when the internet had unlimited capacity :)

I like to see 2 and a 3b possible,
where 3b is Jeffs 3, only the 'gms layer' will try to clone remote nodes to
local addres space

Any reason why 1 should not be thrown away?

bye,
jarl





From bizzaro at geoserve.net  Tue Mar 21 12:11:25 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] vsh web page
Message-ID: <38D7AD3D.4CFAA892@geoserve.net>

I made a simple Web page for 'VSH' (if that is to be the name):

    http://bioinformatics.org/vsh/

with screenshots and links to the respective projects.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 21 11:57:53 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu>
Message-ID: <38D7AA11.3796AA@geoserve.net>

Justin Bradford wrote:
> 
> http://www.levien.com/athshe/prelim-design.html
[...]
> So back when I was looking at these things, I
> talked to Raph about gdome and we discussed the
> possibility of a light-weight wire protocol to
> do DOM like things over the network efficiently.
> 
> The link above describes Raph's thoughts/design
> ideas on such a system.

I'd like to hear comments from Jarl, Dominic, Jean-Marc and Brad on this.

> Now, here is why I think this is extremely useful
> for Loci (or whatever we call it now ;)
> 
> Bioinformatic/computational data can be very
> large, and it would be bad to have to transfer
> big blocks of data when you just need a bit of
> it.

Right.

> Instead, imagine a simple API to a tree-like
> structure. You could quickly browse through the
> structure until you found the right nodes,
> lists, and/or blobs of data. Then you just get
> the data you want. Even substrings of binary
> blobs.

...but using DOM instead of XML.

> Also, it discusses the implementation of locking
> and transaction for these distributed objects,
> which could be also be very useful for things
> like distributed computation and simulation.

Yes.  At a lower level than what we've been considering, these things are
important.  Now that we will be working with low-level objects via Overflow,
we should look into this.

> Anyway, this Athshe describes my original vision
> for distributed communication/object interaction
> in Loci. I'm curious what people think, and if
> such a system would still be useful for Loci's
> current direction.

Currently, we have 3 different ideas about how distribution can be handled:
one for each project.  Our challenge is to bring these ideas together.

Athshe is (yet) another way to implement object distribution and, having
little experience with DOM, is perhaps a bit out of my area of expertise.  So,
I'll defer to comment.  I want to hear what Jarl, Dominic, Jean-Marc and Brad
have to say.

Justin, if you want to see Athshe or an Athshe-like system implemented in VSH,
you're more than welcome to help code it :-)  This is a GNU project afterall.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 21 15:11:10 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] getting started with Loci
Message-ID: <38D7D75E.E3EB9A20@geoserve.net>

This is in Loci's README file:

--------------------------------------------------------

GETTING STARTED WITH LOCI

There are four basic objects you will be working with in Loci:

    (1) Workspace
    (2) Locus
    (3) Connector
    (4) Windowlet

When you first start Loci with the Gnome interface (default start), you will
see an empty 'workspace' (for now).

To place a 'locus' on the workspace, press the right mouse button, and you
will get a pop-up menu:

                    +------------+
                    | Locus    > |
                    | Add      > |
                    | Session  > |
                    | Help     > |
                    +------------+

Choosing the sub-menu Add, you will find a list of loci that can be placed on
the workspace.  Choose a locus.

                        +------+
                        |      |
                   O----|      |---->O
                        |      |
                        +------+
                       [Locus One]

Each locus (represented by an icon) will appear on the workspace with lines
radiating from a central icon.  These lines are 'connectors' and will be used
to connect one locus to another.  The lines with an arrow pointing away from
the icon are 'output connectors'.  Lines without an arrow are 'input
connectors'.

            +------+                +------+
            |      |                |      |
       O----|      |---->O     O----|      |---->O
            |      |                |      |
            +------+                +------+
           [Locus One]             [Locus Two]

You will also notice that there is a red dot at the end of the connectors.
The colors indicate the connection status: red for unconnected, yellow for
connection in progress, and green for connected.  It's simple.

Connecting loci is simple too.  Just drag-and-drop the dot from one connector
onto the dot of another connector.  (Note, you can only connect connectors
that have been unconnected, plus output to input and visa versa.)

            +------+                +------+
            |      |                |      |
       O----|      |------->O-------|      |---->O
            |      |                |      |
            +------+                +------+
           [Locus One]             [Locus Two]

You can examine the GUI for each locus by double-clicking on the center icon.
This will lower a 'windowlet', which is akin to an X window but always stays
on the workspace and attached to its locus.

                        +------+
                        |      |
                   O----|      |---->O
                        |      |
                        +------+
                       [Locus One]
                +----------------------+
                |                      |
                |                      |
                |                      |
                |                      |
                |                      |
                |                      |
                |                      |
                +----------------------+

The windowlet will contain all sorts of goodies related to the function of
the locus.  Double-clicking on the center icon again will raise the
windowlet.

--------------------------------------------------------

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 21 15:35:57 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] G-CHIP (Graphical, Command-Line, Hyperlinked Interface Program)
Message-ID: <38D7DD2D.573B01FA@geoserve.net>

There are some very interesting ideas in this 'paper':

  http://www.geocities.com/SiliconValley/Bay/5371/gchip.html

Jeff


From bizzaro at geoserve.net  Tue Mar 21 16:49:37 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] graphical console
Message-ID: <38D7EE71.A69CFF77@geoserve.net>

I like this mockup of a 'graphical console' from the G-CHIP page (attached). 
A Loci/VSH widget like this could be connected as a viewer/visual for the 'ls'
command.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: g-chip.gif
Type: image/gif
Size: 1652 bytes
Desc: not available
Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000321/9f735090/g-chip-0001.gif
From jarl at casema.net  Wed Mar 22 03:58:28 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
Message-ID: <38D88B34.69264157@casema.net>

> >
> > http://www.levien.com/athshe/prelim-design.html
> >
> > The link above describes Raph's thoughts/design
> > ideas on such a system.
>
> I'd like to hear comments from Jarl, Dominic, Jean-Marc and Brad on this.
>

Logical. I never intended to use XML internally. The way I look at XML is using
it for GUI-Scripts-Core communications, not need for using XML in every phase.


> > Bioinformatic/computational data can be very
> > large, and it would be bad to have to transfer
> > big blocks of data when you just need a bit of
> > it.
> Right.

jep, that's why GMS uses raw data 4kb shunks primairy, and offers optional
machalling to strings\intergers\etc.

>
> ...but using DOM instead of XML.

Why DOM? What does dom have superiour over XML?

>
> > Also, it discusses the implementation of locking
> > and transaction for these distributed objects,
> > which could be also be very useful for things
> > like distributed computation and simulation.

> Athshe is (yet) another way to implement object distribution and, having
> little experience with DOM, is perhaps a bit out of my area of expertise.  So,
> I'll defer to comment.  I want to hear what Jarl, Dominic, Jean-Marc and Brad
> have to say.

Everything can be learned as far as there's a logic need for it :)




From jarl at casema.net  Wed Mar 22 04:10:28 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Overview
References: <38D7D75E.E3EB9A20@geoserve.net>
Message-ID: <38D88E04.841CB466@casema.net>

Hi,

I've been working this weekend a lot on GMS to get some features done, but soon
we'll have
to work a little more coordinated, so I invite everybody to propose the things
they think need
to be done for their 'own' projects in order to bring us 3 together.
I'm trying to plan a very rough time-schedule in which will be working.

I'll be doing some 'testing' using the overflow library with gms, and will code
some planned gms
features, which will take around 2 weeks.
Maybe Overflow or Loci are currently into a phase that will take more time
before any merge is
possible.

-> How long before every project is ready for any merge attempt?

bye,
jarl




From bizzaro at geoserve.net  Wed Mar 22 09:16:40 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net> <38D39AD8.123A5BE1@geoserve.net> <38D41BAC.9F3352C6@casema.net>
Message-ID: <38D8D5C8.C9C029F0@geoserve.net>

jarl van katwijk wrote:
> 
> > Under what circumstances should 'VSH'
> > switch between using Overflow's protocol
> > and GMS's?  Well, if we decide that GMS
> > and Overflow will use DIFFERENT
> > protocols to communicate between themselves
> > (i.e., Overflow-to-Overflow and
> > GMS-to-GMS), we can find these circumstances.
> 
> What does overflow-overflow comm. mean:
> 1) overflow NODE <-> NODE communication
> or 2) overflow SUBNET <-> SUBNET communication?

Definitely 2).  I'm talking about instances of Overflow or Overflow sessions
(same applies to GMS).

> > Perhaps Overflow can use a protocol better
> > for small objects: like CORBA.  And
> > GMS could use a protocol better for moving
> > large files: straight TCP/IP.
> 
> GMS will get a specialised data transport
> protocal, I'm thinking about libgnet.
> (http://www.eecs.umich.edu/~dhelder/misc/gnet/)
> It's a specialised network
> library build on top of glib. So it's portable,
> fast and secure.

I've heard of it, and I think I remember the author proposing it on the
gnome-devel list.

> I think the 'overflow layer' should not use
> corba. That would be too slow and is
> highly vurnerable to exploiting. It should
> have a simple communication mechanism,
> corba would be overkill, also because the
> overflow nodes shouldn't transport data
> into nodes on non-local systems.

Did you hear that, Brad?  ;-)

> I like to
> see the overflow layer as nodes that
> can do anything INSIDE THEIR SUBNET, they
> can not communicate trough the
> overflow layer to another subnet, the
> dataflow must pass the gms layer.

Loci was not going to dictate how applications communicate, if they have their
own protocol.  This has a practical implication for the computational
biologists in our group, because many Internet-based databases are on the Web
(and therefore use HTTP), and many are developing CORBA API's.  If we (the
collaborative) then want to make such a database a node in our system, should
the database's own API be translated?  Why not let it work the way it is
designed to, and let VSH (or whatever) just 'broker' the connection?

Now, regarding Overflow-to-Overflow communication, if we consider an
independently running Overflow system (not used as a library) to be an
application that can communicate with another instance of itself, why not let
it?

The non-broker system you are proposing is still needed for connecting
applications with no Internet API, such as the common UNIX tools.

> I do not care about the numers of nodes, what
> they do, how or why, but they
> should do it all inside their subnet, or
> addressing space. This is critical to me, so
> I want everybody to comment on this.

Okay, as for Overflow, you're saying that node-to-node communication should be
contained within Overflow, like so...

       +------------------------------------+
       |                                    |
       |              Overflow              |
       |                                    |
       |    +--------+         +--------+   |
       |    |        |         |        |   |
       |    |  Node  |________\|  Node  |   |
       |    |        |        /|        |   |
       |    +--------+         +--------+   |
       |                                    |
       +------------------------------------+

But I'm saying that if an Overflow node needs to communicate with another
Overflow node across the Internet, it's not the node that does the
communicating but Overflow (brokering)...

       +-----------------+            +-----------------+
       |                 |            |                 |
       |     Overflow    |            |     Overflow    |
       |                 |            |                 |
       |    +--------+   |            |   +--------+    |
       |    |        |   |___________\|   |        |    |
       |    |  Node  |   |           /|   |  Node  |    |
       |    |        |   |            |   |        |    |
       |    +--------+   |            |   +--------+    |
       |                 |            |                 |
       +-----------------+            +-----------------+

or perhaps GMS (non-brokering)...

    +-----------------------+     +-------------------------+
    |         GMS           |     |           GMS           |
    |  +-----------------+  |     |   +-----------------+   |
    |  |                 |  |     |   |                 |   |
    |  |     Overflow    |  |     |   |     Overflow    |   |
    |  |                 |  |     |   |                 |   |
    |  |    +--------+   |  |     |   |   +--------+    |   |
    |  |    |        |   |  |____\|   |   |        |    |   |
    |  |    |  Node  |   |  |    /|   |   |  Node  |    |   |
    |  |    |        |   |  |     |   |   |        |    |   |
    |  |    +--------+   |  |     |   |   +--------+    |   |
    |  |                 |  |     |   |                 |   |
    |  +-----------------+  |     |   +-----------------+   |
    |                       |     |                         |
    +-----------------------+     +-------------------------+

My point is LITTLE NODES CAN COMMUNICATE ACROSS THE INTERNET TOO! :-)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 22 09:33:50 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net> <38D41E49.F8DB7232@casema.net>
Message-ID: <38D8D9CE.8A6C4D41@geoserve.net>

jarl van katwijk wrote:
> 
> Chart 1 is not acceptable for security and management reasons,

(Chart 1: GMS brokers Overflow-to-Overflow subnet communication)

But *IF* Overflow can can be made secure and managable across the Internet,
then Chart 1 is an option, right?  I mean, it's just like Chart 2, except
Overflow is the 'foreign app'.

> chart 2 is fine, applications communication just the way
> they should have done without any system wrapping them,

(Chart 2: GMS/Overflow broker foreign app communication)

Right.  No arguments here.

> chart 3 is best when the internet had unlimited capacity :)

(Chart 3: No brokering; all communication goes through GMS)

Remember though that all those 2-way communication lines below GMS are kept
within the local host.  There's still only one communication line through the
Internet: between GMS instances.

> I like to see 2 and a 3b possible, where 3b is Jeffs 3,
> only the 'gms layer' will try to clone remote nodes to
> local addres space

Well, if the nodes are small enough, and the cloning process takes less time
than Internet communication, then why not?

> Any reason why 1 should not be thrown away?

If we agree that Overflow will not communicate with anything across the
Internet, then 1 can be thrown away.  I just wanted to leave that option open
to Jean-Marc if he wants to develop an Internet API later on.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 22 09:39:03 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net> <38D88B34.69264157@casema.net>
Message-ID: <38D8DB07.46B392E8@geoserve.net>

Jarl van Katwijk wrote:
> 
> Logical. I never intended to use XML internally. The way I look at XML is using
> it for GUI-Scripts-Core communications, not need for using XML in every phase.

Not at every phase.  But XML is also good for storage, meta-data, and GUI
representations.

> > > Bioinformatic/computational data can be very
> > > large, and it would be bad to have to transfer
> > > big blocks of data when you just need a bit of
> > > it.
> 
> jep, that's why GMS uses raw data 4kb shunks primairy, and offers optional
> machalling to strings\intergers\etc.

Jarl, what do you think about Loci's idea of using XML for GUI
representations?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 22 09:49:23 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Overview
References: <38D7D75E.E3EB9A20@geoserve.net> <38D88E04.841CB466@casema.net>
Message-ID: <38D8DD73.909AB547@geoserve.net>

Jarl van Katwijk wrote:
> 
> I've been working this weekend a lot on GMS
> to get some features done, but soon we'll have
> to work a little more coordinated, so I
> invite everybody to propose the things
> they think need to be done for their 'own'
> projects in order to bring us 3 together.

Brad and I want to propose a well-defined, GUI-to-core, XML-based, command
stream.  But we do need to now how the GMS-Overflow relationship will turn
out.

> I'm trying to plan a very rough time-schedule
> in which will be working.

Loci hasn't been working from a time schedule, but ahead and make one.

> I'll be doing some 'testing' using the
> overflow library with gms, and will code
> some planned gms features, which will
> take around 2 weeks.  Maybe Overflow or
> Loci are currently into a phase that will
> take more time before any merge is possible.

I think Brad may be ready pretty soon to make a GMS-side library for
communication with Loci's GUI.  Is that right, Brad?  It doesn't really matter
for the GUI itself.

> -> How long before every project is ready for any merge attempt?

I'll let Brad answer that for me.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 22 09:55:06 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Overview
References: <38D7D75E.E3EB9A20@geoserve.net> <38D88E04.841CB466@casema.net> <38D8DD73.909AB547@geoserve.net>
Message-ID: <38D8DECA.F029FE6A@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> I think Brad may be ready pretty soon to make a GMS-side library for
> communication with Loci's GUI.

Brad, would it be easier for you to make this 'library' in C++ (rather than
C), since you know C++ better?  Jarl is using a C++ library (Overflow) anyway,
correct?  Is this a viable option?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 22 11:47:31 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] loci snapshot; 20000321
Message-ID: <38D8F923.11788234@geoserve.net>

A new snapshot of Loci is available on the Web:

  http://bioinformatics.org/loci/download/snapshots/loci-20000321.tar.gz

The GUI and core have been completely separated, and there are some
substantial improvements in performance.  Thanks to Brad.  It's worth the
download.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 22 17:48:16 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] gui protocol revisited
Message-ID: <38D94DB0.3353F6C3@geoserve.net>

I think the GUI-to-core *API* that Brad and I have been developing for Loci is
very nice.  It's a simple dialog of request, approval, and pointing to data. 
But what I am questioning is the *protocol* used, which is a standard TCP/IP
socket.

Commands from Loci's XML-based API are streamed across a socket.  Here is a
short example where the GUI (front) requests the addition of a new workspace
(subnet), and the core (middle) performs the addition and approves a change in
the GUI to reflect what the core has done:

<front>
  <add_workspace>
    <workspace id='' >
    </workspace>
  </add_workspace>
</front>
<middle>
  <success>
    <workspace id='workspace1' >
    </workspace>
  </success>
</middle>

So, once again, this API is XML-based and ran through a socket.  My question
is: Should we use CORBA instead?  What are the advantages/disadvantages?  My
MAJOR interest is that we have multiple UIs that can be attached to (and
controlled by) a single core, can run concurrently in the same session, and
can be controlled by one-another (via the front-request, middle-approve API). 
Can we do that with CORBA?

Thoughts?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From chapmanb at arches.uga.edu  Wed Mar 22 19:22:08 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] ICQ Discussion?
Message-ID: <200003230024.TAA14558@archa11.cc.uga.edu>

Hello all;
    Is anyone going to be coding tonight want to discuss some stuff on 
ICQ? Not that I am a big ICQ user, but I thought it might be a way to 
get some stuff resolved/hashed out in a timely fashion.
    I'm kind of feeling in a programming void without direction in 
this kind of limbo stage, but have some time right now (now that tests 
just finished, yay!) to be hacking and need to get myself some 
structure and goals.
    Just a thought. I'll be hanging out tonight at 60354662 while I 
code, so if anyone wants to drop on by :-)
    
Brad



From chapmanb at arches.uga.edu  Wed Mar 22 20:46:30 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] gui protocol revisited
Message-ID: <200003230149.UAA115166@archa11.cc.uga.edu>


> So, once again, this API is XML-based and ran through a socket.  My 
question
> is: Should we use CORBA instead?  What are the 
advantages/disadvantages?  My
> MAJOR interest is that we have multiple UIs that can be attached to 
(and
> controlled by) a single core, can run concurrently in the same 
session, and
> can be controlled by one-another (via the front-request, 
middle-approve
> API). 
> Can we do that with CORBA?
> 
> Thoughts?

I thought that we could keep both the streaming XML dialog *and* 
CORBA, but just in two different places in the program. My idea was 
that we could keep the small middle that we have in Loci right now to 
communiate with the GUI front via the streaming XML dialog you 
describe. 
    This small middle could then also implement the CORBA interfaces 
to pass the information created in the GUI (as a work flow diagram) 
into the "processing" portion of the program. It would make sense to 
pass this as DOM or XML, since it seems like a nice structured way to 
store the data (and since Overflow already uses it to feed its "pull" 
networking system). I think that eventually the processing part could 
move through a work flow diagram in an even "smarter" way (ie. based 
on the fastest way to implement it) which is what Jarl seems to be 
proposing for GMS's with the neural net and genetic algorithms 
processor. Here it seens to makes sense to start with what we have 
(the Overflow pull system) and move into something more robust as 
things move along.
    I think this would make it easier to implement addition front ends 
(since they would only need to deal with XML though a pipe, and not 
implementing a CORBA interface), and will also allow us to re-use the 
code we already have for dealing with communicating with the GUI. In 
addition, I think this will allow us to make a "simpler" CORBA 
interface between the small middle and Overflow and GMS program 
processing functions, since we won't have to worry specifically about 
GUI requirements while coding these.
    Does this make any sense? I think this gives us a lot of reuse of 
code, while maintaing the streaming XML that Jeff digs and the 
CORBA/API interfaces of Overflow and GMS.

Brad



From chapmanb at arches.uga.edu  Wed Mar 22 21:12:59 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Overview
Message-ID: <200003230215.VAA146024@archa12.cc.uga.edu>

 > Jarl van Katwijk wrote:
>> 
>> I've been working this weekend a lot on GMS
>> to get some features done, but soon we'll have
>> to work a little more coordinated, so I
>> invite everybody to propose the things
>> they think need to be done for their 'own'
>> projects in order to bring us 3 together.
> 
> Brad and I want to propose a well-defined, GUI-to-core, XML-based, 
command
> stream.  But we do need to now how the GMS-Overflow relationship 
will turn
> out.

As I just mentioned, I think we can maintain the XML based streaming 
dialog while the GMS/Overflow merger is still in flux.
    What I would like to try and hash out is a plan for a CORBA 
interface with the GMS. I think for a rough outline it would need the 
following parts:

0. Initializing a connection between the "middle"/GUI and the 
processing engine.

1. A way to pass structured information (in the form of XML or DOM) to 
the processing engine, so that it can take this XML and use it to 
build it a pull network or neural net system.

2. A method for requesting a list of programs/libraries registered 
with the processing engine. I think the "middle" should keep XML 
descriptions of these separate from the processing engine, so this way 
we can keep the messaging objects light weight (ie. they don't need 
any knowledge of the description or other info about a program--just 
how to run it and get stuff back from it). This way the "middle" can 
also keep the XML files for programs organized into directories of 
similar programs, so they will be easy to find for the user. However, 
before giving a GUI access to a program representation, it will need 
to confirm with the middle that the 

3. Methods for querying the middle to determine the "status" of a 
process.

4. Methods for returning the generated info (ie. for viewing) to the 
GUI.

Others? This is just some stuff off the top of my head. I would like 
to keep it simple at first, if possible, but still keep all the 
functionality that GMS and Overflow currently have.

>> I'm trying to plan a very rough time-schedule
>> in which will be working.
> 
> Loci hasn't been working from a time schedule, but ahead and make 
one.

I would be happy to have one. I'm all about structure etc. The way my 
life works is that I work on things frantically all at once, and then 
get way behind in everything else and have to work frantically on 
them, but I can try to stick with a schedule :-) I think this would be 
very helpful for coordinating development etc.

>> I'll be doing some 'testing' using the
>> overflow library with gms, and will code
>> some planned gms features, which will
>> take around 2 weeks.  Maybe Overflow or
>> Loci are currently into a phase that will
>> take more time before any merge is possible.
> 
> I think Brad may be ready pretty soon to make a GMS-side library for
> communication with Loci's GUI.  Is that right, Brad?  It doesn't 
really
> matter
> for the GUI itself.
> Brad, would it be easier for you to make this 'library' in C++ 
(rather than
> C), since you know C++ better?  Jarl is using a C++ library 
(Overflow) anyway,
> correct?  Is this a viable option?

Shorely, I'm ready whenever Jarl is ready for me :-) Personally, I 
don't really care whether I use C or C++ and would like to avoid 
mixing languages as little as possible (too late for that, I know.) so 
I wouldn't have a big problem coding in C if that's what Jarl feels 
most comfortable with. As I said before, I don't have much experience 
with procedure oriented languages, but I would like very much to give 
it a shot.

>> -> How long before every project is ready for any merge attempt?
> 
> I'll let Brad answer that for me.

I think we're ready now <keeping my fingers tightly crossed> for any 
kind of merge. As we merge, I think we'll be able to see better what 
kind of functionality we need to add to what we have to support the 
current states of Loci and GMS.
    
    Jeff, you might want to have some discussions with the Overflow 
guys about the GUI. It seems like you both have come up with very 
similar ideas for how things should work, just with different names:

loci -> nodes
workspace -> network and subnetwork

They have some experimental stuff starting up--like iterators and 
threading, which I don't think the current GUI will support. I'm not 
sure how they feel about things and would like to handle this.
    For GMS, it seems like the move to this connect the dots type GUI 
will be a big change from the list oriented stuff it currently has, so 
Jarl's input will also be very important here. 
    We can definately do this, and I think we should start ASAP and 
just start squeezing things together and see how it goes. 
    Let me know where I can help the most and I'll start up there...

Brad





From bizzaro at geoserve.net  Wed Mar 22 22:03:50 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] ICQ Discussion?
References: <200003230024.TAA14558@archa11.cc.uga.edu>
Message-ID: <38D98996.F993D18E@geoserve.net>

Brad Chapman wrote:
> 
>     Just a thought. I'll be hanging out tonight at 60354662 while I
> code, so if anyone wants to drop on by :-)

I should have caught this message a bit earlier.  But I don't even have ICQ/U
set up on my computer.  Heck, I don't believe I've ever used it.  Well, I'm
about to pass out now anyway.  I'll catch you when the Earth faces the Sun and
reply to your other e-mails then :-)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 22 22:05:23 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] ICQ Discussion?
References: <200003230024.TAA14558@archa11.cc.uga.edu> <38D98996.F993D18E@geoserve.net>
Message-ID: <38D989F3.4715E496@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> I'll catch you when the Earth faces the Sun and

Duh.  Let's make that when New England faces the Sun ;-)

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Thu Mar 23 01:57:20 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net> <38D39AD8.123A5BE1@geoserve.net> <38D41BAC.9F3352C6@casema.net> <38D8D5C8.C9C029F0@geoserve.net>
Message-ID: <38D9C04F.1DB8911@casema.net>

> >
> > What does overflow-overflow comm. mean:
> > 1) overflow NODE <-> NODE communication
> > or 2) overflow SUBNET <-> SUBNET communication?
>
> Definitely 2).  I'm talking about instances of Overflow or Overflow sessions
> (same applies to GMS).
>

Good, Jean OK too?

>
> Loci was not going to dictate how applications communicate, if they have their
> own protocol.  This has a practical implication for the computational
> biologists in our group, because many Internet-based databases are on the Web
> (and therefore use HTTP), and many are developing CORBA API's.  If we (the
> collaborative) then want to make such a database a node in our system, should
> the database's own API be translated?  Why not let it work the way it is
> designed to, and let VSH (or whatever) just 'broker' the connection?

Yes, that's why GMS is based upon RAW data transfer (sequencial shunks of 4kb).
Let have the applications interprent the data, VSH should orientate on
distibution.

>
> Now, regarding Overflow-to-Overflow communication, if we consider an
> independently running Overflow system (not used as a library) to be an
> application that can communicate with another instance of itself, why not let
> it?

Because it's too unpredictable: it's very easy to create a 'overflow' compatible
program that exploits this detail\low level.
Jean, any comments?

>
> Okay, as for Overflow, you're saying that node-to-node communication should be
> contained within Overflow, like so...
>
> But I'm saying that if an Overflow node needs to communicate with another
> Overflow node across the Internet, it's not the node that does the
> communicating but Overflow (brokering)...

Ic, but this is where gms is (going to be) powerfull, let the overflow layer
specialise in
processing, let gms do the brokering\security and client\script IO.
It's a question of orientation of the layers. Of the design. Like a
OS kernel: diff. layers
are seperated from each other for a reason, so it's possible to remain the
overview of
the situation and to be able to implement new idears easely.

> or perhaps GMS (non-brokering)...
> My point is LITTLE NODES CAN COMMUNICATE ACROSS THE INTERNET TOO! :-)

Yes, they can, only they talk to an application or the gms layer, nerver to an
other subnet.



From jarl at casema.net  Thu Mar 23 02:07:52 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net> <38D41E49.F8DB7232@casema.net> <38D8D9CE.8A6C4D41@geoserve.net>
Message-ID: <38D9C2C8.F779BB3C@casema.net>

> (Chart 1: GMS brokers Overflow-to-Overflow subnet communication)
>
> But *IF* Overflow can can be made secure and managable across the Internet,
> then Chart 1 is an option, right?  I mean, it's just like Chart 2, except
> Overflow is the 'foreign app'.

That design would ruine the speed and flexibility I see possible for overflow. I
really
want to have the overflow capacity in the VSH system, but it should be made
isolated
in order to keep it's speed and possibilities. And gms already has taken outside
world
IO etc. in account, so why move that functionality to overflow?

>
> > Any reason why 1 should not be thrown away?
>
> If we agree that Overflow will not communicate with anything across the
> Internet, then 1 can be thrown away.  I just wanted to leave that option open
> to Jean-Marc if he wants to develop an Internet API later on.

OK to me, but just for now we need to have a goal for every project (Loci,
Overflow and gms) to work on in order to 'intergrate' them. Later on we
will evuluate.

SO the basis design for a collaboration-pilot is clear to me, I want to know if
everybody else is also OK and we can start discussing the implementation of
this pilot.
Every project please come to a conclussion..



From jarl at casema.net  Thu Mar 23 02:10:07 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net> <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net>
Message-ID: <38D9C34F.83A85DCF@casema.net>

>
> > > > Bioinformatic/computational data can be very
> > > > large, and it would be bad to have to transfer
> > > > big blocks of data when you just need a bit of
> > > > it.
> >
> > jep, that's why GMS uses raw data 4kb shunks primairy, and offers optional
> > machalling to strings\intergers\etc.
>
> Jarl, what do you think about Loci's idea of using XML for GUI
> representations?
>

Hmm? Please explain, I can't pinpoint what you're trying to say.



From jarl at casema.net  Thu Mar 23 02:12:49 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Overview
References: <38D7D75E.E3EB9A20@geoserve.net> <38D88E04.841CB466@casema.net> <38D8DD73.909AB547@geoserve.net>
Message-ID: <38D9C3F1.CD8C5FF3@casema.net>

> >
> > I've been working this weekend a lot on GMS
> > to get some features done, but soon we'll have
> > to work a little more coordinated, so I
> > invite everybody to propose the things
> > they think need to be done for their 'own'
> > projects in order to bring us 3 together.
>
> Brad and I want to propose a well-defined, GUI-to-core, XML-based, command
> stream.  But we do need to now how the GMS-Overflow relationship will turn
> out.

OK, very good. I will have some proposals to, but you people will be
'responsable' for
doing this.

Overflow people \ Jean? You're reading this list to arn't you? Let us hear your
thoughs :)




From jarl at casema.net  Thu Mar 23 02:32:06 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] gui protocol revisited
References: <38D94DB0.3353F6C3@geoserve.net>
Message-ID: <38D9C876.CC7B55CD@casema.net>

>

Hi All,

>
> <front>
>   <add_workspace>
>     <workspace id='' >
>     </workspace>
>   </add_workspace>
> </front>
> <middle>
>   <success>
>     <workspace id='workspace1' >
>     </workspace>
>   </success>
> </middle>
>

Looks nice, one thing: you're talking about 'requests', so will the GUI (and
optional other
'controllers' like scripts or knowledge databases) have (1) the 'structure'
stored internally
or will they (2) 'build' it inside gms\overflow?

GMS uses (2), I think Loci does too, but just to make sure..


>
> So, once again, this API is XML-based and ran through a socket.  My question
> is: Should we use CORBA instead?  What are the advantages/disadvantages?  My
> MAJOR interest is that we have multiple UIs that can be attached to (and
> controlled by) a single core,

Yes, this must be possible. As I mentioned before, other 'controllers' should be
possible.
These will be non-interactive clients like scipt-interpeters and KDB's

> can run concurrently in the same session, and
> can be controlled by one-another (via the front-request, middle-approve API).
> Can we do that with CORBA?

Yes. Every 'client' requests the core for a communication line, or a ORB. After
the comm.
is setup, the client can call the functions descriped by the idl. (see the gms
source for
ADMDFP.idl, which is the GUI->CORE communication api.)

The GUI just calls a LOCAL function like:
    GUI2CORE_add_node()

which will be transported (among other things) by corba to the core and call:
    IMPLEMENTATED_GUI2CORE_add_node()

which is a function located in the core.




From jarl at casema.net  Thu Mar 23 02:41:21 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003230149.UAA115166@archa11.cc.uga.edu>
Message-ID: <38D9CAA1.388FFB50@casema.net>

>
> I thought that we could keep both the streaming XML dialog *and*
> CORBA, but just in two different places in the program. My idea was
> that we could keep the small middle that we have in Loci right now to
> communiate with the GUI front via the streaming XML dialog you
> describe.

A kinda 'script interpeter'.. very good idear

>
> move through a work flow diagram in an even "smarter" way (ie. based
> on the fastest way to implement it) which is what Jarl seems to be
> proposing for GMS's with the neural net and genetic algorithms
> processor.

Yes.
The GAP will also try to apply new succesfull structures that are
generated elsewhere.


>     Does this make any sense? I think this gives us a lot of reuse of
> code, while maintaing the streaming XML that Jeff digs and the
> CORBA/API interfaces of Overflow and GMS.

OK to me. It would also be a very good point for you to start working on.



From jarl at casema.net  Thu Mar 23 02:59:25 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:35 2006
Subject: [Pipet Devel] Overview
References: <200003230215.VAA146024@archa12.cc.uga.edu>
Message-ID: <38D9CEDD.6BCF9EC0@casema.net>

>     What I would like to try and hash out is a plan for a CORBA
> interface with the GMS. I think for a rough outline it would need the
> following parts:
>
> 0. Initializing a connection between the "middle"/GUI and the
> processing engine.
>
> 1. A way to pass structured information (in the form of XML or DOM) to
> the processing engine, so that it can take this XML and use it to
> build it a pull network or neural net system.
>
> 2. A method for requesting a list of programs/libraries registered
> with the processing engine. I think the "middle" should keep XML
> descriptions of these separate from the processing engine, so this way
> we can keep the messaging objects light weight (ie. they don't need
> any knowledge of the description or other info about a program--just
> how to run it and get stuff back from it). This way the "middle" can
> also keep the XML files for programs organized into directories of
> similar programs, so they will be easy to find for the user. However,
> before giving a GUI access to a program representation, it will need
> to confirm with the middle that the
>
> 3. Methods for querying the middle to determine the "status" of a
> process.
>
> 4. Methods for returning the generated info (ie. for viewing) to the
> GUI.
>
> Others? This is just some stuff off the top of my head. I would like
> to keep it simple at first, if possible, but still keep all the
> functionality that GMS and Overflow currently have.

This will be a good starting point.

> I would be happy to have one. I'm all about structure etc. The way my
> life works is that I work on things frantically all at once, and then
> get way behind in everything else and have to work frantically on
> them, but I can try to stick with a schedule :-) I think this would be
> very helpful for coordinating development etc.

I'll be needing one more weekend before the 'merge phase'.
I'm thinkg about around 2 months time for doing a merge-pilot, not
a day-to-day schedule. That might come later :)

>
>
> Shorely, I'm ready whenever Jarl is ready for me :-) Personally, I
> don't really care whether I use C or C++ and would like to avoid
> mixing languages as little as possible (too late for that, I know.) so
> I wouldn't have a big problem coding in C if that's what Jarl feels
> most comfortable with. As I said before, I don't have much experience
> with procedure oriented languages, but I would like very much to give
> it a shot.

C is more suitable for the 'inbetween' functionality gms will get in the
VSH
design. It has more 'hackish' functionality.
But i'm also quite happy about Overflow & GUI being OOP because it makes
sence to build those OOP.

>
> I think we're ready now <keeping my fingers tightly crossed> for any
> kind of merge. As we merge, I think we'll be able to see better what
> kind of functionality we need to add to what we have to support the
> current states of Loci and GMS.

OK, then I will communicate with Brad about the details.. any dessision
that will infuence design etc will be posted pubically ofcourse.

>
> They have some experimental stuff starting up--like iterators and
> threading, which I don't think the current GUI will support. I'm not
> sure how they feel about things and would like to handle this.
>     For GMS, it seems like the move to this connect the dots type GUI
> will be a big change from the list oriented stuff it currently has, so
> Jarl's input will also be very important here.

The GUI is list-orientated. This sucks. The engine is build on a
fully-connected network structure that supports Loci's approarch.
Like I said in one of my very first posts in the Loci mailinglist: I was
looking for a better GUI to replace mine.

>
>     We can definately do this, and I think we should start ASAP and
> just start squeezing things together and see how it goes.
>     Let me know where I can help the most and I'll start up there...

K, maybe it's logical that you'll do this 'in-between' code that talks to

the gui and the core?


bye
jarl



From chapmanb at arches.uga.edu  Thu Mar 23 10:17:50 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Initial idl proposal
Message-ID: <200003231520.KAA48998@archa12.cc.uga.edu>

Hello all;
    Based on the broad design I mentioned yesterday for communicating 
between a small middle (part of the GUI) and the "data processing 
engine" of 'vsh', I drew up an idl as a starting point for this 
communication. This definately need a lot of work and if the first idl 
I've ever written, so I can use lots of comments and criticisms, but 
this will be something to build off of so we can get things going. I 
look forward to hearing comments!

Brad

// gui2dp.idl
//
// IDL for dealing with communication between the GUI (buffered by a 
small 
// middle) and the data processing engine.

/**
 * Organization of modules.
 * vsh: The overall module (the name can change when we decide on a 
formal 
 *      name).
 *   gui2dp: idl for communication between the GUI and the data 
processing 
 *           engine. 
 * 
 */

module vsh {
  module gui2dp {
    // ##### exceptions
    
    /**
     * Requested Id is already being used for another gui.
     */
    exception IdInUse {};
                            
    /**
     * Id number used for requested operation was not found in the 
list of
     * current gui ids being served by the data processor.
     */
    exception IdNotFound{};
                
    /**
     * The information for a node is not yet available because the 
process 
     * has not completed.
     */
    exception InfoNotAvailable{};
                            
    // ##### structs and typedefs
    /**
     * A sequence/list of strings. Useful for returning info like a 
list of 
     * program ids for available programs.
     */
    typedef sequence <string> stringList;
                            
    // ##### interfaces                 
                            
    /**
     * Functions for dealing with a front to middle connection.
     */
    interface Connection {
        /**
         * Initialize a connection from the gui to the data processing 
engine.
         * @parm gui_id - An unique id to represent the gui connection
         * @parm password - The password associated with this id.
         * @returns A boolean indicating the success of the 
initialization
         * @raises IdInUse - The passed gui_id is already in use by 
another gui.
         */
        boolean init_connection(in string gui_id, in string password)
            raises (IdInUse);
        /**
         * Close the connection between the gui and data processing 
engine. 
         * Used when closing down a GUI.
         * @parm gui_id - An unique id to represent the gui connection
         * @parm password - The password associated with this id.
         * @returns A boolean indicating the success of the 
initialization
         * @raises IdNotFound - The passed id was not found in the 
list of 
         *                      ids registerd with the data processing 
engine.
         */
        boolean close_connection(in string gui_id, in string password)
            raises (IdNotFound);
    };
    /**
     * Functions for passing XML information to the data processing 
engine. 
     */
    interface SendProcessingInfo {
        /** 
         * Send a string of XML to the data processing engine to be 
dealt with.
         * @parm xml_to_process: A big string of xml describing the 
nodes/loci
         *       to be processed and the relationship between them.
         * @returns An id to be used for getting information about the
         *          process sent.
         */
        string send_xml(in string xml_to_process);                     
 
    };
        
    /**
     * Functions dealing with a process that has been sent to the 
middle.
     */
    interface ProcessInformation {
        /**
         * Request information the status of the overall process.
         * @parm process_id - the unique id associated with a process.
         * @returns A double representing the progress towards 
completion.
         *          1 indicates completion, 0 indicates the process 
has not
         *          started, numbers between 0 and 1 represent the 
percent 
         *          completion, and -1 indicates an error in the 
process.
         * @raises IdNotFound - If the passed process_id is not found.
         */
        double query_process(in string process_id) raises (IdNotFound);
        
        /**
         * Request information about a particular node/locus in a 
process.
         * @parm process_id - the unique id associated with a process.
         * @parm node_id - the unique id associated with a node.
         * @returns A double representing the progress towards 
completion.
         *          1 indicates completion, 0 indicates the process 
has not
         *          started, numbers between 0 and 1 represent the 
percent 
         *          completion, and -1 indicates an error in the 
process.
         * @raises IdNotFound - if either the process or node id were 
not 
         *         found.
         */
        double query_process_node(in string process_id, in string 
node_id)
            raises (IdNotFound);
                
        /**
         * Get the information for a node so that is can be displayed 
in the 
         * GUI.
         * @parm process_id - the unique id associated with a process.
         * @parm node_id - the unique id associated with a node.
         * @returns A string with the info from the node.
         *          TODO: Need a structured and generalized way to 
return info
         *          and pass it to a visual so it can be seen in the 
GUI. 
         *          Strings aren't going to cut it for everything, 
         *          I don't think.
         * @raises InfoNotAvailable - If the node currently doesn't 
have any 
         *         information to send (ie. process is not complete.
         */
        string get_node_info(in string process_id, in string node_id)
            raises (InfoNotAvailable);  
        
    };
    /**
     * Functions for getting information about programs and libraries 
registered
     * with the processing engine.
     */
    interface ProgramInformation {
        /**
         * Get a list of all the program ids of programs registered 
functional
         * (ie. loaded) with the processing engine.
         * @returns A sequence of strings (list) with the node_ids of 
available
         *          nodes.  
         */
        stringList get_program_ids();
        
        /**
         * Query a specific node to see if it is available.
         * @parm node_id - the unique id of a node (program/function).
         * @returns 1 if the program is loaded, 0 if it is not.
         */ 
        boolean is_available(in string node_id);
        
    };
    
    
  };
};


	



From chapmanb at arches.uga.edu  Thu Mar 23 14:56:44 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Project Design (fwd)
Message-ID: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu>

Hey all;
	This is from a discussion Jarl and I were having, and I thought it
was really relevant to everyone. I like this plan for integrating
everything--whadda you all think?

Brad


---------- Forwarded message ----------
Date: Thu, 23 Mar 2000 19:45:42 +0100
From: jarl van katwijk <jarl@casema.net>
To: Brad Chapman <chapmanb@arches.uga.edu>
Subject: Project Design

>
> Well, the only think I worry about is that Overflow probably doesn't
> want to be "absorbed" into GMS and would like to be able to work
> independently. Couldn't the authentication go in a
> "processing/pre-processing engine" that then directs stuff to either
> GMS or Overflow?

I know the Overflow people dont want their code integrated, but I'm
saying
to this:

                                    Loci based GUI
                                            |
-> Communication SAME AS current Loci's GUI, only enhanced
                                            |
with various extra gms & overflow features.
                                           \/
 Scripts -----> 'Brad's Interperter' :-)
                                            |                    ->
Translates scripts\GUI calls to 'gms api+', which will be a reviced
                                            |
gms api + the current Overflow api. So gms will select what calls
                                            |
are to be handled and which ones are to be passed to overflow
                                           \/
Remote Env.---> GMS based broker --> Remote Environment
                                            |                     -> This
communication is THE SAME as subnet-subnet comm.
                                            |
AND as Oveflow-GUI comm. GMS will emulate that api
                                           \/
Local Env.----> Overflow based processor ---> Local Environment

SO Gms will be adjusted to 'speak' to overflow, overflow stays the same
as it is, only restricted to local
operation. And the Loci GUI will stay the same.

Every body happy?


> This way, this engine can work with only GMS or only
> Overflow, if a user desired.
>
> > Hmm.. I should read (again) the documentation about the
> > "pull data flow model", cant place the name now.
>
> The Overflow guys just call it "pull technology." Basically, it is
> just a recursive call to getOutput() starting at the last node in a
> series and propogating forward to the top node.

Currently gms processes one data-event at a time, so a data-chunk
passed the whole structure before the next is prosessed. This will
have to change to better 'all at once'  processing.





From jarl at casema.net  Thu Mar 23 15:43:27 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Project Design
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu>
Message-ID: <38DA81EF.8645B595@casema.net>

Hi,

Could somebody collect all the design proposals and wishes and stuff and put
them on
the VSH page. Sometimes I'm saying\reading things twice :)
I could do this , but somebody else probably write better english as I do..

bye,
jarl



From jean-marc.valin at hermes.usherb.ca  Thu Mar 23 15:30:17 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Project Design (fwd)
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu>
Message-ID: <38DA7ED9.9D3F46A1@hermes.usherb.ca>

> > Well, the only think I worry about is that Overflow probably doesn't
> > want to be "absorbed" into GMS and would like to be able to work
> > independently. Couldn't the authentication go in a
> > "processing/pre-processing engine" that then directs stuff to either
> > GMS or Overflow?
> 
> I know the Overflow people dont want their code integrated, but I'm
> saying
> to this:

Depends on what you mean by integrated. 90% of Overflow is just a library.
You're actually probably better use that library in GMS instead of the
executables. If you call that integrating Overflow in GMS, well fine, If you
want to wrap the executable, fine too (though I don't think it's the best way).
If you want to take some chunks of code and put that in GMS... have fun(good
luck)!

My only constraint is that I want to be able to use Overflow as a stand alone
application.

> 
>                                     Loci based GUI
>                                             |
> -> Communication SAME AS current Loci's GUI, only enhanced
>                                             |
> with various extra gms & overflow features.
>                                            \/
>  Scripts -----> 'Brad's Interperter' :-)


Speaking of interpreter... I was planning on implementing an Overflow node that
uses the Octave interpreter (actually, only the octave library)...


>                                             |                    ->
> Translates scripts\GUI calls to 'gms api+', which will be a reviced
>                                             |
> gms api + the current Overflow api. So gms will select what calls
>                                             |
> are to be handled and which ones are to be passed to overflow
>                                            \/
> Remote Env.---> GMS based broker --> Remote Environment
>                                             |                     -> This
> communication is THE SAME as subnet-subnet comm.
>                                             |
> AND as Oveflow-GUI comm. GMS will emulate that api
>                                            \/
> Local Env.----> Overflow based processor ---> Local Environment
> 
> SO Gms will be adjusted to 'speak' to overflow, overflow stays the same
> as it is, only restricted to local
> operation. And the Loci GUI will stay the same.
> 
> Every body happy?
> 
> > This way, this engine can work with only GMS or only
> > Overflow, if a user desired.
> >
> > > Hmm.. I should read (again) the documentation about the
> > > "pull data flow model", cant place the name now.
> >
> > The Overflow guys just call it "pull technology." Basically, it is
> > just a recursive call to getOutput() starting at the last node in a
> > series and propogating forward to the top node.
> 
> Currently gms processes one data-event at a time, so a data-chunk
> passed the whole structure before the next is prosessed. This will
> have to change to better 'all at once'  processing.

OK, there's not much technology in "pull" :-)... The idea was that you only
compute what's needed. Also, since we also have loops (even loops withing
loops), we must handle the fact that somethimes (well always) one iteration need
data that comes from a past iteration (even future sometimes, though it's not
any harder). The the recursive getOutput() is the only way to handle all the
dependencies correctly. The other advantage is that it doesn't require any kind
of "job dispatcher". You call getOutput() on the last node, and all the requests
will propagate until you have the answer. A node needs only be aware of the
nodes connected to its input, there's no need for anything "global".

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jarl at casema.net  Thu Mar 23 17:04:07 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] 3/1 and 1/3
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA7ED9.9D3F46A1@hermes.usherb.ca>
Message-ID: <38DA94D7.E40E84B4@casema.net>

>
> Depends on what you mean by integrated. 90% of Overflow is just a library.
> You're actually probably better use that library in GMS instead of the
> executables. If you call that integrating Overflow in GMS, well fine, If you
> want to wrap the executable, fine too (though I don't think it's the best way).
> If you want to take some chunks of code and put that in GMS... have fun(good
> luck)!

Using the libraries is by far the best methode.
I dont want a physical intergration, but there has got to come a general design:
which member does what?

Example: It's absolutely no problem to me that Overflow can connect to
non-local subnets\nodes, but for the collaboration's sake Overflow should
have support to disable this.
Ex2: GMS will continue to support it's own GUI, I like it for testing.

All the flowcharts describe the collaborated situation.

>
> My only constraint is that I want to be able to use Overflow as a stand alone
> application.

Agreed.




From bizzaro at geoserve.net  Thu Mar 23 21:06:14 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003230149.UAA115166@archa11.cc.uga.edu>
Message-ID: <38DACD96.8D0900F3@geoserve.net>

Brad Chapman wrote:
> 
> I thought that we could keep both the streaming XML dialog *and*
> CORBA, but just in two different places in the program. My idea was
> that we could keep the small middle that we have in Loci right now to
> communiate with the GUI front via the streaming XML dialog you
> describe.

Okay, if you like that.  And I guess Jarl likes it too.

>     This small middle could then also implement the CORBA interfaces
> to pass the information created in the GUI (as a work flow diagram)
> into the "processing" portion of the program. It would make sense to
> pass this as DOM or XML, since it seems like a nice structured way to
> store the data (and since Overflow already uses it to feed its "pull"
> networking system). I think that eventually the processing part could
> move through a work flow diagram in an even "smarter" way (ie. based
> on the fastest way to implement it) which is what Jarl seems to be
> proposing for GMS's with the neural net and genetic algorithms
> processor. Here it seens to makes sense to start with what we have
> (the Overflow pull system) and move into something more robust as
> things move along.

I'm a little confused about this 'processing part of the program'.  Is this
something GMS or Overflow has now?  Is it something you'll have to write? 
Also Brad, recall how Loci was going to use an XML database (XDBM) to store
the workflow diagram (network and subnetworks).  How does this relate to the
'processing' part?  Perhaps you can explain this to the others and how we plan
to use XML.  Will this be replaced by a DOM structure?

>     I think this would make it easier to implement addition front ends
> (since they would only need to deal with XML though a pipe, and not
> implementing a CORBA interface), and will also allow us to re-use the
> code we already have for dealing with communicating with the GUI. In
> addition, I think this will allow us to make a "simpler" CORBA
> interface between the small middle and Overflow and GMS program
> processing functions, since we won't have to worry specifically about
> GUI requirements while coding these.

Okay.

>     Does this make any sense? I think this gives us a lot of reuse of
> code, while maintaing the streaming XML that Jeff digs and the
> CORBA/API interfaces of Overflow and GMS.

I was just thinking that if we're using CORBA at all anywhere, why not use it
everywhere?

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 21:42:57 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Overview
References: <200003230215.VAA146024@archa12.cc.uga.edu>
Message-ID: <38DAD631.CE01714C@geoserve.net>

Brad Chapman wrote:
> 
> 2. A method for requesting a list of programs/libraries registered
> with the processing engine. I think the "middle" should keep XML
> descriptions of these separate from the processing engine, so this way
> we can keep the messaging objects light weight (ie. they don't need
> any knowledge of the description or other info about a program--just
> how to run it and get stuff back from it).

Absolutely.  Libraries, etc. for the GUI are only REFERRED to via URI.  This
stuff may already be available on the local host, so to minimize transfer, the
local host is given the OPTION to get a remote copy or use the local one. 
This is an old Loci discussion (initiated by Humberto and Justin IIRC).  Which
reminds me, we should also send library version information in the stream. 
That way, the local system will know if a newer version is available.

> This way the "middle" can
> also keep the XML files for programs organized into directories of
> similar programs, so they will be easy to find for the user. However,
> before giving a GUI access to a program representation, it will need
> to confirm with the middle that the

(? I think you fell asleep typing this ;-))

The others probably don't understand what we mean by 'XML representation'.

> 3. Methods for querying the middle to determine the "status" of a
> process.

Right.  BTW, I want to have a status 'symbol' (no pun intended) displayed for
and with each node.

> I would be happy to have one. I'm all about structure etc. The way my
> life works is that I work on things frantically all at once, and then
> get way behind in everything else and have to work frantically on
> them, but I can try to stick with a schedule :-)

That's hilarious.  You just described the life of every college student :-) 
I'm always trying to get things done at the last minute while neglecting that
which is due later.  Then those other things get done at the last minute, ad
infinitum.

> Shorely, I'm ready whenever Jarl is ready for me :-) Personally, I
> don't really care whether I use C or C++ and would like to avoid
> mixing languages as little as possible (too late for that, I know.)

Homogeneity will come in time.

> I think we're ready now <keeping my fingers tightly crossed> for any
> kind of merge. As we merge, I think we'll be able to see better what
> kind of functionality we need to add to what we have to support the
> current states of Loci and GMS.

Do we need a new CVS module?

>     Jeff, you might want to have some discussions with the Overflow
> guys about the GUI. It seems like you both have come up with very
> similar ideas for how things should work, just with different names:

Yeah, some of the similarities are surprising.

> loci -> nodes
> workspace -> network and subnetwork

Maybe it's more like...

  workflow diagram -> network and subnetworks

  workspace contains the workflow diagram

  composite loci contain 'sub-workspaces', which contain workflow
  diagrams that could be called subnetworks

> They have some experimental stuff starting up--like iterators and
> threading, which I don't think the current GUI will support. I'm not
> sure how they feel about things and would like to handle this.

Hmmm.  Well, I'm flexible with the design of the GUI.

>     For GMS, it seems like the move to this connect the dots type GUI
> will be a big change from the list oriented stuff it currently has, so
> Jarl's input will also be very important here.

Yep.

>     We can definately do this, and I think we should start ASAP and
> just start squeezing things together and see how it goes.
>     Let me know where I can help the most and I'll start up there...

You're definitely the go-between for Jarl and myself.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 21:47:43 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net> <38D39AD8.123A5BE1@geoserve.net> <38D41BAC.9F3352C6@casema.net> <38D8D5C8.C9C029F0@geoserve.net> <38D9C04F.1DB8911@casema.net>
Message-ID: <38DAD74F.2C323010@geoserve.net>

jarl van katwijk wrote:
> 
> Because it's too unpredictable: it's very easy to create a 'overflow' compatible
> program that exploits this detail\low level.
> Jean, any comments?

Okay, if you think so.

> Ic, but this is where gms is (going to be) powerfull, let the overflow layer
> specialise in
> processing, let gms do the brokering\security and client\script IO.
> It's a question of orientation of the layers. Of the design. Like a
> OS kernel: diff. layers
> are seperated from each other for a reason, so it's possible to remain the
> overview of
> the situation and to be able to implement new idears easely.

I agree.

> > My point is LITTLE NODES CAN COMMUNICATE ACROSS THE INTERNET TOO! :-)
> 
> Yes, they can, only they talk to an application or the gms layer, nerver to an
> other subnet.

Good.

If Jean-Marc agrees, then I have no problem with your point of view.  I'm just
considering the best inter-personal or political solution.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 21:50:08 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] node-to-node communication: you can have it 3 ways!
References: <38D39710.D0DC884A@geoserve.net> <38D41E49.F8DB7232@casema.net> <38D8D9CE.8A6C4D41@geoserve.net> <38D9C2C8.F779BB3C@casema.net>
Message-ID: <38DAD7E0.5868D424@geoserve.net>

jarl van katwijk wrote:
> 
> SO the basis design for a collaboration-pilot is clear to me, I want to know if
> everybody else is also OK and we can start discussing the implementation of
> this pilot.
> Every project please come to a conclussion..

I agree with Brad that Loci is pretty much ready for integration, although
there may be some more development in the meantime that does not conflict with
the merger.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 22:09:55 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net> <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
Message-ID: <38DADC83.F000D7B@geoserve.net>

jarl van katwijk wrote:
> 
> > Jarl, what do you think about Loci's idea of using XML for GUI
> > representations?
> 
> Hmm? Please explain, I can't pinpoint what you're trying to say.

Here's the short and simple: The user sets the parameters for nodes via some
kind of interface.  The interface is described via an XML akin to the HTML
forms generated by CGI programs.  It's much quicker than other ways to
transfer G/UIs.

Plus, the XML can be combined to generate more elaborate GUIs.  Here is a Gtk
GUI in GLADE XML as an example of how complex GUIs can be made with XML:

<widget>
  <class>GtkWindow</class>
  <name>window1</name>
  <title>window1</title>
  <type>GTK_WINDOW_TOPLEVEL</type>
  <position>GTK_WIN_POS_NONE</position>
  <allow_shrink>True</allow_shrink>
  <allow_grow>True</allow_grow>
  <auto_shrink>False</auto_shrink>

  <widget>
    <class>GtkFixed</class>
    <name>fixed1</name>

    <widget>
      <class>GtkButton</class>
      <name>button1</name>
      <x>32</x>
      <y>16</y>
      <width>112</width>
      <height>40</height>
      <can_focus>True</can_focus>
      <label>button1</label>
    </widget>
  </widget>
</widget>

Notice the heirarchy of widgets.  NOW THIS IS IMPORTANT: The heirarchy of
nodes in a subnet, plus the XML descriptions of their interfaces, can be used
to arrange widgets and construct a GUI!

This is all in the Loci archives where we discussed 'composited GUIs' and
'constructing the command-line'.  I'll certainly be talking more about this
later, as it is a major design feature of Loci.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 22:19:43 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] gui protocol revisited
References: <38D94DB0.3353F6C3@geoserve.net> <38D9C876.CC7B55CD@casema.net>
Message-ID: <38DADECF.CEA3800C@geoserve.net>

jarl van katwijk wrote:
> 
> Looks nice, one thing: you're talking about 'requests', so will the GUI (and
> optional other
> 'controllers' like scripts or knowledge databases) have (1) the 'structure'
> stored internally
> or will they (2) 'build' it inside gms\overflow?
> 
> GMS uses (2), I think Loci does too, but just to make sure..

I suppose by 'structure' you mean the structure of the network/subnet.  Yes,
the core will handle it.  I consider the interfaces/fronts to be completely
'dumb', acting only on the requests of user and core.

> The GUI just calls a LOCAL function like:
>     GUI2CORE_add_node()
> 
> which will be transported (among other things) by corba to the core and call:
>     IMPLEMENTATED_GUI2CORE_add_node()
> 
> which is a function located in the core.

Okay, but I guess Brad wants to keep GUI-to-core communication as a
datastream.  Brad, will you be attaching the Loci middle to this GMS API?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 22:24:39 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003230149.UAA115166@archa11.cc.uga.edu> <38D9CAA1.388FFB50@casema.net>
Message-ID: <38DADFF7.751299DC@geoserve.net>

jarl van katwijk wrote:
> 
> A kinda 'script interpeter'.. very good idear

I'm glad you think so.  I was also thinking about a 'front' that records this
stream to make a log file of all transactions.  Cool?  This can be used as an
'electronic notebook' when the system helps perform scientific research.

> >     Does this make any sense? I think this gives us a lot of reuse of
> > code, while maintaing the streaming XML that Jeff digs and the
> > CORBA/API interfaces of Overflow and GMS.
> 
> OK to me. It would also be a very good point for you to start working on.

I agree.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 22:30:53 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Overview
References: <200003230215.VAA146024@archa12.cc.uga.edu> <38D9CEDD.6BCF9EC0@casema.net>
Message-ID: <38DAE16D.FC5BC1DD@geoserve.net>

jarl van katwijk wrote:
> 
> C is more suitable for the 'inbetween' functionality gms will get in the
> VSH design. It has more 'hackish' functionality.
> But i'm also quite happy about Overflow & GUI being OOP because it makes
> sence to build those OOP.

I'm happy about the Overflow code staying C++.  It's a great choice being OO
and binary executable.  Python/Gnome is probably the best choice for the GUI
too.

> OK, then I will communicate with Brad about the details.. any dessision
> that will infuence design etc will be posted pubically ofcourse.

Keep as much on the list as possible :-)

> K, maybe it's logical that you'll do this 'in-between' code that talks to
> the gui and the core?

Yep.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 22:50:48 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Project Design (fwd)
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu>
Message-ID: <38DAE618.B900F9CC@geoserve.net>

Brad Chapman wrote:
> 
> SO Gms will be adjusted to 'speak' to overflow, overflow stays the same
> as it is, only restricted to local
> operation. And the Loci GUI will stay the same.
> 
> Every body happy?

I don't know if this is relevant to this thread, but I would like the GUI to
have Overflow instances contained in separate nodes.  So, Overflow networks
are VSH subnets.  And, Overflow 'workspaces' are displayed in VSH
'windowlets'.  This really fits in with the 'Overflow networks are local'
idea, and it's what I was thinking of when I drew up those flowcharts.

> Currently gms processes one data-event at a time, so a data-chunk
> passed the whole structure before the next is prosessed. This will
> have to change to better 'all at once'  processing.

Just so that Loci gets its 2 cents in on this design discussion, I was
thinking Loci would use a 'processor centric' approach to the problem.  That
is, whatever processes or modifies data in a network is first to be called. 
This isn't any different from the 'pull' approach when you think about it, but
there are some other considerations that should be given to this special class
of nodes.  Afterall, we're dealing with data, and the modification of those
data affects the whole system.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 22:56:12 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Project Design
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA81EF.8645B595@casema.net>
Message-ID: <38DAE75C.4A071918@geoserve.net>

jarl van katwijk wrote:
> 
> Could somebody collect all the design proposals and wishes and stuff and put
> them on
> the VSH page. Sometimes I'm saying\reading things twice :)

Alright, but in what form?  As e-mail or a little more polished?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 23:07:22 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] 3/1 and 1/3
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA7ED9.9D3F46A1@hermes.usherb.ca> <38DA94D7.E40E84B4@casema.net>
Message-ID: <38DAE9FA.9919576C@geoserve.net>

jarl van katwijk wrote:
> 
> Example: It's absolutely no problem to me that Overflow can connect to
> non-local subnets\nodes, but for the collaboration's sake Overflow should
> have support to disable this.
> Ex2: GMS will continue to support it's own GUI, I like it for testing.

> > My only constraint is that I want to be able to use Overflow as a stand alone
> > application.

At the very LEAST, a collaboration should mean that the respective projects
(Loci/GMS/Oveflow) don't continue development in such a way as to incorporate
incompatibilities with VSH.  If you're not excited about VSH right now, then
fine, maybe you will be later.  But you need to...

  (1) Tell us what features you'll be adding to your program
  (2) Consider whether or not features will work in the VSH system
  (3) Be prepared to change your program to incorporate a VSH feature

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 23:15:29 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] gui protocol revisited
References: <38D94DB0.3353F6C3@geoserve.net> <38D9C876.CC7B55CD@casema.net> <38DADECF.CEA3800C@geoserve.net>
Message-ID: <38DAEBE1.38933BFB@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> I consider the interfaces/fronts to be completely
> 'dumb', acting only on the requests of user and core.

This is how it works:

    USER:  Hey VSH, do this for me!
    GUI:   Duh...Hey Core, can I do this?
    CORE:  Yeah, go ahead Gui.
    GUI:   Duh...Okee dokee.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Thu Mar 23 23:24:23 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] 3/1 and 1/3
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA7ED9.9D3F46A1@hermes.usherb.ca> <38DA94D7.E40E84B4@casema.net> <38DAE9FA.9919576C@geoserve.net>
Message-ID: <38DAEDF7.3FD2E7A0@geoserve.net>

"J.W. Bizzaro" wrote:
> 
>   (1) Tell us what features you'll be adding to your program
>   (2) Consider whether or not features will work in the VSH system
>   (3) Be prepared to change your program to incorporate a VSH feature

Of course, you can always fork your project and create two: one for VSH and
one for your own experimental purposes.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Fri Mar 24 00:22:59 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] 3/1 and 1/3
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA7ED9.9D3F46A1@hermes.usherb.ca>
 <38DA94D7.E40E84B4@casema.net> <38DAE9FA.9919576C@geoserve.net>
Message-ID: <38DAFBB3.FB7B40@hermes.usherb.ca>

> At the very LEAST, a collaboration should mean that the respective projects
> (Loci/GMS/Oveflow) don't continue development in such a way as to incorporate
> incompatibilities with VSH.  If you're not excited about VSH right now, then
> fine, maybe you will be later.  But you need to...
> 
>   (1) Tell us what features you'll be adding to your program
>   (2) Consider whether or not features will work in the VSH system
>   (3) Be prepared to change your program to incorporate a VSH feature

That's no problem... Overflow is built in a very modular way. All the
functionality is built into nodes (C++ classes). If you don't like some of them,
you just don't use them, but they don't interfere. If you "rm" a library you
don't want, things will still work (without recompiling). The same for the
Overflow types (like neural nets, vector quantizers, ...). As for supporting
VSH, every new feature will simply be a new node in Overflow.


About the architecture... what about this: Everything that's done locally
(including running other exceutables) is handled by Overflow and all networking
stuff would be handled by GMS, which would take care of connecting all Overflow
processes (here I'm talking about "process" in a weak sense) through a network.
This way there wouldn't be any duplication. Also, every node-to-node connection
in the GMS part of the GUI would be assumed to be "network-able".

Concerning the GUI, I don't think it would be hard to plug in what we've got so
far in another app. Or we can just embed the current GUI in the new app using
bonobo. I don't know enough about this to say which is best.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jarl at casema.net  Fri Mar 24 02:12:55 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003230149.UAA115166@archa11.cc.uga.edu> <38DACD96.8D0900F3@geoserve.net>
Message-ID: <38DB1577.481B6BB4@casema.net>

> > communiate with the GUI front via the streaming XML dialog you
> > describe.
>
> Okay, if you like that.  And I guess Jarl likes it too.

Yes. sure is.

>
> I'm a little confused about this 'processing part of the program'.  Is this
> something GMS or Overflow has now?  Is it something you'll have to write?

After some time when the 'basics' work, non-technical users will be excluded
from using VHS, so there must be some meganism to automatie the creation of
data flow.
1) applications can supply a VSH 'script' that intergrate that application to
VSH,
2) to analyze 'good' structures and apply them to other instances of VSH.

1) is easy, but 2) requires an anaylize 'core' (CFS\GAP on the the GMS page)
Both are far-future idears though :)




From jarl at casema.net  Fri Mar 24 02:17:12 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Overview
References: <200003230215.VAA146024@archa12.cc.uga.edu> <38DAD631.CE01714C@geoserve.net>
Message-ID: <38DB1678.BE533885@casema.net>

> The others probably don't understand what we mean by 'XML representation'.

The dataflow structure?




From jarl at casema.net  Fri Mar 24 02:20:15 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net> <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net> <38DADC83.F000D7B@geoserve.net>
Message-ID: <38DB172F.E77443B6@casema.net>

> > Hmm? Please explain, I can't pinpoint what you're trying to say.
>
> Notice the heirarchy of widgets.  NOW THIS IS IMPORTANT: The heirarchy of
> nodes in a subnet, plus the XML descriptions of their interfaces, can be used
> to arrange widgets and construct a GUI!
>
> This is all in the Loci archives where we discussed 'composited GUIs' and
> 'constructing the command-line'.  I'll certainly be talking more about this
> later, as it is a major design feature of Loci.
>

OK, got it.



From jarl at casema.net  Fri Mar 24 02:25:43 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Project Design
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA81EF.8645B595@casema.net> <38DAE75C.4A071918@geoserve.net>
Message-ID: <38DB1877.A240EDF5@casema.net>

> >
> > Could somebody collect all the design proposals and wishes and stuff and put
> > them on
> > the VSH page. Sometimes I'm saying\reading things twice :)
>
> Alright, but in what form?  As e-mail or a little more polished?
>

just copy-paste the emails, but organised by subject.
More polished would be nice, if somebody got the time :)



From jarl at casema.net  Fri Mar 24 02:38:47 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] 3/1 and 1/3
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA7ED9.9D3F46A1@hermes.usherb.ca> <38DA94D7.E40E84B4@casema.net> <38DAE9FA.9919576C@geoserve.net>
Message-ID: <38DB1B87.BDF127D5@casema.net>

> At the very LEAST, a collaboration should mean that the respective projects
> (Loci/GMS/Oveflow) don't continue development in such a way as to incorporate
> incompatibilities with VSH.  If you're not excited about VSH right now, then
> fine, maybe you will be later.  But you need to...
>
>   (1) Tell us what features you'll be adding to your program
>   (2) Consider whether or not features will work in the VSH system
>   (3) Be prepared to change your program to incorporate a VSH feature
>

OK to me.
It seems gms will get the most changes, but that's no problem.
I'm only saying the coming time the VSH project will contains 3 types of code:
1) general code, 2) collaborate code & 3) standalone code
I think these dont need any further explanation.

bye,
jarl



From jarl at casema.net  Fri Mar 24 02:40:15 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] gui protocol revisited
References: <38D94DB0.3353F6C3@geoserve.net> <38D9C876.CC7B55CD@casema.net> <38DADECF.CEA3800C@geoserve.net> <38DAEBE1.38933BFB@geoserve.net>
Message-ID: <38DB1BDF.593E393F@casema.net>

>
> This is how it works:
>
>     USER:  Hey VSH, do this for me!
>     GUI:   Duh...Hey Core, can I do this?
>     CORE:  Yeah, go ahead Gui.
>     GUI:   Duh...Okee dokee.
>

hehehe :)
Jean: how ready is 'your' speech recognizer?

bye,
jarl



From jean-marc.valin at hermes.usherb.ca  Fri Mar 24 02:28:53 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net>
Message-ID: <38DB1935.E2A18780@hermes.usherb.ca>

> > Notice the heirarchy of widgets.  NOW THIS IS IMPORTANT: The heirarchy of
> > nodes in a subnet, plus the XML descriptions of their interfaces, can be used
> > to arrange widgets and construct a GUI!
> >
> > This is all in the Loci archives where we discussed 'composited GUIs' and
> > 'constructing the command-line'.  I'll certainly be talking more about this
> > later, as it is a major design feature of Loci.

Am I getting everything wrong or could this XML representation of a GUI be
handled by libglade? After all glade stores the GUI in XML and libglade can
display it in an application.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From chapmanb at arches.uga.edu  Fri Mar 24 09:37:44 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] gui protocol revisited
Message-ID: <200003241440.JAA131438@archa10.cc.uga.edu>

Jeff wrote:
> Okay, but I guess Brad wants to keep GUI-to-core communication as a
> datastream.  Brad, will you be attaching the Loci middle to this GMS 
API?

    Well, I wanted to attach it to the idl I sent out yesterday. I was 
hoping that we could work on this and make it a standard general idl 
for communicating between the gui (which we have implemented as the 
replacable front gui and the small middle communicating via xml data 
streams) and the processing engines. Of course, the idl is not very 
good and is just what I randomly thought up, but I was hoping we could 
work from that to create a more general idl which is based on the 
starting point I sent out.

Jeff wrote:
> I'm a little confused about this 'processing part of the program'.  
Is this
> something GMS or Overflow has now?  Is it something you'll have to 
write? 
   
     I'm not really very clear right now how GMS and Overflow will 
combine and work together (I'll need to dig more into the code to know 
this for sure) to make the generalize "processing engine" that I have 
been talking about, which is why I've been referring to it so vaguely. 
As I said, I'd like this combination of GMS and Overflow to wrap into 
one idl (the one I sent out).
    Along these lines, I'd like to try and dig more into Overflow and 
GMS code and start working with it. This will probably help us firm up 
the communication between the 'middle scripting engine' and the 
'processing part' and help get the idl straightened out and all. So... 
I was going to throw myself out to the wolves and ask if Jarl and 
Jean-Marc and Dominic have parts of GMS and Overflow that they'd like 
me to work on/with, to help towards our integration goal. These can be 
small short term projects or whatever, but I'd like to help in this 
regard, and it would help me be more familiar with what is going on 
and actually be able to make more non-vague suggestions :-) 
    So could we work something like this out?

> How does this relate to the
> 'processing' part?  Perhaps you can explain this to the others and 
how we plan
> to use XML.  Will this be replaced by a DOM structure?

Well, in the idl I'm passing the XML representation (more on this 
below) as a string right now. I'd like this to be passed as DOM 
eventually, but 4Dom doesn't support this right now (supposedly it is 
coming soon) so it isn't really an option right now.
  
Jeff wrote:
> > The others probably don't understand what we mean by 'XML 
representation'.

Jarl wrote:
> The dataflow structure?

Right-o! When I say xml representation I'm just again defining a vague 
term to represent the way that nodes/loci are linked together in XML. 
The form of this needs to be worked out explicitly so that the 
'processing part' can take this xml and convert it into a form that is 
useful for it to go through and do things. When we can hash out 
this representation, then I can change the middle scripting part so 
that it delivers this XML in the right format.
  
    I think we sort of have a design down, although at least in my 
thinking, the XML representation, the makeup of the processing engine, 
and the 'middle scripting engine' and 'processing part' idls are the 
most important things we need to tackle in terms of internal structure.
    
Brad



From jarl at casema.net  Fri Mar 24 10:12:48 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:36 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003241440.JAA131438@archa10.cc.uga.edu>
Message-ID: <38DB85F0.3D3BDE0@casema.net>

>
>     Well, I wanted to attach it to the idl I sent out yesterday.

Yes, I will work on the IDL this weekend, we'll can have a pretty good
one done in a few days. Implementing the idl will take some longer.

>
>      I'm not really very clear right now how GMS and Overflow will
> combine and work together (I'll need to dig more into the code to know
> this for sure)

Overflow will work as it does standalone, and gms will pass on the calls
from and to the gui.

> As I said, I'd like this combination of GMS and Overflow to wrap into
> one idl (the one I sent out).

Yes, right.

>     Along these lines, I'd like to try and dig more into Overflow and
> GMS code and start working with it. This will probably help us firm up
> the communication between the 'middle scripting engine' and the
> 'processing part' and help get the idl straightened out and all. So...
> I was going to throw myself out to the wolves and ask if Jarl and
> Jean-Marc and Dominic have parts of GMS and Overflow that they'd like
> me to work on/with, to help towards our integration goal. These can be
> small short term projects or whatever, but I'd like to help in this
> regard, and it would help me be more familiar with what is going on
> and actually be able to make more non-vague suggestions :-)
>     So could we work something like this out?

Sure, but first let us define that GUI <-> Engine idl. When we got that
ready, things will be much more clear I guess.

>     I think we sort of have a design down, although at least in my
> thinking, the XML representation, the makeup of the processing engine,
> and the 'middle scripting engine' and 'processing part' idls are the
> most important things we need to tackle in terms of internal structure.
>

Yes, the overflow people agreed to this design too, so let's make The idl
and make a list of features that the 3 projects need to implement in order

to let the layers work together.

bye,
jarl



From jarl at casema.net  Fri Mar 24 14:33:21 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
Message-ID: <38DBC301.819DF35A@casema.net>

Jean-Marc Valin wrote:

> > > Notice the heirarchy of widgets.  NOW THIS IS IMPORTANT: The heirarchy of
> > > nodes in a subnet, plus the XML descriptions of their interfaces, can be used
> > > to arrange widgets and construct a GUI!
> > >
> > > This is all in the Loci archives where we discussed 'composited GUIs' and
> > > 'constructing the command-line'.  I'll certainly be talking more about this
> > > later, as it is a major design feature of Loci.
>
> Am I getting everything wrong or could this XML representation of a GUI be
> handled by libglade? After all glade stores the GUI in XML and libglade can
> display it in an application.
>

This would requiere translation of the XML code, Glade knows about Gtk+ and
GNOME widgets, not about nodes and relations.
But I think basically both use the same approach.



From jean-marc.valin at hermes.usherb.ca  Fri Mar 24 14:17:19 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] GMS- Overflow communication
Message-ID: <38DBBF3F.92D21212@hermes.usherb.ca>

Hi,

I've been thinking a bit about how GMS and Overflow should communicate. Here's a
proposal.

GMS links with the Overflow libraries. 
When it needs to compute something, it calls getOutput() on an Overflow subnet
*in a thread* and returns the result.
Meanwhile, the input node(s) of the Overflow network use GMS callbacks to get
their data.

I think this would be both the simplest (no socket/pipe/shared memory required)
and the fastest (though speed doesn't matter much here) to do things. 

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Fri Mar 24 16:06:07 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] 3/1 and 1/3
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA7ED9.9D3F46A1@hermes.usherb.ca>
	 <38DA94D7.E40E84B4@casema.net> <38DAE9FA.9919576C@geoserve.net> <38DAFBB3.FB7B40@hermes.usherb.ca>
Message-ID: <38DBD8BF.10AC39FA@geoserve.net>

Jean-Marc Valin wrote:
> 
> That's no problem... Overflow is built in a very modular way. All the
> functionality is built into nodes (C++ classes). If you don't like some of them,
> you just don't use them, but they don't interfere. If you "rm" a library you
> don't want, things will still work (without recompiling). The same for the
> Overflow types (like neural nets, vector quantizers, ...). As for supporting
> VSH, every new feature will simply be a new node in Overflow.

Very nice.

> About the architecture... what about this: Everything that's done locally
> (including running other exceutables) is handled by Overflow and all networking
> stuff would be handled by GMS, which would take care of connecting all Overflow
> processes (here I'm talking about "process" in a weak sense) through a network.
> This way there wouldn't be any duplication. Also, every node-to-node connection
> in the GMS part of the GUI would be assumed to be "network-able".

This is EXACTLY what I was thinking of.  Brad and Jarl, is this not what you
guys were thinking?

> Concerning the GUI, I don't think it would be hard to plug in what we've got so
> far in another app. Or we can just embed the current GUI in the new app using
> bonobo. I don't know enough about this to say which is best.

I don't dislike Overflow's GUI.  I just believe GMS and Overflow should have
GUIs that work alike.  Plus, we've come up with some very nice GUI features
for Loci that I'd like to see VSH have.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 16:11:43 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003230149.UAA115166@archa11.cc.uga.edu> <38DACD96.8D0900F3@geoserve.net> <38DB1577.481B6BB4@casema.net>
Message-ID: <38DBDA0F.19CA34CC@geoserve.net>

jarl van katwijk wrote:
> 
> 1) applications can supply a VSH 'script' that intergrate that application to
> VSH,

You're talking about wrapping or plugging-in a foreign application, correct? 
What do you think about having Overflow be the 'wrapping environment'?  By
this I mean the wrapping 'script' is an Overflow network.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 16:15:27 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Overview
References: <200003230215.VAA146024@archa12.cc.uga.edu> <38DAD631.CE01714C@geoserve.net> <38DB1678.BE533885@casema.net>
Message-ID: <38DBDAEE.BFD99F1C@geoserve.net>

jarl van katwijk wrote:
> 
> > The others probably don't understand what we mean by 'XML representation'.
> 
> The dataflow structure?

No, all those little details that need to be stored about a node: location,
type, mime-types, libraries, input parameters, what other nodes are connected
to this node.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 16:34:07 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
Message-ID: <38DBDF4F.EC52662D@geoserve.net>

Jean-Marc Valin wrote:
> 
> Am I getting everything wrong or could this XML representation of a GUI be
> handled by libglade? After all glade stores the GUI in XML and libglade can
> display it in an application.

You're getting pretty darn close :-)  The only difference between libglade (or
pyglade, both written by the same guy) and what we'll use, is that the *glades
specify a gtk interface.  But, since we want to have other kinds of
interfaces, we'll make something (by changing pyglade) more generic.

So, instead of a tag like this:

    <widget>
        <GtkRadioButton.../>
    </widget>

we'll have something like this:

    <parameter>
        <selection.../>
    </parameter>

which can then be 'translated' into gtk, qt, html, etc., depending on the
front-end used with VSH.

But keep in mind that this information needs to be transferred across the
Internet in our Internet-distributed VSH system.  This is part of what Brad
and I call an 'XML representation'.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 16:44:01 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003241440.JAA131438@archa10.cc.uga.edu>
Message-ID: <38DBE1A1.1FA2E35B@geoserve.net>

Brad Chapman wrote:
> 
> Jeff wrote:
> > > The others probably don't understand what we mean by 'XML
> representation'.
> 
> Jarl wrote:
> > The dataflow structure?
> 
> Right-o! When I say xml representation I'm just again defining a vague
> term to represent the way that nodes/loci are linked together in XML.

Okay, Brad is talking about an 'XML representation' of the dataflow
structure.  I was referring to an 'XML representation' of a node.  The way we
designed Loci, the latter is a subset of the former.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 16:53:43 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Front to middle communication
References: <200003241448.JAA156374@archa10.cc.uga.edu>
Message-ID: <38DBE3E7.6DE03B9C@geoserve.net>

Brad Chapman wrote:
> 
> Wow! Did you really write this? CORBA everywhere, yay! :-)

:-)  Well, with Loci, we didn't have CORBA anywhere.  So, keeping it all
CONSISTENT with sockets was a worthy cause.  My preference is for consistency
(and simplicity) rather than for sockets.

>     I think if we want to consider CORBA communication between the
> front GUI and the middle the most important thing is to consider how
> easy it will be for other people to implement front ends. Will CORBA
> be more difficult than sockets?

Ah, an ancient riddle, my young Locian.  Perhaps the spirit of the Loci will
show you the way ;-)

>     We can define a small CORBA interface that will mirror what we
> have now with sockets, and this could provide us some additional
> flexibility in terms of passing data (and might help deal with the
> problem I mentioned earlier about how to pass large amount of data
> from databases or files to the front).

I think it's something to work for, providing we can still have multiple
fronts control the middle and each other.

>     I don't really want to make this change right yet unless we have a
> big problem we can't overcome with since sockets are working and we'll
> have millions of other changes to make to fit in with the 'vsh'
> project collaboration.
>     What do you all think?

Why don't we put it on the back burner for now.  You're correct that it's
something we can do later.  The merger is more important right now.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 17:11:15 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Moving forward in the middle
References: <200003241507.KAA102964@archa11.cc.uga.edu>
Message-ID: <38DBE803.E66D87E0@geoserve.net>

Brad Chapman wrote:
> 
>     As I mentioned, I've pulled myself out of the GUI comletely so
> that I won't be interfering there any more :-) As a result this has
> led me to start thinking about how to implement stuff xml-wise and not
> gui-wise

Great.  Oh, the wonders of abstraction!

>     Specifically what I'm blathering about is that I've just about
> (keeping my fingers crossed here!) got a generalized loci
> disconnection communication protocol finished (and partly implemented
> in the GUI, but only during deletion of loci, and I have not thought
> about providing a way for generalized disconnecting of loci). This
> weekend I'm hoping to tackle persistance/saving of loci in the
> back/private directory and, if I am really lucky, start thinking about
> how to try to whip up a connection diagram into a good XML
> representation (of course, we have to work the basis of this out with
> the other vsh guys).

Right.  How does Overflow 'store' its network?

> Do you have ideas about how these things should
> work in the GUI? I'll need the GUI to test things out and make sure
> I'm implementing them properly, and just thought we could try and
> coordinate stuff and get this all working beautifully as soon as we
> can.
>     What do you think? What are your plans?

Sure, I'll be here :-)

As for the way the GUI will handle locus/node disconnection, it will be done
through a popup menu on a button3 click.  The popup menu will actually be
different for each of the 4 main GUI objects:

  (1) Workspace
  (2) Locus/node
  (3) Connector
  (4) Windowlet

Clicking button3 on a connector will bring up a menu for that connector:

    +---------------+
    | Connector  >  +---------------+
    | Session    >  | Details...    |
    | Help       >  | Add         > |
    +---------------| Disconnect    |
                    +---------------+

(Note that it doesn't work this way now, but this is my plan.)

And choosing 'Disconnect' will....disconnect the connector.  The dot color
will turn from green to yellow, during the wait for approval, and then to red,
showing that it is disconnected.

What else do you need to know?  Do you want me to get this working this
weekend?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 17:19:48 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
		 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
		 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca> <38DBC301.819DF35A@casema.net>
Message-ID: <38DBEA04.C5CBD3BD@geoserve.net>

jarl van katwijk wrote:
> 
> > Am I getting everything wrong or could this XML representation of a GUI be
> > handled by libglade? After all glade stores the GUI in XML and libglade can
> > display it in an application.
> >
> 
> This would requiere translation of the XML code, Glade knows about Gtk+ and
> GNOME widgets, not about nodes and relations.
> But I think basically both use the same approach.

:-) Again, there is some confusion here.  Jean-Marc and I are talking about
the representation of a node's interface in XML.  You (and Brad) have been
using the term 'XML representation' to mean the dataflow structure of a
network.

We should probably add 'of a node' or 'of a network' to this term to avoid
confusion :-)

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Fri Mar 24 19:09:49 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003230149.UAA115166@archa11.cc.uga.edu> <38DACD96.8D0900F3@geoserve.net> <38DB1577.481B6BB4@casema.net> <38DBDA0F.19CA34CC@geoserve.net>
Message-ID: <38DC03CC.3A5B5CEF@casema.net>

> >
> > 1) applications can supply a VSH 'script' that intergrate that application to
> > VSH,
>
> You're talking about wrapping or plugging-in a foreign application, correct?

Yes. See also the GConf project.
It's about 'programs registring into gnome', so much relavant info is availeble
for wrapping an application. Less need to build code for every single application,

maybe we could write a general one once we use GConf.

Example: A email applet could simply register itself as a handler for the smtp
and pop mimetypes. GMS will collect this data from the gconf database, and
export the needed structure to activate this dataflow inside Overflow.


> What do you think about having Overflow be the 'wrapping environment'?  By
> this I mean the wrapping 'script' is an Overflow network.
>

I think that we have the same design in mind.




From jean-marc.valin at hermes.usherb.ca  Fri Mar 24 19:41:22 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net>
Message-ID: <38DC0B32.F42D6405@hermes.usherb.ca>

> > > Am I getting everything wrong or could this XML representation of a GUI be
> > > handled by libglade? After all glade stores the GUI in XML and libglade can
> > > display it in an application.
> > >
> >
> > This would requiere translation of the XML code, Glade knows about Gtk+ and
> > GNOME widgets, not about nodes and relations.
> > But I think basically both use the same approach.
> 
> :-) Again, there is some confusion here.  Jean-Marc and I are talking about
> the representation of a node's interface in XML.  You (and Brad) have been
> using the term 'XML representation' to mean the dataflow structure of a
> network.
> 
> We should probably add 'of a node' or 'of a network' to this term to avoid
> confusion :-)

OK, I'll try to summerize the way XML is used in Overflow with an example:


<?xml version="1.0"?>
<Document>

A document is a file and can contain many networks (it's the equivalent of a .c)

  <Network type="subnet" name="MAIN">

A network if the C equivalent of a function. Each document must contain a MAIN
network.

    <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
      <Parameter name="VALUE" type="subnet_param" value="IN"/>
    </Node>

This is a node definition. The name is arbitrary, while the type corresponds to
an Overflow node class. x and y are for the GUI. The parameters have to
correspond to what the node expects.

    <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
      <Parameter name="FRAME_SIZE" type="int" value="256"/>
    </Node>
    <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
      <Parameter name="COMMAND" type="string" value="cat"/>
    </Node>
    <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
      <Parameter name="VALUE" type="string" value="frames"/>
    </Node>
    <Node name="node8" type="OutputStream" x="43.000000" y="190.000000"/>
    <Node name="node9" type="Save" x="243.000000" y="119.000000"/>
    <Link from="node2" output="OUTPUT" to="node5" input="INPUT"/>
    <Link from="node1" output="VALUE" to="node2" input="INPUT"/>
    <Link from="node6" output="VALUE" to="node8" input="INPUT"/>
    <Link from="node5" output="TRAIN_IN" to="node9" input="OBJECT"/>
    <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>

Links define which node is connected to which. Input and output names need to be
defined since nodes can have more than one input/output.

    <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>

NetOutput means which node (and which output of that node) will be used as the
output of the whole network. The same applies to NetInput and NetCondition.

  </Network>
  <Network type="iterator" name="MAIN_LOOP">
    <Node name="node1" type="AudioStream" x="-526.000000" y="93.000000">
      <Parameter name="LENGTH" type="subnet_param" value="FRAME_SIZE"/>
      <Parameter name="ENCODING" type="string" value="LIN16"/>
      <Parameter name="STREAM_TYPE" type="string" value="FILE"/>
    </Node>
    <Node name="node2" type="NotDone" x="-344.000000" y="45.000000"/>
    <Node name="node10" type="FrameAccumulator" x="-336.000000" y="131.000000">
      <Parameter name="LENGTH" type="int" value="256"/>
      <Parameter name="CACHEALL" type="int" value="1"/>
    </Node>
    <Link from="node1" output="OUTPUT" to="node2" input="INPUT"/>
    <Link from="node1" output="OUTPUT" to="node10" input="INPUT"/>
    <NetInput name="INPUT" node="node1" terminal="INPUT"/>
    <NetCondition name="CONDITION" node="node2" terminal="OUTPUT"/>
    <NetOutput name="TRAIN_IN" node="node10" terminal="OUTPUT"/>
  </Network>
  <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>
</Document>

Questions?

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Fri Mar 24 22:37:20 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
	 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
Message-ID: <38DC3470.D68AF122@geoserve.net>

Jean-Marc Valin wrote:
> 
> OK, I'll try to summerize the way XML is used in Overflow with an example:
> 
> <?xml version="1.0"?>
> <Document>
> 
> A document is a file and can contain many networks (it's the equivalent of a .c)

Then this is something like Loci's 'workspace' or 'workflow diagram', or
Jarl's 'dataflow structure'.

>   <Network type="subnet" name="MAIN">
> 
> A network if the C equivalent of a function. Each document must contain a MAIN
> network.
> 
>     <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
>       <Parameter name="VALUE" type="subnet_param" value="IN"/>
>     </Node>
>
> This is a node definition. The name is arbitrary,

The name is akin to Loci's identification number, which is really a URI to the
XML representation of the node.

> while the type corresponds to an Overflow node class.

Great, this is just like Loci's locus types/classes.

> x and y are for the GUI.

I think for VSH we should have the GUI store that information.  If we are to
have more than one interface available for VSH, x and y will not always be
relevant.  This is what we were going to do with Loci.

>  The parameters have to correspond to what the node expects.

This is like a mime-type for link I/O?  This is also something Loci was going
to define.

>     <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
>       <Parameter name="FRAME_SIZE" type="int" value="256"/>
>     </Node>
>     <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
>       <Parameter name="COMMAND" type="string" value="cat"/>
>     </Node>
>     <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
>       <Parameter name="VALUE" type="string" value="frames"/>
>     </Node>

I'm not sure what you mean by value=256, cat, or frames.

>     <Node name="node8" type="OutputStream" x="43.000000" y="190.000000"/>
>     <Node name="node9" type="Save" x="243.000000" y="119.000000"/>
>     <Link from="node2" output="OUTPUT" to="node5" input="INPUT"/>
>     <Link from="node1" output="VALUE" to="node2" input="INPUT"/>
>     <Link from="node6" output="VALUE" to="node8" input="INPUT"/>
>     <Link from="node5" output="TRAIN_IN" to="node9" input="OBJECT"/>
>     <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>
> 
> Links define which node is connected to which. Input and output names need to be
> defined since nodes can have more than one input/output.

Excellent, just like Loci.  Although Loci embedded <link> (what Loci calls
<input> and <output> 'connectors') in with the node information.  This is how
Loci defines a node (loci == node, although 'locus' is singular):

  <loci type='document' >
    <widget>file_select</widget>
    <input xml:link='simple'  href='baselocus.xml' >
    </input>
    <output xml:link='simple'  href='workspace1/viewer1.xml' >
    </output>
  </loci>

(Brad, shouldn't we also have locus 'id=' in here?)

In this example, when we say 'type=', we mean the same thing you do with
'type='.  But our type 'document' is a node that stores data: a file.  It's
not where Loci sets up a network.  What Overflow calls 'document', Loci calls
'workspace'.

<widget> is the name of the interface component to be used.  We're working on
using more XML here, and the 'widget' representation in XML can be located
anywhere on the Internet.

<input> and <output> define the links (connectors) for this particular node. 
href points to the XML representation of the node it is connected to. 
xml:link is from an XML document linking tool that Brad has been using.

Loci also gives the node an ID (not shown above) that is the location/URI of
the node's XML representation.  Here the GUI of Loci requests that a node
called 'viewer1' gets linked to a node called 'document1':

  <connect>
    <input id='workspace1/viewer1.xml' >
    </input>
    <output id='workspace1/document1.xml' >
    </output>
  </connect>

Other information related to a node (e.g., library URI and version number) can
be kept between <loci> and </loci>

>     <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>
> 
> NetOutput means which node (and which output of that node) will be used as the
> output of the whole network. The same applies to NetInput and NetCondition.

What is the actual output?  The screen, a file?  Or is this something the user
defines?

>   </Network>
>   <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>

In this case, the whole 'document' gets a 'parameter' tag?

When I first came across Overflow (when you posted the announcement on
gnotices), I noted to Brad how remarkably similar some of its features are to
Loci's.  This is particularly true for the XML representation of a network.

This is something GMS (which uses no XML) needs to implement as well.  And
since Overflow and Loci are so similar, we should choose one definition.  Can
you comment on what you like or dislike about Loci's use of XML for network
representation?  Let's hammer this out right away.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Fri Mar 24 22:40:34 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003230149.UAA115166@archa11.cc.uga.edu> <38DACD96.8D0900F3@geoserve.net> <38DB1577.481B6BB4@casema.net> <38DBDA0F.19CA34CC@geoserve.net> <38DC03CC.3A5B5CEF@casema.net>
Message-ID: <38DC3532.47422EF0@geoserve.net>

jarl van katwijk wrote:
> 
> Yes. See also the GConf project.
> It's about 'programs registring into gnome', so much relavant info is availeble
> for wrapping an application. Less need to build code for every single application,
> 
> maybe we could write a general one once we use GConf.
> 
> Example: A email applet could simply register itself as a handler for the smtp
> and pop mimetypes. GMS will collect this data from the gconf database, and
> export the needed structure to activate this dataflow inside Overflow.

Are you saying VSH will need GConf installed to do wrapping?  Or are you
saying that we'll use something like GConf?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Fri Mar 24 23:20:57 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
 <38DC3470.D68AF122@geoserve.net>
Message-ID: <38DC3EA9.F84B3EDF@hermes.usherb.ca>

"J.W. Bizzaro" a ?crit :

> > OK, I'll try to summerize the way XML is used in Overflow with an example:
> >
> > <?xml version="1.0"?>
> > <Document>
> >
> > A document is a file and can contain many networks (it's the equivalent of a .c)
> 
> Then this is something like Loci's 'workspace' or 'workflow diagram', or
> Jarl's 'dataflow structure'.

Something I forgot to say about documents... if this document is in the
VFLOW_PATH, then the whole document can be used as a node in another document.

> 
> >   <Network type="subnet" name="MAIN">
> >
> > A network if the C equivalent of a function. Each document must contain a MAIN
> > network.

BTW, the network type can be three things: "subnet", "iterator" or
"threadedIterator". The "threadedIterator" is very experimental, so I will not
talk about it. A "subnet" is the C equivalent of a function, while the
"iterator" is the equivalent (not exact) of a loop. Actually an iterator is the
equivalent of a function that looks like

void foo(...)
{
   while(condition)
   {
      do something
   }
}

> >
> >     <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
> >       <Parameter name="VALUE" type="subnet_param" value="IN"/>
> >     </Node>
> >
> > This is a node definition. The name is arbitrary,
> 
> The name is akin to Loci's identification number, which is really a URI to the
> XML representation of the node.
> 
> > while the type corresponds to an Overflow node class.
> 
> Great, this is just like Loci's locus types/classes.
> 

Just to make sure everyone understand, the definition is the equivalent of a
function call, not a function prototype/implementation.

> > x and y are for the GUI.
> 
> I think for VSH we should have the GUI store that information.  If we are to
> have more than one interface available for VSH, x and y will not always be
> relevant.  This is what we were going to do with Loci.
> 

I didn't like that either. It was just simpler to have everything in one file
(since this is the only info that was GUI-dependent)

> >  The parameters have to correspond to what the node expects.
> 
> This is like a mime-type for link I/O?  This is also something Loci was going
> to define.

I'm not sure what you're talking about...

> 
> >     <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
> >       <Parameter name="FRAME_SIZE" type="int" value="256"/>
> >     </Node>
> >     <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
> >       <Parameter name="COMMAND" type="string" value="cat"/>
> >     </Node>
> >     <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
> >       <Parameter name="VALUE" type="string" value="frames"/>
> >     </Node>
> 
> I'm not sure what you mean by value=256, cat, or frames.
> 

Here's the C equivalent:

void foo(int aNumber);

main()
{
   foo(256);
}

This would translate to:

<Node name="node1" type="foo">
   <Parameter name="aNumber" type="int" value="256"/>
</Node>

Also about parameters... Here is a macro that appears on top of each file that
implements a node:

NODE_INFO(FIR,"Signal:DSP", "INPUT:FILTER", "OUTPUT",
"LENGTH:CONTINUOUS:NONCAUSAL")

Here, FIR is the name of the class (the node's "type"), "Signal:DSP" is just a
classification for the menus, "INPUT" and "FILTER" are the names of the inputs
and "OUTPUT" is the name of the output. This node accepts 3 parameters
(arguments): LENGTH, CONTINUOUS and NONCAUSAL.

When the node is loaded (its library is dynamically linked), a function puts
that info in a node "database", which the GUI uses to know what node are
available (and takes what arguments).

> >     <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>
> >
> > Links define which node is connected to which. Input and output names need to be
> > defined since nodes can have more than one input/output.
> 
> Excellent, just like Loci.  Although Loci embedded <link> (what Loci calls
> <input> and <output> 'connectors') in with the node information.  This is how
> Loci defines a node (loci == node, although 'locus' is singular):
> 
>   <loci type='document' >
>     <widget>file_select</widget>
>     <input xml:link='simple'  href='baselocus.xml' >
>     </input>
>     <output xml:link='simple'  href='workspace1/viewer1.xml' >
>     </output>
>   </loci>

We used to have links as a property of the nodes (before we switched to XML).
The reason we changed is that is that a Link in the GUI is an actual object, so
it was just simpler.

> (Brad, shouldn't we also have locus 'id=' in here?)
> 
> In this example, when we say 'type=', we mean the same thing you do with
> 'type='.  But our type 'document' is a node that stores data: a file.  It's
> not where Loci sets up a network.  What Overflow calls 'document', Loci calls
> 'workspace'.
> 
> <widget> is the name of the interface component to be used.  We're working on
> using more XML here, and the 'widget' representation in XML can be located
> anywhere on the Internet.
> 
> <input> and <output> define the links (connectors) for this particular node.
> href points to the XML representation of the node it is connected to.
> xml:link is from an XML document linking tool that Brad has been using.
> 
> Loci also gives the node an ID (not shown above) that is the location/URI of
> the node's XML representation.  Here the GUI of Loci requests that a node
> called 'viewer1' gets linked to a node called 'document1':
> 
>   <connect>
>     <input id='workspace1/viewer1.xml' >
>     </input>
>     <output id='workspace1/document1.xml' >
>     </output>
>   </connect>
> 
> Other information related to a node (e.g., library URI and version number) can
> be kept between <loci> and </loci>
> 
> >     <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>
> >
> > NetOutput means which node (and which output of that node) will be used as the
> > output of the whole network. The same applies to NetInput and NetCondition.
> 
> What is the actual output?  The screen, a file?  Or is this something the user
> defines?

The output is what getOutput() returns... you can do whatever you like with it.
batchflow (text mode tool) writes it to stdout, vflow (the GUI tool) writes it
in a subwindow. If you want to save it in a file, all you need to do is to have
the last node be a "Save" node. ...or the last node can be a soundcard output...
In most cases, the output of the MAIN network is a nil object.

> 
> >   </Network>
> >   <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>
> 
> In this case, the whole 'document' gets a 'parameter' tag?
> 
> When I first came across Overflow (when you posted the announcement on
> gnotices), I noted to Brad how remarkably similar some of its features are to
> Loci's.  This is particularly true for the XML representation of a network.
> 
> This is something GMS (which uses no XML) needs to implement as well.  And
> since Overflow and Loci are so similar, we should choose one definition.  Can
> you comment on what you like or dislike about Loci's use of XML for network
> representation?  Let's hammer this out right away.

I'm not that much a fan of XML... The only reason I used it was that it saved me
so much time (instead of writing a parser myself). So Anything that does the job
is OK for me.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca



From bizzaro at geoserve.net  Fri Mar 24 23:50:14 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
	 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
	 <38DC3470.D68AF122@geoserve.net> <38DC3EA9.F84B3EDF@hermes.usherb.ca>
Message-ID: <38DC4586.178AE513@geoserve.net>

Jean-Marc Valin wrote:
> 
> Something I forgot to say about documents... if this document is in the
> VFLOW_PATH, then the whole document can be used as a node in another document.

Right.  Loci can embed workspaces in workspaces too.

> BTW, the network type can be three things: "subnet", "iterator" or
> "threadedIterator". The "threadedIterator" is very experimental, so I will not
> talk about it. A "subnet" is the C equivalent of a function, while the
> "iterator" is the equivalent (not exact) of a loop. Actually an iterator is the
> equivalent of a function that looks like
> 
> void foo(...)
> {
>    while(condition)
>    {
>       do something
>    }
> }

Understood.

> > Great, this is just like Loci's locus types/classes.
> >
> 
> Just to make sure everyone understand, the definition is the equivalent of a
> function call, not a function prototype/implementation.

Okay, I was showing you some actual locus implementation in XML.  Loci doesn't
keep workspace information in a single flat file like Overflow does.  Loci
(right now) uses the local filesystem to help organize the structure.  I'll
let you, Jarl and Brad work out how VSH will do it.

> > This is like a mime-type for link I/O?  This is also something Loci was going
> > to define.
> 
> I'm not sure what you're talking about...

I mean, when Overflow defines a link, does it also define the type of data
that is passed through the link?

> > I'm not sure what you mean by value=256, cat, or frames.
> >
> 
> Here's the C equivalent:
> 
> void foo(int aNumber);
> 
> main()
> {
>    foo(256);
> }
> 
> This would translate to:
> 
> <Node name="node1" type="foo">
>    <Parameter name="aNumber" type="int" value="256"/>
> </Node>

Gotcha, these are akin to function parameters in C.

> When the node is loaded (its library is dynamically linked), a function puts
> that info in a node "database", which the GUI uses to know what node are
> available (and takes what arguments).

How are you making this 'database'?  Is it an external db like mySQL or your
own creation?

> We used to have links as a property of the nodes (before we switched to XML).
> The reason we changed is that is that a Link in the GUI is an actual object, so
> it was just simpler.

That's a good point.  Loci also treats connectors as objects.  But they are
always a subclass of loci (nodes).

> > What is the actual output?  The screen, a file?  Or is this something the user
> > defines?
> 
> The output is what getOutput() returns... you can do whatever you like with it.
> batchflow (text mode tool) writes it to stdout, vflow (the GUI tool) writes it
> in a subwindow. If you want to save it in a file, all you need to do is to have
> the last node be a "Save" node. ...or the last node can be a soundcard output...
> In most cases, the output of the MAIN network is a nil object.

Excellent.  In Loci, we were going to have unconnected output connectors
(links) be 'outputs' of this type.  If these outputs are left unconnected by
the user, Loci would send the data to the parent of the workspace (document). 
And if there was no parent (the workspace is the highest-level workspace),
Loci would send the data to stdout.

> I'm not that much a fan of XML... The only reason I used it was that it saved me
> so much time (instead of writing a parser myself). So Anything that does the job
> is OK for me.

I like it.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Sat Mar 25 00:02:06 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
	 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
Message-ID: <38DC484E.781D6528@geoserve.net>

Jean-Marc Valin wrote:
> 
> OK, I'll try to summerize the way XML is used in Overflow with an example:

Brad and Jarl, perhaps GMS can reuse much of Overflow's code for handling XML
and network construction?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Sat Mar 25 00:08:02 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
 <38DC3470.D68AF122@geoserve.net> <38DC3EA9.F84B3EDF@hermes.usherb.ca> <38DC4586.178AE513@geoserve.net>
Message-ID: <38DC49B2.CD741462@hermes.usherb.ca>

> > > Great, this is just like Loci's locus types/classes.
> > >
> >
> > Just to make sure everyone understand, the definition is the equivalent of a
> > function call, not a function prototype/implementation.
> 
> Okay, I was showing you some actual locus implementation in XML.  Loci doesn't
> keep workspace information in a single flat file like Overflow does.  Loci
> (right now) uses the local filesystem to help organize the structure.  I'll
> let you, Jarl and Brad work out how VSH will do it.
> 
> > > This is like a mime-type for link I/O?  This is also something Loci was going
> > > to define.
> >
> > I'm not sure what you're talking about...
> 
> I mean, when Overflow defines a link, does it also define the type of data
> that is passed through the link?

Well, that's not necessary since all Overflow objects derive from the Object
class. Of course, some nodes require their input to be a certain type, so they
will check the type. However some nodes, like "save", don't care about the type,
since all objects know how to save themselves. Also some nodes, like "load"
don't even know what type they will return (it depends on what's in the file
their loading). BTW, the virtual getOutput() method defined in the base Node
class returns a type ObjectRef, which is a reference-counted pointer (aka
"smart-pointer") to an Object (the base class). For those who know the language,
our object system is a lot inspired by Smalltalk.

> > When the node is loaded (its library is dynamically linked), a function puts
> > that info in a node "database", which the GUI uses to know what node are
> > available (and takes what arguments).
> 
> How are you making this 'database'?  Is it an external db like mySQL or your
> own creation?

It's not a database in that sense, it's just a structure (I use an STL map) that
maps a node name to a set of information for the GUI (inputs, outputs,
parameters, ...).

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Sat Mar 25 01:05:30 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Jungle Monkey
Message-ID: <38DC572A.D7852AB1@geoserve.net>

I think this is something like Napster:

  http://www.eecs.umich.edu/~dhelder/misc/jm/

which could be used to make distributed VSH nodes easily accessible across the
Internet.

Jeff


From bizzaro at geoserve.net  Sat Mar 25 01:10:32 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] hOpla
Message-ID: <38DC5858.9C6F2AF2@geoserve.net>

What do you think of this, Brad?

  http://hopla.linuxbe.org/en/

Jeff


From jarl at casema.net  Sat Mar 25 05:51:25 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
	 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
	 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
	 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
Message-ID: <38DC9A2D.C9F83702@casema.net>

>

Hi all,

very intersesting post from Jean,

>
> A document is a file and can contain many networks (it's the equivalent of a .c)

> A network if the C equivalent of a function. Each document must contain a MAIN
> network.

Logical.

>
> NetOutput means which node (and which output of that node) will be used as the
> output of the whole network. The same applies to NetInput and NetCondition.
> Questions?
>

Can I find a overview somewhere in the Overflow code?
I've been looking in it, gives me the impression every node type is a separate c
source.
But the structure is clear, and I like it :)

bye,
jarl



From jarl at casema.net  Sat Mar 25 06:10:22 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] gui protocol revisited
References: <200003230149.UAA115166@archa11.cc.uga.edu> <38DACD96.8D0900F3@geoserve.net> <38DB1577.481B6BB4@casema.net> <38DBDA0F.19CA34CC@geoserve.net> <38DC03CC.3A5B5CEF@casema.net> <38DC3532.47422EF0@geoserve.net>
Message-ID: <38DC9E9E.6A1CAD03@casema.net>

> >
> > Yes. See also the GConf project.
> > It's about 'programs registring into gnome', so much relavant info is availeble
> > for wrapping an application. Less need to build code for every single application,
> >
> > maybe we could write a general one once we use GConf.
> >
> > Example: A email applet could simply register itself as a handler for the smtp
> > and pop mimetypes. GMS will collect this data from the gconf database, and
> > export the needed structure to activate this dataflow inside Overflow.
>
> Are you saying VSH will need GConf installed to do wrapping?  Or are you
> saying that we'll use something like GConf?
>

No, not right now. But GConf will be in GNOME 2.0. So at that time this application
information is available in every gnome environment. Let's stick to what Overflow
current does, but using the GConf registry will become very tempting once it's becoming

supported by more-and-more gnome applications.



From jarl at casema.net  Sat Mar 25 06:14:09 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
		 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
		 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
		 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca> <38DC484E.781D6528@geoserve.net>
Message-ID: <38DC9F81.5431E931@casema.net>

> Jean-Marc Valin wrote:
> >
> > OK, I'll try to summerize the way XML is used in Overflow with an example:
>
> Brad and Jarl, perhaps GMS can reuse much of Overflow's code for handling XML
> and network construction?
>

As said before, gms will just act passive on most of the XML descriptions.



From jarl at casema.net  Sat Mar 25 08:05:04 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Idl
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
			 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
			 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
			 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca> <38DC484E.781D6528@geoserve.net> <38DC9F81.5431E931@casema.net>
Message-ID: <38DCB97F.5BFC6B52@casema.net>

Hi,

I've been adding some to the idl brad created,
i'll work more on it tonight\tomorrow, but this is what I came up with.

bye,
jarl
-------------- next part --------------
// gui2dp.idl
//
// IDL for dealing with communication between the GUI (buffered by a small
// middle) and the data processing engine.
//
// Changelog:
//		23 Mar 2000		: Brad, creation 
//		25 Mar 2000		: Jarl, added gms naming

/**
 * Organization of modules.
 * vsh: The overall module (the name can change when we decide on a formal
 *      name).
 * gui2dp: idl for communication between the GUI (interpeter?) and the 
 *      data processing engine.
 * ADM: Automatic Definition Maker, a GUI or script that defines structures.
 * DFB: DataFlow Broker, layer that does the distribution\authentication\etc
 * DFP: DataFlow Procesor, layer that does the 'real' dataflow work.
 * 
 */

module vsh {
  module ADM2DFB {
    // ##### exceptions
    /**
     * Requested Id is already being used for another gui.
     */
    exception IdInUse {};
                            
    /**
     * Id number used for requested operation was not found in the list of
     * current gui ids being served by the data processor.
     */
    exception IdNotFound{};
                
    /**
     * The information for a node is not yet available because the process
     * has not completed.
     */
    exception InfoNotAvailable{};
                            
    // ##### structs and typedefs
    /**
     * A sequence/list of strings. Useful for returning info like a list of
     * program ids for available programs.
     */
    typedef sequence <string> stringList;
                            
    // ##### interfaces                 
                            
    /**
     * Functions for dealing with a front to middle connection.
     * TODO: define 'front' and 'middle' ??
     */
    //interface Connection {
	interface ADMRegister {
        /**
         * Initialize a connection from the gui to the data processing engine.
         * @parm gui_id - An unique id to represent the gui connection
         * @parm password - The password associated with this id.
         * @returns A boolean indicating the success of the initialization
         * @raises IdInUse - The passed gui_id is already in use by another gui.
         */
        boolean init(in string adm_id, in string password)
            raises (IdInUse);
        /**
         * Close the connection between the gui and data processing engine.
         * Used when closing down a GUI.
         * @parm dfb_id - An unique id to represent the gui connection
         * @parm password - The password associated with this id.
         * @returns A boolean indicating the success of the initialization
         * @raises IdNotFound - The passed id was not found in the list of
         *                      ids registerd with the data processing engine.
         */
        boolean release(in string adm_id, in string password)
            raises (IdNotFound);
    };
    /**
     * Functions for passing XML information to the data processing engine.
     */
    interface XML {
        /** 
         * Send a string of XML to the data processing engine to be dealt with.
         * @parm xml_to_process: A big string of xml describing the nodes/loci
         *       to be processed and the relationship between them.
         * @returns An id to be used for getting information about the
         *          process sent.
         */
        // XML defines
        string xml_structure(in string xml_document);                     
     }

     /**
      * Functions for dealing with 'subnets' (brokering\gms nodes)
      */
     interface SUBNET {
		uri_t new(in string dname, in string ddescription, in string datadescription, in string allowedfilter);
        boolean destruct(in uri_t uri);
		boolean init(in uri_t uri);
        boolean activate(in uri_t uri);
        uri_t duplicate(in uri_t uri);
            // set details
            boolean SETdname(in uri_t uri, in string displayedname);
            boolean SETstatus(in uri_t uri, in status_t status);
			boolean SETdataDescription(in uri_t uri, in string datadescription);
            boolean SETdDescription(in uri_t uri, in string displayeddescription);
            boolean SETallowedFilter(in uri_t uri, in string allowedfilter);
            boolean SETconfiguration(in uri_t uri, in string configruntime);
            // get details
            string GETdname(in uri_t uri);
            status_t GETstatus(in uri_t uri);
            string GETdataDescription(in uri_t uri);
            string GETdDescription(in uri_t uri);
            string GETallowedFilter(in uri_t uri);
            string GETconfiguration(in uri_t uri);
            MDO_t GEThistoricdata(in uri_t uri, in long history);
      }

      /**
       * Functions for single node access (cq overflow nodes)
       */
      interface NODE {
        // TODO: add more.. 
        uri_t new(in dfp_nodetype_t dfp_nodetype, in dfp_nodeparam_t dfp_nodeparameter);
        boolean destruct(in uri_t uri);
            // set single dfp nodes details
            boolean SETstatus(in uri_t uri, in status_t status); // TODO: can overflow handle 'nodes status' ??
            boolean SETparameter(in uri_t uri, in string parameters);
            boolean ADDparameter(in uri_t uri, in string parameter);
            // get details
            status_t GETstatus(in uri_t uri);
            string GETparameters(in uri_t uri);
    };
        
    /**
     * Functions dealing with a process that has been sent to the middle.
     */
    interface ProcessInformation {
     // TODO: move to ADM, SUBNET and NODE interface?
        /**
         * Request information the status of the overall process.
         * @parm process_id - the unique id associated with a process.
         * @returns A double representing the progress towards completion.
         *          1 indicates completion, 0 indicates the process has not
         *          started, numbers between 0 and 1 represent the percent
         *          completion, and -1 indicates an error in the process
         * @raises IdNotFound - If the passed process_id is not found.
         */
        double query_process(in string process_id) raises (IdNotFound);
        
        /**
         * Request information about a particular node/locus in a process.
         * @parm process_id - the unique id associated with a process.
         * @parm node_id - the unique id associated with a node.
         * @returns A double representing the progress towards completion.
         *          1 indicates completion, 0 indicates the process has not
         *          started, numbers between 0 and 1 represent the percent
         *          completion, and -1 indicates an error in the process.
         * @raises IdNotFound - if either the process or node id were not
         *         found.
         */
        double query_process_node(in string process_id, in string node_id)
            raises (IdNotFound);
                
        /**
         * Get the information for a node so that is can be displayed in the
         * GUI.
         * @parm process_id - the unique id associated with a process.
         * @parm node_id - the unique id associated with a node.
         * @returns A string with the info from the node.
         *          TODO: Need a structured and generalized way to return info
         *          and pass it to a visual so it can be seen in the GUI
         *          Strings aren't going to cut it for everything, 
         *          I don't think.
         * @raises InfoNotAvailable - If the node currently doesn't have any
         *         information to send (ie. process is not complete.
         */
        string get_node_info(in string process_id, in string node_id)
            raises (InfoNotAvailable);  
        
    };
    /**
     * Functions for getting information about programs and libraries registered
     * with the processing engine.
     */
    interface ProgramInformation {
        /**
         * Get a list of all the program ids of programs registered functional
         * (ie. loaded) with the processing engine.
         * @returns A sequence of strings (list) with the node_ids of availeble
         *          nodes.  
         */
        stringList get_program_ids();
        
        /**
         * Query a specific node to see if it is available.
         * @parm node_id - the unique id of a node (program/function).
         * @returns 1 if the program is loaded, 0 if it is not.
         */ 
        boolean is_available(in string node_id);
        
    };
    
  };
};
From jean-marc.valin at hermes.usherb.ca  Sat Mar 25 12:25:06 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca>
 <38DC9A2D.C9F83702@casema.net>
Message-ID: <38DCF672.B7579D8@hermes.usherb.ca>

> very intersesting post from Jean,
> 
> >
> > A document is a file and can contain many networks (it's the equivalent of a .c)
> 
> > A network if the C equivalent of a function. Each document must contain a MAIN
> > network.
> 
> Logical.

The only difference with a C program is that if you need many .n (that's how we
name the overflow files), you need one MAIN in each, while in C you only need
one main. Only the MAIN network is accessible from outside.

> 
> >
> > NetOutput means which node (and which output of that node) will be used as the
> > output of the whole network. The same applies to NetInput and NetCondition.
> > Questions?
> >
> 
> Can I find a overview somewhere in the Overflow code?

What kind of overview?

> I've been looking in it, gives me the impression every node type is a separate c
> source.
> But the structure is clear, and I like it :)

Yes, all node is in a separate .cc (If it starts with the NODE_INFO macro, it's
a node). Actually, all directories, except data-flow and vflow only contain
nodes (the two ohers contain nodes, but also other stuff). Lots of the node even
have the class definition in the .cc, since the code never explicitly uses that
class (the NODE_INFO simply puts a Factory for that class in a global
"database"). The only nodes that need to have the class in a .h are the ones
that are inherited. For example, we have a Probe node that's a bit like a
debugger (Next, Stop, Continue) and we have a TextProbe and PlotProbe node that
add a display (text of plot) to the Probe.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From chapmanb at arches.uga.edu  Sat Mar 25 13:06:45 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:37 2006
Subject: [Pipet Devel] Idl
Message-ID: <200003251809.NAA124632@archa12.cc.uga.edu>

Jarl wrote:
> I've been adding some to the idl brad created,
> i'll work more on it tonight\tomorrow, but this is what I came up 
with.

Thanks for looking at this.In general I'd like to try and make the 
interface smaller (since I'm lazy and want to implement as few 
interfaces and functions as possible :) 
    Also, the uri_t you are referring to (as a reference to a node or 
subnet) is the same thing I was referring to using two references, a 
process_id and a node_id. Maybe we could combine the two ideas by 
having the uri_t be a string concatenation of the two separated by a 
color (ie. uri_t = 'process_id:node_id').

Here are my specific thoughts (a lot of this is just talking out loud, 
so please bare with me!):

For the SUBNET interface:

    These are for accessing whole subnets or programs (so GMS type 
nodes) after they have been sent a workflow/xml_structure to process 
(ie. a whole bunch of connected nodes), right? 
    My question is, do we really need this level of control so that 
someone can
manipulate a process so much while it is running? The way I was 
thinking of the GUI and processing engine working together are that 
the user spends some time getting a nice process of connected nodes 
set up, and then hits run and waits for the output to come out at the 
specific nodes/subnets they are interested in. If this is the case, 
then why and when would they need to access a process to remove, 
activate, destruct, add, etc. nodes and subnets?
    Am I being naive here in thinking that we don't need all of this 
control of a running process? Perhaps what we do need though it a 
method for cancelling and run and getting as much information as you 
can from it...

    So can we drop these following functions out and leave it up to 
the processor to get this information from the xml that is sent?
 
uri_t new(in string dname, in string ddescription, in string 
datadescription, in string allowedfilter);
boolean destruct(in uri_t uri);
boolean init(in uri_t uri);
boolean activate(in uri_t uri);
uri_t duplicate(in uri_t uri);
    
    boolean SETdname(in uri_t uri, in string displayedname);

    boolean SETallowedFilter(in uri_t uri, in string allowedfilter);
    boolean SETconfiguration(in uri_t uri, in string configruntime);
			  boolean SETdataDescription(in uri_t uri, in string 
datadescription);
    string GETdataDescription(in uri_t uri);
    string GETdname(in uri_t uri);

For the following functions--do we need the processor to know about 
the description of the program? Can all of this type of information by 
stored in xml in the "middle scripting thingy"?

    boolean SETdDescription(in uri_t uri, in string 
displayeddescription);

    string GETdDescription(in uri_t uri);

For the following functions: I don't really understand the set status 
(how does the scripting engine set the status of a process?). I tried 
to include GETstatus type stuff in my query_process function. But 
maybe we should split this querying to be specific for a subnet or 
node?

    boolean SETstatus(in uri_t uri, in status_t status);
    status_t GETstatus(in uri_t uri);

For the following, I'm not really positive why the middle would need 
to know this stuff? Do you want the GUI to restrict only allowed 
things to be connected? This seems *really* hard to maintain--maybe it 
would be better to have the processing engine generate errors and then 
propogate them back to the GUI so the user can fix them? The idl 
probably does need a better way to return back error messages.

    string GETallowedFilter(in uri_t uri);
    string GETconfiguration(in uri_t uri);

For the following, do we still need the processing engine to store a 
log/history of events, or should this be handled by the middle 
scripting part, which can maintain logs for all its communication. 

    MDO_t GEThistoricdata(in uri_t uri, in long history);


For the NODE interface:
    I think the same kind of comments I made above apply here: how 
about if we just send all of this information to the middle as XML. I 
think that the current Overflow XML will be a good point to start with 
and we can work from there. What I need to get working is a way to 
turn the directory/heirarchy based file storage system that Loci 
currently has into the type of flat XML that Overflow uses, so I can 
pass this. While the heirarchial structure is best for dealing with 
the GUI (since I have to make lots of changes to it as the GUI 
changes, it is easier to break it up into smaller chunks for 
manageability and speed), the flat file format is what we need so that 
we can pass a single file to the processing engine to process (and to 
stay compatible with what Overflow does since they have a good thing 
going). We also will not want to pass all of the info that is stored 
in XML in the middle (like descriptions of programs, etc.) to the 
processing engine, so some kind of transformation will be needed. I 
need to start trying to think about how to do this, but I hope it 
shouldn't be too bad...
    Thanks again for checking out the idl, Jarl. Hopefully all of us 
can get it hashed out over the next few days without a problem.

Brad



From gvd at redpoll.pharmacy.ualberta.ca  Sat Mar 25 15:21:30 2000
From: gvd at redpoll.pharmacy.ualberta.ca (Gary Van Domselaar)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] 3/1 and 1/3
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA7ED9.9D3F46A1@hermes.usherb.ca>
		 <38DA94D7.E40E84B4@casema.net> <38DAE9FA.9919576C@geoserve.net> <38DAFBB3.FB7B40@hermes.usherb.ca> <38DBD8BF.10AC39FA@geoserve.net>
Message-ID: <38DD1FCA.427709C4@redpoll.pharmacy.ualberta.ca>

"J.W. Bizzaro" wrote:

> > About the architecture... what about this: Everything that's done locally
> > (including running other exceutables) is handled by Overflow and all networking
> > stuff would be handled by GMS, which would take care of connecting all Overflow
> > processes (here I'm talking about "process" in a weak sense) through a network.
> > This way there wouldn't be any duplication. Also, every node-to-node connection
> > in the GMS part of the GUI would be assumed to be "network-able".
> 
> This is EXACTLY what I was thinking of.  Brad and Jarl, is this not what you
> guys were thinking?

I'll add my $cdn 0.02 here.  This seems like the most sensible solution.
Integrating Overflow and GMS this way allows the data to flow
transparently, >and< preserves Overflow's speed-optimized data flow
protocol.

> 
> > Concerning the GUI, I don't think it would be hard to plug in what we've got so
> > far in another app. Or we can just embed the current GUI in the new app using
> > bonobo. I don't know enough about this to say which is best.
> 
> I don't dislike Overflow's GUI.  I just believe GMS and Overflow should have
> GUIs that work alike.  Plus, we've come up with some very nice GUI features
> for Loci that I'd like to see VSH have.

I agree.  From the user's perspective, both GMS and Overflow behave
alike; they should a common interface.  That way, the end user need not
be aware of which 'engine' is behind the interface, which is the way is
should be.


Regards,

g.
-- 
                                  Gary Van Domselaar
                               gary@bioinformatics.org
                          http://www.bioinformatics.org/~gary
                ----------------------------------------------------
                           bioinformatics.org: The Open Lab
                            http://www.bioinformatics.org/


From bizzaro at geoserve.net  Sat Mar 25 22:06:36 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] XML compression
Message-ID: <38DD7EBC.15FA77B7@geoserve.net>

http://www.xml.com/pub/2000/03/22/deviant/index.html

Jeff


From jarl at casema.net  Sun Mar 26 07:46:58 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Naming of three layers
References: <38DD7EBC.15FA77B7@geoserve.net>
Message-ID: <38DE06C2.7A5E54D3@casema.net>

Hi all,

Bard and I are doing some work defining the IDL of the
CORBA communications between
the GUI and the CORE, and we need to have so naming
settled:

I propose using this labelling for the three layers
VSH will be build from:
- The DDL, or Dataflow Definitions Layer. This name will
replace 'GUI', because it doesn't need to be graphical, it
only needs to supply a XML document.
- The DBL, or Dataflow Brokering Layer, which will replace
'gms'.
- The DPL, or  Dataflow Processing Layer, which replaces
'Overflow'

Some feedback please, maybe remove 'Dataflow' in all
three?

bye,
jarl



From jarl at casema.net  Sun Mar 26 09:43:56 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Idl2
References: <200003251809.NAA124632@archa12.cc.uga.edu>
Message-ID: <38DE222C.E3EEEC24@casema.net>

Hi,

Checked the syntax and compiled the idl :)

bye,
jarl
-------------- next part --------------
// DDL2DBL.idl
//
// IDL for dealing with communication between the GUI (buffered by a small
// middle) and the data processing engine.
//
// Changelog:
//		23 Mar 2000		: Brad, creation 
//		25 Mar 2000		: Jarl, added gms naming
//      26 Mar 2000     : Renamed again :) added\remove some functions
//                        Actually compiled this :)

/**
 * Organization of modules.
 * vsh: The overall module (the name can change when we decide on a formal
 *      name).
 * gui2dp: idl for communication between the GUI (interpeter?) and the 
 *      data processing engine.
 * ADM: Automatic Definition Maker, a GUI or script that defines structures.
 * DFB: DataFlow Broker, layer that does the distribution\authentication\etc
 * DFP: DataFlow Procesor, layer that does the 'real' dataflow work.
 * 
 */

module VSH {
  /**
   * Functions exported by the DataFlow Broker
   */
  module DBL {
    // ##### exceptions
    // ----------------

    // Requested Id is already being used for another gui.
    exception DDLIdInUse {};
    // Id number used for requested operation was not found in the list of
    // current gui ids being served by the data processor.
    exception DDLIdNotFound{};
    // The information for a node is not yet available because the process
    // has not completed.
    exception DDLInfoNotAvailable{};
                            
    // The DBL\DFP couldn't accept the XML document and so didn't allocate an uri.
    exception URIUndefined{};
    // The uri is already defined.
    exception URIInUse{};
    // The uri cant be found, it does not excist or is not accessable.
    exception URINotFound{};

    // Requested information was not found.
    exception INFONotAvailable{};

    // ##### structs, enums and typedefs
    // ---------------------------------

    // A sequence/list of strings. Useful for returning info like a list of
    // program ids for available programs.
    typedef sequence <string> stringlistT;
    // DDL identification number, or 'account name'.
    typedef long ddlidT;
    // Passwords
    typedef string passwordT;

    // Connection type:
    //   -1 = Unconnected
    //    0 = Unknown
    //    1 = Plain connection
    //    2 = Encrypted connection, based on {TODO:type} encryption
    enum connectionE {
        UNKNOWN,
        UNCONNECTED,
        PLAIN,
        ENCRYPTED1
    };

    // Status type:
    //   -1 = Error (TODO: Other negative values to define various errors)
    //    0 = Unknown
    //    1 = Passing
    //    2 = Blocking
    //    4 = Temporary
    //    8 = Isolated
    typedef long statusT;
    
    // URI type, a 128bits locating id:
    // bits 0-31 : instance id
    // bits 32-63 : brokering id, or subnet id
    // bits 64-127 : node id
    struct uriT {
        long instanceID;
        long subnetID;
        long nodeID;
    };
                            
    // ##### interfaces                 
    // ----------------
                            
    // Functions for dealing with a front to middle connection.
    // TODO: define 'front' and 'middle' ??

    //interface Connection {
	interface CONNECTION {
    // TODO: maybe introduce a 'access level', so ADM's can be limited?
        /** setup
         * Initialize a connection from the gui to the data processing engine.
         * @parm    ddl_id - An unique id to represent the gui connection
         * @parm    password - The password associated with this id.
         * @parm    connecttype - requested connection type.
         * =returns A boolean indicating the success of the initialization
         * +raises  IdInUse - The passed gui_id is already in use by another gui.
         */
        boolean setup(in ddlidT ddlid, in passwordT password, in connectionE connecttype)
            raises (DDLIdInUse);
        /** release
         * Close the connection between the gui and data processing engine.
         * Used when closing down a GUI.
         * @parm    ddl_id - An unique id to represent the gui connection
         * @parm    password - The password associated with this id.
         * =returns A boolean indicating the success of the initialization
         * +raises  IdNotFound - The passed id was not found in the list of
         *          ids registerd with the data processing engine.
         */
        boolean release(in ddlidT ddlid, in passwordT password)
            raises (DDLIdNotFound);
        /** check
         * Checks if the DDL with id 'ddl_id' has been connected
         * @parm    ddl_id - An unique id to represent the gui connection
         * =returns type of connection
         * +raises  DDLInfoNotAvailable - this connection isn't setup correctly
         */
        connectionE check(in ddlidT ddlid)
            raises (DDLInfoNotAvailable);
    };

    /**
     * Functions for passing XML information to the data processing engine.
     */
    interface REPRESENTATION {
    // Functions dealing with XML document syncronisation.

        /** upload
         * Uploads a XML document to the DBL, which will pass most of it to the DPL.
         * @parm    xml_to_process: A big string of xml describing the nodes/loci
         *          to be processed and the relationship between them.
         * @parm    ddl_id - The ID of the DDL that uploads the document
         * @parm    password - The password associated with this id.
         * =returns the URI the uploaded structure got assigned by the DBL.
         * +raises  URIUndefined, the DBL\DFP couldn't accept the XML document. Use 
         *          DATAFLOW::check to see what is wrong
         */
        uriT upload(in string xmldocument, in ddlidT ddlid, in passwordT password)
            raises (URIUndefined);

        /** update
         * Updates an excisting structure.
         * @parm    xml_document - The XML document
         * @parm    uri - the uri id of the structure to be overwritten
         * @parm    ddl_id - The ID of the DDL that uploads the document
         * @parm    password - The password associated with this id.
         * =returns boolean success status.
         * +raises  URIInUse, the uri is currently running or the DDL is not authorized 
         *          to overwrite it.
         */
        boolean update(in string xmldocument, in uriT uri, in ddlidT ddlid, in passwordT password)
            raises (URIInUse);

        /** download
         * Downloads a XML document from the DBL to the DDL.
         * @parm    uri - uri of the structure to be downloaded.
         * @parm    ddl_id - The ID of the DDL that uploads the document
         * @parm    password - The password associated with this id.
         * =returns XML document.
         * +raises  URINotFound when uri doesn't excist or is not reachable.
         */
        string download(in uriT uri, in ddlidT ddlid, in passwordT password)
            raises (URINotFound);

        /** status
         * Used for checking the status of an URI 
         * @parm    uri - uri of the to be checked structure
         * @parm    ddl_id - The ID of the DDL that uploads the document
         * @parm    password - The password associated with this id.
         * =returns status of the structure uri
         * +raises  URINotFound when uri doesn't excist or is not reachable.
         */
        statusT check(in uriT uri, in ddlidT ddlid, in passwordT password)
            raises (URINotFound);
     };

    /**
     * Functions dealing with a process that has been sent to the middle.
     */
    interface STATUS {
    // Functions that give status information of the core (DBL and DPL)
    // Functions for getting information about programs and libraries registered
    // with the processing engine.
    // TODO: Jarl should put more work in checking these :)

        /** 
         * Request information the status of the overall process.
         * @parm process_id - the unique id associated with a process.
         * @returns A double representing the progress towards completion.
         *          1 indicates completion, 0 indicates the process has not
         *          started, numbers between 0 and 1 represent the percent
         *          completion, and -1 indicates an error in the process
         * @raises IdNotFound - If the passed process_id is not found.
         */
       //TODO: this function is replace by REPRESENTATION::check ????  double query_process(in string process_id) raises (IdNotFound);
        
        /**
         * Request information about a particular node/locus in a process.
         * @parm process_id - the unique id associated with a process.
         * @parm node_id - the unique id associated with a node.
         * @returns A double representing the progress towards completion.
         *          1 indicates completion, 0 indicates the process has not
         *          started, numbers between 0 and 1 represent the percent
         *          completion, and -1 indicates an error in the process.
         * @raises IdNotFound - if either the process or node id were not
         *         found.
         */
        //TODO: see above?? double query_process_node(in string process_id, in string node_id)
        //    raises (IdNotFound);
                
        /**
         * Get the information for a node so that is can be displayed in the
         * GUI.
         * @parm process_id - the unique id associated with a process.
         * @parm node_id - the unique id associated with a node.
         * @returns A string with the info from the node.
         *          TODO: Need a structured and generalized way to return info
         *          and pass it to a visual so it can be seen in the GUI
         *          Strings aren't going to cut it for everything, 
         *          I don't think.
         * @raises InfoNotAvailable - If the node currently doesn't have any
         *         information to send (ie. process is not complete.
         */
        string getnodeinfo(in passwordT processid, in uriT uri)
            raises (INFONotAvailable);  
        
        /**
         * Get a list of all the program ids of programs registered functional
         * (ie. loaded) with the processing engine.
         * @returns A sequence of strings (list) with the node_ids of availeble
         *          nodes.  
         */
        stringlistT getprogramids();
        
        /**
         * Query a specific node to see if it is available.
         * @parm node_id - the unique id of a node (program/function).
         * @returns 1 if the program is loaded, 0 if it is not.
         */ 
        boolean isavailable(in uriT uri);
        
    };
    
  };
};
From chapmanb at arches.uga.edu  Sun Mar 26 12:00:39 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Naming of three layers
Message-ID: <200003261703.MAA80502@archa12.cc.uga.edu>

Jarl wrote:
> I propose using this labelling for the three layers
> VSH will be build from:

As you suggested, I'd be for dropping the leading dataflow (otherwise 
they all seem too much alike :)

> - The DDL, or Dataflow Definitions Layer. This name will
> replace 'GUI', because it doesn't need to be graphical, it
> only needs to supply a XML document.

My only comment on this is that the DDL (or DL or whatever) is 
actually made up of two sections. The first is the GUI, or text based 
front that the user will interact with. The second is the 'scripting 
layer' which communicates with the GUI through a streaming XML dialog.
    These are very separate entities, since the plan is that the 
current Gnome UI could also be web interface or a 'natural language' 
interface, running with the same 'scripting layer'.
    So we should have two different terms to describe these (although 
I'm not creative enough to come up with good ones :-)

Brad







From jarl at casema.net  Sun Mar 26 12:51:47 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu>
Message-ID: <38DE4E33.4F59B7DC@casema.net>

>
> As you suggested, I'd be for dropping the leading dataflow (otherwise
> they all seem too much alike :)

OK, fine to me.
SO: what about this:

GL : Graphical layer - GUI
DL: Definition Layer - scripting\XML layer
BL: Brokering Layer - brokering
PL: Processing Layer - dataflow handler

Comments?




From jarl at casema.net  Sun Mar 26 14:58:42 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] 3/1 and 1/3
References: <Pine.A41.4.10.10003231452560.115156-100000@arch14.cc.uga.edu> <38DA7ED9.9D3F46A1@hermes.usherb.ca>
			 <38DA94D7.E40E84B4@casema.net> <38DAE9FA.9919576C@geoserve.net> <38DAFBB3.FB7B40@hermes.usherb.ca> <38DBD8BF.10AC39FA@geoserve.net> <38DD1FCA.427709C4@redpoll.pharmacy.ualberta.ca>
Message-ID: <38DE6BF2.6761C03F@casema.net>

> > > About the architecture... what about this: Everything that's done locally
> > > (including running other exceutables) is handled by Overflow and all networking
> > > stuff would be handled by GMS, which would take care of connecting all Overflow
> > > processes (here I'm talking about "process" in a weak sense) through a network.
> > > This way there wouldn't be any duplication. Also, every node-to-node connection
> > > in the GMS part of the GUI would be assumed to be "network-able".
> >
> > This is EXACTLY what I was thinking of.  Brad and Jarl, is this not what you
> > guys were thinking?

We'll disable (do not use) the features that collide with this.

>
>
> I agree.  From the user's perspective, both GMS and Overflow behave
> alike; they should a common interface.  That way, the end user need not
> be aware of which 'engine' is behind the interface, which is the way is
> should be.

Yes, even the gui wont notice the diff., there will be one function 'upload()'
called on the BL ('gms') orb.




From jean-marc.valin at hermes.usherb.ca  Sun Mar 26 15:32:13 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net>
Message-ID: <38DE73CD.4505B80D@hermes.usherb.ca>

> GL : Graphical layer - GUI
> DL: Definition Layer - scripting\XML layer
> BL: Brokering Layer - brokering
> PL: Processing Layer - dataflow handler

Fine with me! (what about Visual Layer for the GUI, it's VSH after all...)

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Sun Mar 26 17:02:02 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net> <38DE73CD.4505B80D@hermes.usherb.ca>
Message-ID: <38DE88DA.25044FA2@geoserve.net>

Jean-Marc Valin wrote:
> 
> > GL : Graphical layer - GUI
> > DL: Definition Layer - scripting\XML layer
> > BL: Brokering Layer - brokering
> > PL: Processing Layer - dataflow handler
> 
> Fine with me! (what about Visual Layer for the GUI, it's VSH after all...)

The only trouble I had was with the names GL and DL.  I agree with Brad that
'DL' doesn't really cover the user interface and should be part of 'the core'.

But 'Graphical Layer' doesn't allow for non-graphical user interfaces, such as
web and text-based interfaces.

Jean-Marc's suggestion of 'Visual Layer' is nice, since it COULD be applied to
non-graphical interfaces.  But what about a speech-based interface?  Or a
session-recording interface?  Those certainly are not 'visual'.

I was thinking that 'Interfaces Layer' (IL) might work, but the word
'interface' does not always mean USER interface and could be confusing in the
context of VSH.  So, how about we bring back 3 letters and do this?

  UIL: User Interfaces Layer - user interfaces
  DDL: Dataflow Definition Layer - scripting\XML layer
  DBL: Dataflow Brokering Layer - brokering
  DPL: Dataflow Processing Layer - dataflow handler

(BTW, the argument that 'Dataflow' is redundant could also be applied to
'Layer' ;-))

There are also the 'foreign applications' and 'objects' to consider.  We could
add a...

  OAL: Objects and Applications Layer

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From chapmanb at arches.uga.edu  Sun Mar 26 17:07:08 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] new idl
Message-ID: <200003262209.RAA86676@archa12.cc.uga.edu>


Hello all;
    Jarl and I did some more talking and work on the idl, and here is 
the updated version for your reading pleasure. Note that I am using 
the DL, BL, PL naming conventions, but this can be easily changed 
depending on what we decide for names (how about D, B, P for maximum 
lack of redundancy :-). This is a sort of _finalized_ idl for now (ie. 
Jarl and I both seem to like it) so comments from everyone are *very* 
welcome. Have fun with it!

Brad

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/octet-stream
Size: 15419 bytes
Desc: not available
Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000326/bba091d0/attachment.obj
From chapmanb at arches.uga.edu  Sun Mar 26 17:16:03 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] new idl
Message-ID: <200003262218.RAA71008@archa11.cc.uga.edu>

Oops, I screwed stuff up and that last idl didn't attach so well. (I'm 
also using an open source beta version mailer with lots o' bugs, so 
maybe I can pretend to blame it on that :-) How about if I just paste 
it on in? 
    Hope this is better :)

Brad

// dl2bl.idl
//
// IDL for dealing with communication between the GUI (buffered by a 
small
// middle) and the data processing engine.
//
// Changelog:
//      23 Mar 2000     : Brad, creation 
//      25 Mar 2000     : Jarl, added gms naming
//      26 Mar 2000     : Renamed again :) added\remove some functions
//                        Actually compiled this :)
//      27 Mar 2000     : Brad, case changes, moved a few functions 
around
//                        Added a struct for returning more info about 
a 
//                        node/process.
/**
 * Acronyms
 * vsh: The overall module (the name can change when we decide on a 
formal
 *      name).
 * dl2bl: idl for communication between the scripting/XML layer and 
the 
 *      data processing engine.
 * dl : definition layer, involved in the creation of an XML script to 
define
 *  structures.
 * bl: Brokering Layer, layer that does the 
distribution\authentication\etc
 * pl: Processing Layer, layer that does the 'real' dataflow work.
 * 
 */
module vsh {
  /**
   * Communication between the definition layer (XML script generator) 
and
   * the brokering layer, which distributes the XML info to the 
processing 
   * layer.
   */
  module dl2bl {
    // ##### exceptions
    // ----------------
    /**
     * Requested Id is already being used for another gui.
     */
    exception DLIdInUse {};
    /**
     * Id number used for requested operation was not found in the 
list of
     * current gui ids being served by the data processor.
     */
    exception DLIdNotFound{};
    /**
     * The information for a node is not yet available because the 
process
     * has not completed.
     */
    exception DLInfoNotAvailable{};                     
    /**
     * The definition layer couldn't accept the XML document and so 
didn't 
     * allocate an uri.
     */
    exception URIUndefined{};
    /**
     * The uri is already defined.
     */
    exception URIInUse{};
    /**
     * The uri cant be found, it does not excist or is not accessable.
     */
    exception URINotFound{};
    /**
     * Requested information was not found.
     */
    exception InfoNotAvailable{};
    
    // ##### structs, enums and typedefs
    // ---------------------------------
    /**
     * DL identification number, or 'account name'.
     */
    typedef long DLIdT;
    /**
     * Passwords.
     */
    typedef string passwordT;
    /**
     * Connection type:
     *   -1 = Unconnected
     *    0 = Unknown
     *    1 = Plain connection
     *    2 = Encrypted connection, based on {TODO:type} encryption
     */
    enum connectionE {
        UNKNOWN,
        UNCONNECTED,
        PLAIN,
        ENCRYPTED1
    };
    /**
     * Current state of a node or process:
     * -1 = Error (TODO: Other negative values to define various 
errors)
     *  0 = Unknown
     *  1 = Passing
     *  2 = Blocking
     *  4 = Temporary
     *  8 = Isolated
     */
    typedef long stateT;
    
    /**
     * A value between zero and one indicating the process towards 
completion
     * for a process, subnet or node. This is useful if you have 
really long
     * programs and want to check on their status.
     * 1 -> finished (100% done)
     * intermediate -> % towards completion
     * 0 -> not started (0% done)
     */
    typedef double progressT;
    
    /**
     * Struct for returning the status of a particular node or process.
     * @mem state - The current state of the node or process.
     * @mem progress - The progress towards completion of the node or 
     *                 process.
     */
    struct statusT{
        stateT state;
        progressT progress;
    };
        
    /**
     * URI type, a 128bits locating id:
     * bits 0-31 : instance id
     * bits 32-63 : brokering id, or subnet id
     * bits 64-127 : node i
     * uriT is in a IP (Internet protocol) style
     *      uriT = 2.0.0 -> All subnets and nodes local in instance 2
     *      uriT = 74.35.0 -> All nodes in subnet 35 of instance 74.
     *      uriT = 1.2.3 -> Node 3 in subnet 2 of instance 1.
     */
    struct uriT {
        long instanceID;
        long subnetID;
        long nodeID;
    };
    
    /**
     * A sequence/list of uriTs. Useful for returning info like a list 
of
     * program ids for available programs.
     * 
     */
    typedef sequence <string> uriListT;
                            
    // ##### interfaces                 
    // ----------------
                            
    /**
     * Functions for establishing a connection between the definition 
layer
     * and brokering layer (which controls access to the processing 
layer).
     */
    interface Connection {
        // TODO: introduce a 'access level', so ADM's can be limited
        
        /** setup
         * Initialize a connection from the dl to the 
brokering/processing 
         * layer.
         * @parm    dlid- An unique id to represent the connection.
         * @parm    password - The password associated with this id.
         * @parm    connecttype - requested connection type.
         * @returns A boolean indicating the success of the 
initialization.
         * @raises  DLIdInUse - The passed dlid is already in use by 
another 
         * gui.
         * TODO: Add more exceptions to raise as we see what problems 
come up.
         */
        boolean setup(in DLIdT dlid, in passwordT password, in 
connectionE 
                      connecttype) 
            raises (DLIdInUse);
        /** release
         * Close the connection between the definition layer and the 
brokering
         * processing layer
         * @parm    dlid - An unique id to represent the connection.
         * @parm    password - The password associated with this id.
         * @returns A boolean indicating the success of the 
initialization
         * @raises  DLIdNotFound - The passed id was not found in the 
list of
         *          ids registerd with the data processing engine.
         */
        boolean release(in DLIdT dlid, in passwordT password)
            raises (DLIdNotFound);
            
        /** checkConnection
         * Checks if the DL with id 'dlid' has been connected.
         * @parm    dlid - An unique id represting the connection.
         * @returns type of connection
         * @raises  DLInfoNotAvailable - this connection isn't setup 
correctly
         */
        connectionE checkConnection(in DLIdT dlid)
            raises (DLInfoNotAvailable);
    };
    /**
     * Functions for passing XML information to the data processing 
engine
     * and synchronizing the document.
     */
    interface XmlRepresentation {
        /** upload
         * Uploads a XML document to the DL, which will pass most of 
it to the 
         * PL.
         * @parm    xmldocument: A big string of xml describing the 
nodes/loci
         *          to be processed and the relationship between them.
         * @parm    dlid - The ID of the DL that uploads the document
         * @parm    password - The password associated with this id.
         * @returns the URI the uploaded structure got assigned by the 
DL.
         * @raises  URIUndefined, the DL/PL couldn't accept the XML 
document. 
         *          Use RetrieveInfo::check to see what is wrong.
         */
        uriT upload(in string xmldocument, in DLIdT dlid, in passwordT 
password)
            raises (URIUndefined);
            
        /** update
         * Updates an excisting structure.
         * @parm    xmldocument - The XML document
         * @parm    uri - the uri id of the structure to be 
overwritten.
         * @parm    dlid - The ID of the DL that uploads the document
         * @parm    password - The password associated with this id.
         * @returns boolean success status.
         * @raises  URIInUse, the uri is currently running or the DDL 
is not 
         *          authorized to overwrite it.         
         */
        boolean update(in string xmldocument, in uriT uri, in DLIdT 
dlid, 
                       in passwordT password)
            raises (URIInUse);
        
        /** download
         * Downloads a XML document from the BL to the DL.
         * @parm    uri - uri of the structure to be downloaded.
         * @parm    dld - The ID of the DL that uploaded the document
         * @parm    password - The password associated with this id.
         * @returns An XML document.
         * @raises  URINotFound when uri doesn't exist or is not 
reachable.
         */
        string download(in uriT uri, in DLIdT dlid, in passwordT 
password)
            raises (URINotFound);
            
     };
    /**
     * Functions to provide status information for the core (brokering 
and 
     * processing layers), and for getting information about programs 
and
     * libraries registered with the processing engine.
     */
    interface RetrieveInfo {
        /** checkStatus
         * Used for checking the status of an URI 
         * @parm    uri - uri of the structure to be checked.
         * @parm    dlid - The ID of the DL that uploads the document
         * @parm    password - The password associated with this id.
         * @returns Status of the structure uri.
         * @raises  URINotFound when uri doesn't excist or is not 
reachable.
         */
        statusT checkStatus(in uriT uri, in DLIdT dlid, in passwordT 
password)
            raises (URINotFound);
                
        /**
         * Get the information for a node so that is can be displayed 
in
         * the visual user interface.
         * @parm processid - the unique id associated with a process.
         * @parm uri - uri of the node to be checked.
         * @returns A string with the info from the node.
         *          TODO: Need a structured and generalized way to 
return info
         *          and pass it to a visual so it can be seen in the 
GUI
         *          Strings aren't going to cut it for everything, 
         *          I don't think.
         * @raises InfoNotAvailable - If the node currently doesn't 
have any
         *         information to send (ie. process is not complete).
         */
        string getNodeInfo(in passwordT processid, in uriT uri)
            raises (InfoNotAvailable);  
        
        /**
         * Get a list of all the program ids of programs registered 
functional
         * (ie. loaded) with the processing engine.
         * @returns A sequence of uri's with ids of available 
         *          nodes/subnets/programs. These will be returned 
with leading 
         *          zeros. For instance you could get back a list like 
the 
         *          following:
         *          ['0.0.27', '0.0.345', '0.0.123'...] 
         *          It will be up to the definition layer to keep a 
dictionary 
         *          to translate between these uri's and the actual 
program 
         *          names.
         */
        uriListT getProgramIds();
        
        /**
         * Query a specific node to see if it is available.
         * @parm node_id - the unique id of a node (program/function).
         * @returns 1 if the program is loaded, 0 if it is not.
         * @raises URINotFound - If the requested URI can not be 
located.
         */ 
        boolean isAvailable(in uriT uri)
            raises (URINotFound); 
    };
    
  };
};





From bizzaro at geoserve.net  Sun Mar 26 17:23:27 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] new idl
References: <200003262209.RAA86676@archa12.cc.uga.edu>
Message-ID: <38DE8DDF.914A9968@geoserve.net>

Brad Chapman wrote:
> 
> Note that I am using
> the DL, BL, PL naming conventions, but this can be easily changed
> depending on what we decide for names (how about D, B, P for maximum
> lack of redundancy :-).

For maximum READABILITY, you could just use...

  definitions
  brokering
  processing

:-)

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Sun Mar 26 17:36:07 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net>
 <38DE73CD.4505B80D@hermes.usherb.ca> <38DE88DA.25044FA2@geoserve.net>
Message-ID: <38DE90D7.AE208D29@hermes.usherb.ca>

>   UIL: User Interfaces Layer - user interfaces
>   DDL: Dataflow Definition Layer - scripting\XML layer
>   DBL: Dataflow Brokering Layer - brokering
>   DPL: Dataflow Processing Layer - dataflow handler
> 
> (BTW, the argument that 'Dataflow' is redundant could also be applied to
> 'Layer' ;-))

Yes, what about:
  -User Interface
  -Definition
  -Brokering
  -Processing

Since we all know they are all layers and that everything's about dataflow
anyway.


> There are also the 'foreign applications' and 'objects' to consider.  We could
> add a...
> 
>   OAL: Objects and Applications Layer
I don't consider objects to be a layer... they're just objects (now that's new
;-) )

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Sun Mar 26 17:54:34 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net>
	 <38DE73CD.4505B80D@hermes.usherb.ca> <38DE88DA.25044FA2@geoserve.net> <38DE90D7.AE208D29@hermes.usherb.ca>
Message-ID: <38DE952A.E3CCE58C@geoserve.net>

Jean-Marc Valin wrote:
> 
> I don't consider objects to be a layer... they're just objects (now that's new
> ;-) )

Technically, applications (and maybe objects) are not part of VSH.  But in
Loci, they were called 'back-endware'.  These 'dataflow * layers' were
collectively called 'middleware', and the user interfaces were called
'front-endware' in Loci.  I think these terms can still be used.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Sun Mar 26 17:57:37 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:38 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net>
	 <38DE73CD.4505B80D@hermes.usherb.ca> <38DE88DA.25044FA2@geoserve.net> <38DE90D7.AE208D29@hermes.usherb.ca>
Message-ID: <38DE95E1.F8E9E74E@geoserve.net>

Jean-Marc Valin wrote:
> 
> Yes, what about:
>   -User Interface
>   -Definition
>   -Brokering
>   -Processing

Or messing around with word forms...

  User Interfaces
  Definitions
  Brokers
  Processors

And I still like...

  Objects and Applications :-)

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Sun Mar 26 18:08:31 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net>
		 <38DE73CD.4505B80D@hermes.usherb.ca> <38DE88DA.25044FA2@geoserve.net> <38DE90D7.AE208D29@hermes.usherb.ca> <38DE95E1.F8E9E74E@geoserve.net>
Message-ID: <38DE986F.CCABCA7A@geoserve.net>

If we can agree on this:

   User Interfaces  ----------------->  Front-endware

               _______________
   Definitions                |
   Brokers                    |------>  Middleware
   Processors  _______________|

   Objects and Applications  -------->  Back-endware

then DL, BL and PL are all 'layers' of the 'middleware' that you are writing
the IDL for.  And we can probably continue to say 'front' and 'back' as we did
with Loci.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Sun Mar 26 18:57:39 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] hOpla
Message-ID: <38DEA3F3.EA19A1D1@geoserve.net>

 
-------------- next part --------------
An embedded message was scrubbed...
From: Brad Chapman <chapmanb@arches.uga.edu>
Subject: [Pipet Devel] Re: [Pipet Devel] hOpla
Date: Sun, 26 Mar 2000 18:37:24 -0500
Size: 3794
Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000326/63692a50/attachment.mht
From jarl at casema.net  Mon Mar 27 01:14:10 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net>
	 <38DE73CD.4505B80D@hermes.usherb.ca> <38DE88DA.25044FA2@geoserve.net> <38DE90D7.AE208D29@hermes.usherb.ca>
Message-ID: <38DEFC32.D0548BF@casema.net>

> > (BTW, the argument that 'Dataflow' is redundant could also be applied to
> > 'Layer' ;-))
>
> Yes, what about:
>   -User Interface
>   -Definition
>   -Brokering
>   -Processing
>

OK, but inside the source code I'm in favoir of using the 'L' (UI,DL,BL & PL),
just to keep track of the function names, I get mad if I have to stare a few houres

at single character names.

>
> Since we all know they are all layers and that everything's about dataflow
> anyway.
>
> > There are also the 'foreign applications' and 'objects' to consider.  We could
> > add a...
> >
> >   OAL: Objects and Applications Layer
> I don't consider objects to be a layer... they're just objects (now that's new
> ;-) )

I agree, the objects are making up the layers.




From jarl at casema.net  Mon Mar 27 01:15:31 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net>
		 <38DE73CD.4505B80D@hermes.usherb.ca> <38DE88DA.25044FA2@geoserve.net> <38DE90D7.AE208D29@hermes.usherb.ca> <38DE952A.E3CCE58C@geoserve.net>
Message-ID: <38DEFC83.8BA603B5@casema.net>

>
> > I don't consider objects to be a layer... they're just objects (now that's new
> > ;-) )
>
> Technically, applications (and maybe objects) are not part of VSH.  But in
> Loci, they were called 'back-endware'.  These 'dataflow * layers' were
> collectively called 'middleware', and the user interfaces were called
> 'front-endware' in Loci.  I think these terms can still be used.
>

OK, but they are a step futher into detail as the 'layer view'.



From jean-marc.valin at hermes.usherb.ca  Mon Mar 27 01:22:49 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Naming of three layers
References: <200003261703.MAA80502@archa12.cc.uga.edu> <38DE4E33.4F59B7DC@casema.net>
 <38DE73CD.4505B80D@hermes.usherb.ca> <38DE88DA.25044FA2@geoserve.net> <38DE90D7.AE208D29@hermes.usherb.ca>
 <38DEFC32.D0548BF@casema.net>
Message-ID: <38DEFE39.9A5E3A86@hermes.usherb.ca>

> > >   OAL: Objects and Applications Layer
> > I don't consider objects to be a layer... they're just objects (now that's new
> > ;-) )
> 
> I agree, the objects are making up the layers.

Then, it would be "toolbox" (well, there might be a better name) layer, since
the objects can't be separated from their methods/algorithms. Then I could
consider my neural net class library (toolbox) to be part of this layer.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Mon Mar 27 23:49:34 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] overflow<->loci translation and 3 basic gui objects
Message-ID: <38E039DE.BFA7B00B@geoserve.net>

It's an understatement to say that VSH is a complex system.  But from the
user's point of view, it should be simple.  So, my motto for the development
of the VSH GUI is K.I.S.: Keep It Simple!

That said, I want define 3 basic objects for the GUI.  I started out with 4
under Loci:

  (1) Workspace
  (2) Locus (node)
  (3) Connector
  (4) Windowlet

But thinking about this more recently, I believe (1) and (4) are the same. 
Afterall, workspaces are always in windowlets (windows) and windows always
contain workspaces.

So, how about this.  The 3 basic GUI objects:

  (1) Window
  (2) Locus (node)
  (3) Connector

(Note: a 'network' is a collection of nodes and probably shouldn't be
considered a separate object by the GUI.)

This way we have wonderful nesting and recursion: Windows contain nodes, nodes
contain connectors and windows, windows contain nodes, etc., etc.

Okay, now this brings me to how objects are named in both Overflow and Loci. 
As we have seen, both applications are remarkably similar, varying (so it
seems) mostly by how things are named.  Here is a quick translation between
the two:

  Overflow             Loci
  --------             --------
  Document             Windowlet/workspace
  Network/subnet       Workflow diagram
  Node                 Locus
  Link                 Connector

We need to standardize on these names.  I can settle with using Overflow's
terminology in every case except for 'document'.  Since VSH will be dealing
with large text files often called 'documents', I think that name is
confusing.  So, I propose we (including Overflow) change this to something
else.

Since the Loci equivalent to 'document' is 'windowlet' (and 'workspace'), and
since I mention above those two can be considered the same object, I propose
'document' be changed to 'window'.

Also, I'm not sure whether I like 'link' or 'connector' better.  What do you
guys like?  We could also consider 'line' and 'branch'.

So, maybe VSH could use the following (basic and collective) object names:

  Overflow             Loci                   VSH
  --------             --------               ---------
  Document             Windowlet/workspace    Window
  Network/subnet       Workflow diagram       Network
  Node                 Locus                  Node
  Link                 Connector              Link (or connector)

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 00:13:27 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net>
Message-ID: <38E03F77.6751C58F@geoserve.net>

"J.W. Bizzaro" wrote:
>
> So, how about this.  The 3 basic GUI objects:
> 
>   (1) Window
>   (2) Locus (node)
>   (3) Connector

Also, in GUI terms, the node is an icon.

You know, if we really want to go wacky, the core can define 3 basic objects:

  (1) Network
  (2) Node
  (3) Link

Networks contain nodes, nodes contain links and networks, networks contain
nodes, nodes contain links and networks, networks contain nodes, nodes contain
links and networks, networks contain nodes, nodes contain links and networks,
networks contain nodes, nodes contain links and networks, networks contain
nodes, nodes contain links and networks, networks contain nodes, hahaha!

So, in Overflow, 'document' can be replaced with 'node', no?

<?xml version="1.0"?>
<Node>
  <Network type="subnet" name="MAIN">
    <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
      <Parameter name="VALUE" type="subnet_param" value="IN"/>
    </Node>
    <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
      <Parameter name="FRAME_SIZE" type="int" value="256"/>
    </Node>
    <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
      <Parameter name="COMMAND" type="string" value="cat"/>
    </Node>
    <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
      <Parameter name="VALUE" type="string" value="frames"/>
    </Node>
    <Node name="node8" type="OutputStream" x="43.000000" y="190.000000"/>
    <Node name="node9" type="Save" x="243.000000" y="119.000000"/>
    <Link from="node2" output="OUTPUT" to="node5" input="INPUT"/>
    <Link from="node1" output="VALUE" to="node2" input="INPUT"/>
    <Link from="node6" output="VALUE" to="node8" input="INPUT"/>
    <Link from="node5" output="TRAIN_IN" to="node9" input="OBJECT"/>
    <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>
    <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>
  </Network>
  <Network type="iterator" name="MAIN_LOOP">
    <Node name="node1" type="AudioStream" x="-526.000000" y="93.000000">
      <Parameter name="LENGTH" type="subnet_param" value="FRAME_SIZE"/>
      <Parameter name="ENCODING" type="string" value="LIN16"/>
      <Parameter name="STREAM_TYPE" type="string" value="FILE"/>
    </Node>
    <Node name="node2" type="NotDone" x="-344.000000" y="45.000000"/>
    <Node name="node10" type="FrameAccumulator" x="-336.000000"
y="131.000000">
      <Parameter name="LENGTH" type="int" value="256"/>
      <Parameter name="CACHEALL" type="int" value="1"/>
    </Node>
    <Link from="node1" output="OUTPUT" to="node2" input="INPUT"/>
    <Link from="node1" output="OUTPUT" to="node10" input="INPUT"/>
    <NetInput name="INPUT" node="node1" terminal="INPUT"/>
    <NetCondition name="CONDITION" node="node2" terminal="OUTPUT"/>
    <NetOutput name="TRAIN_IN" node="node10" terminal="OUTPUT"/>
  </Network>
  <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>
</Node>

Why not?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 00:12:56 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net>
Message-ID: <38E03F58.BBBD4343@hermes.usherb.ca>

> It's an understatement to say that VSH is a complex system.  But from the
> user's point of view, it should be simple.  So, my motto for the development
> of the VSH GUI is K.I.S.: Keep It Simple!

Well, maybe we should have more than one GUI, or at least a GUI with a couple
"modes". There are many applications that require a different interface:
 -a GUI equivalent to something like "ls -l | awk '{print $2}' | sed 's/aa/bb/'
| sort -u > somewhere" (a la Loci)
 -a system to run complex distributed tasks (a la GMS)
 -a signal processing framework a la Matlab/Simulink/Lab View (a la Overflow)

I don't see how the same interface can handle all those correctly.

> 
> That said, I want define 3 basic objects for the GUI.  I started out with 4
> under Loci:
> 
>   (1) Workspace
>   (2) Locus (node)
>   (3) Connector
>   (4) Windowlet
> 
> But thinking about this more recently, I believe (1) and (4) are the same.
> Afterall, workspaces are always in windowlets (windows) and windows always
> contain workspaces.
> 
> So, how about this.  The 3 basic GUI objects:
> 
>   (1) Window
>   (2) Locus (node)
>   (3) Connector
> 
> (Note: a 'network' is a collection of nodes and probably shouldn't be
> considered a separate object by the GUI.)

Well, a network would be the equivalent of a "group" in a vector drawing app.

> 
> This way we have wonderful nesting and recursion: Windows contain nodes, nodes
> contain connectors and windows, windows contain nodes, etc., etc.
> 
> Okay, now this brings me to how objects are named in both Overflow and Loci.
> As we have seen, both applications are remarkably similar, varying (so it
> seems) mostly by how things are named.  Here is a quick translation between
> the two:
> 
>   Overflow             Loci
>   --------             --------
>   Document             Windowlet/workspace
>   Network/subnet       Workflow diagram
>   Node                 Locus
>   Link                 Connector

I prefer workspace de document (which is too generic). I prefer Node though, a
month ago I had never heard the word Loci/Locus (sorry, engligh is my second
language). I don't mind Link of Connector (link is shorter, bout that's all)...
pipe, maybe?..

As for Network... we have to be careful... I Overflow (though it's not
implemented exactly that way), you can see "Network" as a base class and
"subnet", "iterator" and "threadedIterator" (possibly more later) as subclasses
of Network.


> 
> We need to standardize on these names.  I can settle with using Overflow's
> terminology in every case except for 'document'.  Since VSH will be dealing
> with large text files often called 'documents', I think that name is
> confusing.  So, I propose we (including Overflow) change this to something
> else.
> 
> Since the Loci equivalent to 'document' is 'windowlet' (and 'workspace'), and
> since I mention above those two can be considered the same object, I propose
> 'document' be changed to 'window'.
> 
> Also, I'm not sure whether I like 'link' or 'connector' better.  What do you
> guys like?  We could also consider 'line' and 'branch'.
> 
> So, maybe VSH could use the following (basic and collective) object names:
> 
>   Overflow             Loci                   VSH
>   --------             --------               ---------
>   Document             Windowlet/workspace    Window
>   Network/subnet       Workflow diagram       Network
>   Node                 Locus                  Node
>   Link                 Connector              Link (or connector)
> 

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Tue Mar 28 00:20:38 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
			 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
			 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
			 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca> <38DC484E.781D6528@geoserve.net> <38DC9F81.5431E931@casema.net>
Message-ID: <38E04126.BA262981@geoserve.net>

jarl van katwijk wrote:
> 
> > Jean-Marc Valin wrote:
> > >
> > > OK, I'll try to summerize the way XML is used in Overflow with an example:
> >
> > Brad and Jarl, perhaps GMS can reuse much of Overflow's code for handling XML
> > and network construction?
> 
> As said before, gms will just act passive on most of the XML descriptions.

I don't know about that.  GMS will need to construct, parse and execute
Internet-distributed networks just as Overflow does with local networks.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 00:29:32 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net>
Message-ID: <38E0433C.2B97AB08@hermes.usherb.ca>

> So, in Overflow, 'document' can be replaced with 'node', no?

No. You're confusing two things here: in C we's say "function call" vs "function
implementation". The <Node> tag refers to a function call (a node
instantiation), while the <Network> or <Document> can be viewed as a function
implementation (node definition) from other nodes. 


> 
> <?xml version="1.0"?>
> <Node>
>   <Network type="subnet" name="MAIN">
>     <Node name="node1" type="Constant" x="-294.000000" y="101.000000">
>       <Parameter name="VALUE" type="subnet_param" value="IN"/>
>     </Node>
>     <Node name="node5" type="MAIN_LOOP" x="-26.000000" y="108.000000">
>       <Parameter name="FRAME_SIZE" type="int" value="256"/>
>     </Node>
>     <Node name="node2" type="ExecStream" x="-165.000000" y="98.000000">
>       <Parameter name="COMMAND" type="string" value="cat"/>
>     </Node>
>     <Node name="node6" type="Constant" x="-162.000000" y="186.000000">
>       <Parameter name="VALUE" type="string" value="frames"/>
>     </Node>
>     <Node name="node8" type="OutputStream" x="43.000000" y="190.000000"/>
>     <Node name="node9" type="Save" x="243.000000" y="119.000000"/>
>     <Link from="node2" output="OUTPUT" to="node5" input="INPUT"/>
>     <Link from="node1" output="VALUE" to="node2" input="INPUT"/>
>     <Link from="node6" output="VALUE" to="node8" input="INPUT"/>
>     <Link from="node5" output="TRAIN_IN" to="node9" input="OBJECT"/>
>     <Link from="node8" output="OUTPUT" to="node9" input="STREAM"/>
>     <NetOutput name="OUTPUT" node="node9" terminal="OUTPUT"/>
>   </Network>
>   <Network type="iterator" name="MAIN_LOOP">
>     <Node name="node1" type="AudioStream" x="-526.000000" y="93.000000">
>       <Parameter name="LENGTH" type="subnet_param" value="FRAME_SIZE"/>
>       <Parameter name="ENCODING" type="string" value="LIN16"/>
>       <Parameter name="STREAM_TYPE" type="string" value="FILE"/>
>     </Node>
>     <Node name="node2" type="NotDone" x="-344.000000" y="45.000000"/>
>     <Node name="node10" type="FrameAccumulator" x="-336.000000"
> y="131.000000">
>       <Parameter name="LENGTH" type="int" value="256"/>
>       <Parameter name="CACHEALL" type="int" value="1"/>
>     </Node>
>     <Link from="node1" output="OUTPUT" to="node2" input="INPUT"/>
>     <Link from="node1" output="OUTPUT" to="node10" input="INPUT"/>
>     <NetInput name="INPUT" node="node1" terminal="INPUT"/>
>     <NetCondition name="CONDITION" node="node2" terminal="OUTPUT"/>
>     <NetOutput name="TRAIN_IN" node="node10" terminal="OUTPUT"/>
>   </Network>
>   <Parameter name="IN" type="string" value="/data/ntt/french/*/*"/>
> </Node>
> 
> Why not?
> 
> Jeff
> --
>                       +----------------------------------+
>                       |           J.W. Bizzaro           |
>                       |                                  |
>                       | http://bioinformatics.org/~jeff/ |
>                       |                                  |
>                       |        BIOINFORMATICS.ORG        |
>                       |           The Open Lab           |
>                       |                                  |
>                       |    http://bioinformatics.org/    |
>                       +----------------------------------+
> 
> _______________________________________________
> pipet-devel maillist  -  pipet-devel@bioinformatics.org
> http://bioinformatics.org/mailman/listinfo/pipet-devel

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From bizzaro at geoserve.net  Tue Mar 28 00:52:56 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Idl
References: <200003251809.NAA124632@archa12.cc.uga.edu>
Message-ID: <38E048B8.8DAE0B9C@geoserve.net>

Brad Chapman wrote:
> 
> For the SUBNET interface:
> 
>     These are for accessing whole subnets or programs (so GMS type
> nodes) after they have been sent a workflow/xml_structure to process
> (ie. a whole bunch of connected nodes), right?
>     My question is, do we really need this level of control so that
> someone can
> manipulate a process so much while it is running? The way I was
> thinking of the GUI and processing engine working together are that
> the user spends some time getting a nice process of connected nodes
> set up, and then hits run and waits for the output to come out at the
> specific nodes/subnets they are interested in. If this is the case,
> then why and when would they need to access a process to remove,
> activate, destruct, add, etc. nodes and subnets?

I for one am very much in favor of this level of control.  As computational
biologists, we may be working on analyses involving terabytes of data and days
of calculations.  Should the network just sit there unusable during these
analyses?  Should entire jobs be scrapped if it turns out one node needs to be
moved?

I think this level of control is akin to the use of threading in an
application.  If threading is not used for compute-intensive tasks, the GUI
freezes up and the application is unusable.  Sure, it's still running, and you
can always stop it.  But the loss in usability is both perceived by the user
and quite real.

>     Am I being naive here in thinking that we don't need all of this
> control of a running process? Perhaps what we do need though it a
> method for cancelling and run and getting as much information as you
> can from it...

Well, it may be simpler to not worry about it at this stage.

> For the following functions--do we need the processor to know about
> the description of the program? Can all of this type of information by
> stored in xml in the "middle scripting thingy"?

This is mostly useful to the user and should be kept as XML meta-data.  I
don't know what use the core has for it.

> For the following functions: I don't really understand the set status
> (how does the scripting engine set the status of a process?). I tried
> to include GETstatus type stuff in my query_process function. But
> maybe we should split this querying to be specific for a subnet or
> node?

The back-end or applications should set the status, which is reported back to
the user through the datastream.

> For the following, I'm not really positive why the middle would need
> to know this stuff? Do you want the GUI to restrict only allowed
> things to be connected? This seems *really* hard to maintain--maybe it
> would be better to have the processing engine generate errors and then
> propogate them back to the GUI so the user can fix them? The idl
> probably does need a better way to return back error messages.

Hmmm.  I have been in favor of the GUI allowing only compatible connections
(same data type).  If we assign types (e.g., mime) to data, we can determine
what will and won't work.

> For the following, do we still need the processing engine to store a
> log/history of events, or should this be handled by the middle
> scripting part, which can maintain logs for all its communication.

We can make a 'front' that records the datastream to a file.

>     I think the same kind of comments I made above apply here: how
> about if we just send all of this information to the middle as XML. I
> think that the current Overflow XML will be a good point to start with
> and we can work from there.

Okay by me.

> What I need to get working is a way to
> turn the directory/heirarchy based file storage system that Loci
> currently has into the type of flat XML that Overflow uses, so I can
> pass this. While the heirarchial structure is best for dealing with
> the GUI (since I have to make lots of changes to it as the GUI
> changes, it is easier to break it up into smaller chunks for
> manageability and speed), the flat file format is what we need so that
> we can pass a single file to the processing engine to process (and to
> stay compatible with what Overflow does since they have a good thing
> going).

You pretty much summed up Gary's and my own thoughts on this network structure
defined with XML.  A flat file is easy to store and pass but difficult to
query.  Breaking it up makes it easy to query but difficult to store and
pass.  The solution seemed to be the use of a real database.

> We also will not want to pass all of the info that is stored
> in XML in the middle (like descriptions of programs, etc.) to the
> processing engine, so some kind of transformation will be needed. I
> need to start trying to think about how to do this, but I hope it
> shouldn't be too bad...

EVERYTHING THAT IS NOT PASSED THROUGH THE INFRASTRUCTURE IS GIVEN A URI!  XML
descriptions of GUIs, libraries of widgets, etc. are not put into this
datastream but given a link so that the fronts and backs can get the stuff
themselves.  This way, the system remains neutral about protocol, and things
can be spread throughout the Internet.

Cheers.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 01:13:25 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca>
Message-ID: <38E04D85.811D236C@geoserve.net>

Jean-Marc Valin wrote:
> 
> Well, maybe we should have more than one GUI, or at least a GUI with a couple
> "modes". There are many applications that require a different interface:
>  -a GUI equivalent to something like "ls -l | awk '{print $2}' | sed 's/aa/bb/'
> | sort -u > somewhere" (a la Loci)
>  -a system to run complex distributed tasks (a la GMS)
>  -a signal processing framework a la Matlab/Simulink/Lab View (a la Overflow)
> 
> I don't see how the same interface can handle all those correctly.

Ahhh, but this is where we use XML to define GUIs!  As I mentioned earlier,
XML documents can be arranged according to the arrangement of nodes in a
network.  Then the XML can be translated into a Gtk interface using something
like libglade.

What I haven't brought up yet is that 'windows' can switch between views.  At
one moment, a window can show a network of nodes.  The next moment, this
window can be switched to show Gtk widgets arranged as I mention above.

> I prefer workspace de document (which is too generic). I prefer Node though, a
> month ago I had never heard the word Loci/Locus (sorry, engligh is my second
> language).

Loci/locus is actually a Latin word, meaning location :-)

> I don't mind Link of Connector (link is shorter, bout that's all)...
> pipe, maybe?..

Pipe may be confused with Unix pipes, which are local.  Link is okay, and is
reminiscent of HTML links.

> As for Network... we have to be careful... I Overflow (though it's not
> implemented exactly that way), you can see "Network" as a base class and
> "subnet", "iterator" and "threadedIterator" (possibly more later) as subclasses
> of Network.

So, what are nodes a subclass of?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 28 00:12:30 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net>
Message-ID: <38E03F3E.8DB57A50@casema.net>

>   So, my motto for the development
> of the VSH GUI is K.I.S.: Keep It Simple!

maybe Super Simpel :-)
but serious: good, and consistant naming is very important.

>   (1) Window
>   (2) Locus (node)
>   (3) Connector

What is the function of a 'connector'?

> Also, I'm not sure whether I like 'link' or 'connector' better.  What do you
> guys like?  We could also consider 'line' and 'branch'.

So a 'link'\'connector' is a relation between nodes?

>
>
> So, maybe VSH could use the following (basic and collective) object names:
>
>   Overflow             Loci                   VSH
>   --------             --------               ---------
>   Document             Windowlet/workspace    Window
>   Network/subnet       Workflow diagram       Network
>   Node                 Locus                  Node
>   Link                 Connector              Link (or connector)

What is the diff. between a Window and a Network?




From bizzaro at geoserve.net  Tue Mar 28 01:17:05 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
Message-ID: <38E04E61.B2634CC5@geoserve.net>

Jean-Marc Valin wrote:
> 
> No. You're confusing two things here: in C we's say "function call" vs "function
> implementation". The <Node> tag refers to a function call (a node
> instantiation), while the <Network> or <Document> can be viewed as a function
> implementation (node definition) from other nodes.

You can't say the <Node> tag refers to BOTH function calls AND
implementations, to make it more recursive?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 01:18:52 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E04D85.811D236C@geoserve.net>
Message-ID: <38E04ECC.7B84B42D@hermes.usherb.ca>

> So, what are nodes a subclass of?

The Node class doesn't have any parent, though it might inherit from Object if
we need to (probably). ...and Network inherits from Node.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 01:22:11 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
 <38E04E61.B2634CC5@geoserve.net>
Message-ID: <38E04F93.C45AC9FE@hermes.usherb.ca>

> > No. You're confusing two things here: in C we's say "function call" vs "function
> > implementation". The <Node> tag refers to a function call (a node
> > instantiation), while the <Network> or <Document> can be viewed as a function
> > implementation (node definition) from other nodes.
> 
> You can't say the <Node> tag refers to BOTH function calls AND
> implementations, to make it more recursive?

If you can find a way to do that in C, you have found how to do it with
Overflow. If not, forget it :-). I don't see how, it's really two different
things... If I had to replace Document by something existing, I'd choose Network
(which, remember, derives from Node!).

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01@gel.usherb.ca


From jarl at casema.net  Tue Mar 28 00:29:51 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net>
Message-ID: <38E0434F.F5096E45@casema.net>

>
> Also, in GUI terms, the node is an icon.
>
> You know, if we really want to go wacky, the core can define 3 basic objects:
>
>   (1) Network
>   (2) Node
>   (3) Link
>

1) Strictly technical a 'network' defines a 'border' around a set of nodes. Every
node
and link within this border has the same brokering characteristics: all is local,
all
has the same security, the same accessability, etc.
2) From the users point of view this definition of a network doesn't apply: a
user
defines the network by 'functionality' or by 'a project'.

You people have 2) in mind dont you?




From bizzaro at geoserve.net  Tue Mar 28 01:35:11 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui
 objects
 objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E04D85.811D236C@geoserve.net> <38E04ECC.7B84B42D@hermes.usherb.ca>
Message-ID: <38E0529F.4CEED298@geoserve.net>

Jean-Marc Valin wrote:
> 
> > So, what are nodes a subclass of?
> 
> The Node class doesn't have any parent, though it might inherit from Object if
> we need to (probably). ...and Network inherits from Node.

Maybe I should say 'subset' rather than 'subclass', so that I'm not confusing
OO terminology with ordinary logic.

This is what I'm trying to saying in logical terms:

  The Overflow way:

      'Documents' contain networks
      Networks are of types subnet and iterator
      Networks contain nodes and links

  But can't we say...

      Nodes contain links and networks
      Networks are of types subnet and iterator
      Networks contain nodes
      Nodes contain links and networks
      ...


Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 28 00:32:41 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
				 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
				 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
				 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca> <38DC484E.781D6528@geoserve.net> <38DC9F81.5431E931@casema.net> <38E04126.BA262981@geoserve.net>
Message-ID: <38E043F9.8F59E6F7@casema.net>

> > >
> > > Brad and Jarl, perhaps GMS can reuse much of Overflow's code for handling XML
> > > and network construction?
> >
> > As said before, gms will just act passive on most of the XML descriptions.
>
> I don't know about that.  GMS will need to construct, parse and execute
> Internet-distributed networks just as Overflow does with local networks.
>

Yes, but the BL (former gms) will only detect the 'borders', based on
accesability\security
and locallity. All other 'detail' XML code wont be parsed, it will be passes to the
PL.



From bizzaro at geoserve.net  Tue Mar 28 01:43:15 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F3E.8DB57A50@casema.net>
Message-ID: <38E05483.20A45B19@geoserve.net>

jarl van katwijk wrote:
>
> > Also, I'm not sure whether I like 'link' or 'connector' better.  What do you
> > guys like?  We could also consider 'line' and 'branch'.
> 
> So a 'link'\'connector' is a relation between nodes?

In Loci terminology, a connector=link is passage for data to flow between
nodes.  It also indicates relations, such as which runs first and parent-child
relations.

Maybe it means something different in Overflow terminology.  Jean-Marc?

> > So, maybe VSH could use the following (basic and collective) object names:
> >
> >   Overflow             Loci                   VSH
> >   --------             --------               ---------
> >   Document             Windowlet/workspace    Window
> >   Network/subnet       Workflow diagram       Network
> >   Node                 Locus                  Node
> >   Link                 Connector              Link (or connector)
> 
> What is the diff. between a Window and a Network?

Okay, these are PURELY GUI TERMS.  A window is pretty much an X-Window that
gives you a view of the network or networks.  They can't be the same, because
they look very much different to the user.  Plus, a window can contain more
than one network.

But I think the core can combine window and node.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 01:48:39 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E0418D.A96D3D63@casema.net>
Message-ID: <38E055C7.ECE117C0@geoserve.net>

jarl van katwijk wrote:
> 
> Have different modes like you suggest:
> - normal mode: editing nodes and 'relations'
> - access mode: editing the security\accessabilities (of the brokering level)
> - detail mode: editing the PL (overflow) nodes

I see 3 modes/views:

  (1) Network normal view: Nodes and relations are visible at
      full size (icons and subwindows shown)
  (2) Network world view: Nodes and relations are visible and
      fit entirely in the window (icons and subwindows not shown)
  (3) GUI view: Gtk widgets from all nodes are visible

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 01:51:18 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0434F.F5096E45@casema.net>
Message-ID: <38E05666.499068C@geoserve.net>

jarl van katwijk wrote:
> 
> 2) From the users point of view this definition of a network doesn't apply: a
> user defines the network by 'functionality' or by 'a project'.
> 
> You people have 2) in mind dont you?

I suppose 2).  But I'd define a network as simply 'a contiguously connected
collection of nodes'.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 28 00:49:30 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:39 2006
Subject: [Pipet Devel] Idl
References: <200003251809.NAA124632@archa12.cc.uga.edu> <38E048B8.8DAE0B9C@geoserve.net>
Message-ID: <38E047E9.1DCA2937@casema.net>

> >     My question is, do we really need this level of control so that
> > someone can manipulate a process so much while it is running?
>
> I for one am very much in favor of this level of control.  As computational
> biologists, we may be working on analyses involving terabytes of data and days
> of calculations.  Should the network just sit there unusable during these
> analyses?  Should entire jobs be scrapped if it turns out one node needs to be
> moved?
>
> I think this level of control is akin to the use of threading in an
> application.  If threading is not used for compute-intensive tasks, the GUI
> freezes up and the application is unusable.  Sure, it's still running, and you
> can always stop it.  But the loss in usability is both perceived by the user
> and quite real.
>

For all non corba techies: the DL (or gui) is corba based, this means the DL runs

as a seperate application, and never freeses. Calling a corba function will
always
have a response, so it is the non-threaded corba server (the BL) that would
freeze. This freeze will only take as long as the processing that corba call will
take,
being a few nano seconds.

>
> > For the following functions--do we need the processor to know about
> > the description of the program? Can all of this type of information by
> > stored in xml in the "middle scripting thingy"?
>
> This is mostly useful to the user and should be kept as XML meta-data.  I
> don't know what use the core has for it.

The 'description' is merly a identification for the security meganism.

>
> The back-end or applications should set the status, which is reported back to
> the user through the datastream.

Also. But the DL must be able to modify the status of a node\subnet too.
This wont happen too often, but things like renicing or stalling will be needed.

>
> Hmmm.  I have been in favor of the GUI allowing only compatible connections
> (same data type).  If we assign types (e.g., mime) to data, we can determine
> what will and won't work.

The BL will be able to marchall (data conversion). This wont conflict with the
type-casting
the PL has, the marchalling will be like manipulating High-endian\Low-endian
formatting,
plain-encrypted formats, etc.

>
> > For the following, do we still need the processing engine to store a
> > log/history of events, or should this be handled by the middle
> > scripting part, which can maintain logs for all its communication.
>
> We can make a 'front' that records the datastream to a file.

The BL (gms) already does this (see messaging_mobject_structs.h -> histryIO)
But logging the PL nodes wont be touched by this. This should be optional,
because logging every node will have huge CPU load.




From bizzaro at geoserve.net  Tue Mar 28 01:53:44 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: Ideas for network distributed objects
References: <Pine.OSF.4.03.10003162331320.23260-100000@busboy.sped.ukans.edu> <38D7AA11.3796AA@geoserve.net>
					 <38D88B34.69264157@casema.net> <38D8DB07.46B392E8@geoserve.net> <38D9C34F.83A85DCF@casema.net>
					 <38DADC83.F000D7B@geoserve.net> <38DB172F.E77443B6@casema.net> <38DB1935.E2A18780@hermes.usherb.ca>
					 <38DBC301.819DF35A@casema.net> <38DBEA04.C5CBD3BD@geoserve.net> <38DC0B32.F42D6405@hermes.usherb.ca> <38DC484E.781D6528@geoserve.net> <38DC9F81.5431E931@casema.net> <38E04126.BA262981@geoserve.net> <38E043F9.8F59E6F7@casema.net>
Message-ID: <38E056F8.FFA5A9FB@geoserve.net>

jarl van katwijk wrote:
> 
> Yes, but the BL (former gms) will only detect the 'borders', based on
> accesability\security
> and locallity. All other 'detail' XML code wont be parsed, it will be passes to the
> PL.

And the PL is....Overflow?  What precisely will be constructing and executing
this Internet-based network?  Is this Brad's DL?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 01:58:45 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E0418D.A96D3D63@casema.net> <38E055C7.ECE117C0@geoserve.net>
Message-ID: <38E05825.851881C1@geoserve.net>

"J.W. Bizzaro" wrote:
> 
> > Have different modes like you suggest:
> > - normal mode: editing nodes and 'relations'
> > - access mode: editing the security\accessabilities (of the brokering level)
> > - detail mode: editing the PL (overflow) nodes

I actually want to try and hide these 'modes' from the user.  They should be
indistinguishable from each other.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 02:01:00 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
	 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca>
Message-ID: <38E058AC.26F3FA21@geoserve.net>

Jean-Marc Valin wrote:
> 
> If you can find a way to do that in C, you have found how to do it with
> Overflow. If not, forget it :-). I don't see how, it's really two different
> things... If I had to replace Document by something existing, I'd choose Network
> (which, remember, derives from Node!).

I didn't realize Overflow had that limitation.  Perhaps then the GMS layer can
provide the recursion?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 02:08:42 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Idl
References: <200003251809.NAA124632@archa12.cc.uga.edu> <38E048B8.8DAE0B9C@geoserve.net> <38E047E9.1DCA2937@casema.net>
Message-ID: <38E05A7A.6B78395A@geoserve.net>

jarl van katwijk wrote:
> 
> > This is mostly useful to the user and should be kept as XML meta-data.  I
> > don't know what use the core has for it.
> 
> The 'description' is merly a identification for the security meganism.

Alright, I was talking about a 'GUI description' :-)

> > The back-end or applications should set the status, which is reported back to
> > the user through the datastream.
> 
> Also. But the DL must be able to modify the status of a node\subnet too.
> This wont happen too often, but things like renicing or stalling will be needed.

However it happens, I'd like the core to be continuously reporting status
information back to the GUI.  I'll make some little (animated?) indicators to
show the user the status of each node.

> > Hmmm.  I have been in favor of the GUI allowing only compatible connections
> > (same data type).  If we assign types (e.g., mime) to data, we can determine
> > what will and won't work.
> 
> The BL will be able to marchall (data conversion). This wont conflict with the
> type-casting the PL has, the marchalling will be like manipulating
> High-endian\Low-endian formatting, plain-encrypted formats, etc.

Will the core be able to say 'yes, this connection will work' and 'no, this
connection won't work'?

> > We can make a 'front' that records the datastream to a file.
> 
> The BL (gms) already does this (see messaging_mobject_structs.h -> histryIO)
> But logging the PL nodes wont be touched by this. This should be optional,
> because logging every node will have huge CPU load.

Oh, okay.  Yes, very much a resource hog.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 02:22:42 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F58.BBBD4343@hermes.usherb.ca> <38E0418D.A96D3D63@casema.net> <38E055C7.ECE117C0@geoserve.net> <38E04920.78735DA3@casema.net>
Message-ID: <38E05DC2.BB77F60D@geoserve.net>

jarl van katwijk wrote:
> 
> >   (1) Network normal view: Nodes and relations are visible at
> >       full size (icons and subwindows shown)
> >   (2) Network world view: Nodes and relations are visible and
> >       fit entirely in the window (icons and subwindows not shown)
> >   (3) GUI view: Gtk widgets from all nodes are visible
> 
> The "GTk widget" from a node?
> A node has a gtk-widget or is a gtk-widget 'created' by the gui to visualise it?

Each node has user-settable parameters (see what pops up when you double-click
on an Overflow node).  As I mentioned before, these parameters are
'represented by XML' tags and translated into Gtk widgets so that the
parameters can be set.  This XML makes up the 'XML representation of a GUI'.

The XML can then be arranged according to the arrangement of nodes in a
network.  The result is a 'composite' GUI.

This seems to be difficult for everyone to grasp.  I'm not sure anyone on the
Loci list understood me either, except maybe Gary.  Gary, can you explain this
differently?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 02:27:16 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
			 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E058AC.26F3FA21@geoserve.net> <38E04BFF.85220082@casema.net>
Message-ID: <38E05ED4.2DA647A5@geoserve.net>

jarl van katwijk wrote:
> 
> > > If you can find a way to do that in C, you have found how to do it with
> > > Overflow. If not, forget it :-). I don't see how, it's really two different
> > > things... If I had to replace Document by something existing, I'd choose Network
> > > (which, remember, derives from Node!).
> >
> > I didn't realize Overflow had that limitation.  Perhaps then the GMS layer can
> > provide the recursion?
> 
> Uhh.. sorry, dont get it what this 'call\implementation' is about?
> Somebody could explain?

In procedural languages you define a procedure with its parameters.  This is
what Jean-Marc means by 'implementation'.  Then somewhere else in the program
you give the procedure 'call'.

But Jean-Marc, you can call a procedure from within its implementation, giving
you some recursion, right?  Can Overflow do that?

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Tue Mar 28 02:36:41 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
	 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca>
Message-ID: <38E06109.7F52F78E@geoserve.net>

Jean-Marc Valin wrote:
> 
> If you can find a way to do that in C, you have found how to do it with
> Overflow. If not, forget it :-). I don't see how, it's really two different
> things... If I had to replace Document by something existing, I'd choose Network
> (which, remember, derives from Node!).

Then how about this:

<?xml version="1.0"?>
<GMSNode>
  <OverflowNetwork type="subnet" name="MAIN">
    <OverflowNode name="node1" type="Constant" x="-294.000000" y="101.000000">
      <OverflowParameter name="VALUE" type="subnet_param" value="IN"/>
    </OverflowNode>
    ...
    <OverflowLink from="node2" output="OUTPUT" to="node5" input="INPUT"/>
    ...
  </OverflowNetwork>
  <OverflowParameter name="IN" type="string" value="/data/ntt/french/*/*"/>
</GMSNode>

Or what would these be called in the newspeak?  BLNode, PLNode?

There will be SOME differences between the 2 types of networks, so we probably
shouldn't use the same words/tags for both.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Tue Mar 28 02:21:48 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Idl
References: <200003251809.NAA124632@archa12.cc.uga.edu> <38E048B8.8DAE0B9C@geoserve.net> <38E047E9.1DCA2937@casema.net> <38E05A7A.6B78395A@geoserve.net>
Message-ID: <38E05D8C.B6EDD876@casema.net>

>
> However it happens, I'd like the core to be continuously reporting status
> information back to the GUI.  I'll make some little (animated?) indicators to
> show the user the status of each node.

No problem, in the IDL there's this function :
    statusT uriStatus(in uriT uri, in DLIdT dlid, in passwordT password)

> Will the core be able to say 'yes, this connection will work' and 'no, this
> connection won't work'?

No. Neither will it ruin data transactions that are incompatible, it will marshal
them automatically. But this surly does have it's limitations, we'll have to work
on what will and what wont be converted by the BL and PL. This is something
which hasn't high priority right now.



From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 08:53:06 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
				 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E058AC.26F3FA21@geoserve.net> <38E04BFF.85220082@casema.net> <38E05ED4.2DA647A5@geoserve.net>
Message-ID: <38E0B942.85996C69@hermes.usherb.ca>

"J.W. Bizzaro" a ?crit :
> 
> jarl van katwijk wrote:
> >
> > > > If you can find a way to do that in C, you have found how to do it with
> > > > Overflow. If not, forget it :-). I don't see how, it's really two different
> > > > things... If I had to replace Document by something existing, I'd choose Network
> > > > (which, remember, derives from Node!).
> > >
> > > I didn't realize Overflow had that limitation.  Perhaps then the GMS layer can
> > > provide the recursion?
> >
> > Uhh.. sorry, dont get it what this 'call\implementation' is about?
> > Somebody could explain?
> 
> In procedural languages you define a procedure with its parameters.  This is
> what Jean-Marc means by 'implementation'.  Then somewhere else in the program
> you give the procedure 'call'.
> 
> But Jean-Marc, you can call a procedure from within its implementation, giving
> you some recursion, right?  Can Overflow do that?

Yes, you can do that in Overflow. There are two ways to do it, but you don't
need to have <Node> mean two different things. You can see in TimeFilter.cc that
the getOutput method calls this->getOutput() so the output is the node is fed
back in the node. I'd call this implicit recursion. The other way is not fully
implemented yet. When it is, you'll be able to explicitly connect the output of
a node to its input (or the input of a node "upstream"). The overflow model
already supports that, but for now, the network initialization will fail.

Also, about node parameters. I think there's a little misunderstanding. The
<Parameter> tag in the <Node> doesn't say which parameters are available, but
which argument is passed to the node. The parameter "declarations" are in two
places:

-For built-in nodes (.cc files), the parameters are declared as an argument to
NODE_INFO on top of the file.
-For networks (that can be used as nodes), every time there's an argument of
type "subnet_param" for a node in that network, the network will have this
parameter. example:

<Network name="foo" type="subnet">
   <Node name="bar" type="SomeNode">
      <Parameter name="param1" type="Int" value="1"/>
      <Parameter name="param2" type="subnet_param" value="myParam"/>
   </Node>
</Network>

Here, when including network "foo" as a node, you will need to provide a
parameter named "myParam" that will be substituted as "param2" in node bar.

	Jean-Marc


From jean-marc.valin at hermes.usherb.ca  Tue Mar 28 09:04:11 2000
From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
		 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E06109.7F52F78E@geoserve.net>
Message-ID: <38E0BBDB.E94607F4@hermes.usherb.ca>

> <?xml version="1.0"?>
> <GMSNode>
>   <OverflowNetwork type="subnet" name="MAIN">
>     <OverflowNode name="node1" type="Constant" x="-294.000000" y="101.000000">
>       <OverflowParameter name="VALUE" type="subnet_param" value="IN"/>
>     </OverflowNode>
>     ...
>     <OverflowLink from="node2" output="OUTPUT" to="node5" input="INPUT"/>
>     ...
>   </OverflowNetwork>
>   <OverflowParameter name="IN" type="string" value="/data/ntt/french/*/*"/>
> </GMSNode>

You can't say GMSNode, since this could also be included as an Overflow Node.
BTW, the reason for "Document" is that XML requires ONE toplevel tag and since I
can have many networks in my file, I used Document, which only refers to the
fact that everything is in a file. But, hat about using the same type of things
that Pascal programs have: Nested functions. You'd have a toplevel network,
which would contain other networks...

<Network "name....>
   <Node ...>
   </Node>
   .
   .
   <Network ....>
      <Network...
      </Network>
   ..
   </Network>
</Network>

	JEan-Marc


From jarl at casema.net  Tue Mar 28 08:31:04 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <38E039DE.BFA7B00B@geoserve.net> <38E03F77.6751C58F@geoserve.net> <38E0433C.2B97AB08@hermes.usherb.ca>
			 <38E04E61.B2634CC5@geoserve.net> <38E04F93.C45AC9FE@hermes.usherb.ca> <38E06109.7F52F78E@geoserve.net> <38E0BBDB.E94607F4@hermes.usherb.ca>
Message-ID: <38E0B418.8C7E04B4@casema.net>

>
> You can't say GMSNode, since this could also be included as an Overflow Node.

Yes, do not take notice of the BL while defining the XML structure, it will
detect\parse
tags like <Network>. Just ignore the brokering layer.

>
> BTW, the reason for "Document" is that XML requires ONE toplevel tag and since I
> can have many networks in my file, I used Document, which only refers to the
> fact that everything is in a file. But, hat about using the same type of things
> that Pascal programs have: Nested functions. You'd have a toplevel network,
> which would contain other networks...

The brokering layer will come heavy into play during runtime when these nested
structures are setup, the BL will cut them into pieces, or brokering nodes.




From gvd at martin.pharmacy.ualberta.ca  Tue Mar 28 15:49:16 2000
From: gvd at martin.pharmacy.ualberta.ca (Gary Van Domselaar)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
In-Reply-To: <38E05DC2.BB77F60D@geoserve.net> from "J.W. Bizzaro" at Mar 28, 2000 07:22:42 AM
Message-ID: <200003282049.NAA32328@martin.pharmacy.ualberta.ca>

> 
> jarl van katwijk wrote:
> > 
> > >   (1) Network normal view: Nodes and relations are visible at
> > >       full size (icons and subwindows shown)
> > >   (2) Network world view: Nodes and relations are visible and
> > >       fit entirely in the window (icons and subwindows not shown)
> > >   (3) GUI view: Gtk widgets from all nodes are visible
> > 
> > The "GTk widget" from a node?
> > A node has a gtk-widget or is a gtk-widget 'created' by the gui to visualise it?
> 
> Each node has user-settable parameters (see what pops up when you double-click
> on an Overflow node).  As I mentioned before, these parameters are
> 'represented by XML' tags and translated into Gtk widgets so that the
> parameters can be set.  This XML makes up the 'XML representation of a GUI'.
> 
> The XML can then be arranged according to the arrangement of nodes in a
> network.  The result is a 'composite' GUI.
> 
> This seems to be difficult for everyone to grasp.  I'm not sure anyone on the
> Loci list understood me either, except maybe Gary.  Gary, can you explain this
> differently?

Sure.  There are three basic 'interface' concepts here.  The first concept deals with
dynamically obtaining and generating an interface from 'back-end' application or object. This
interface is for supplying the user definable options to the back end during execution.
The interface is supplied (from Overflow) as a high-level XML document to the client/user
interface. Of course, the XML contains extra information that may not be seen or manipulated
by the user, but is important in describing the application or object. But I digress. 
Assuming a Gtk/GNOME-based client, the 'application' would appear as an icon. Double-clicking
the icon reveals its interface for adjusting options. The client communicates this information
to the DL, which updates that application's XML representation, and stores it in a database.

The second concept concerns the connections/links made from one application/node/icon to
another.  The user creates a 'network' by connecting the applications together. The XML
representations get updated by the DL to reflect their relationships. In the end you have a
number of small XML files, one per application. The connectivity information is included a)in
the XML files, or b) in a 'parent' file (also XML).  I prefer b) because it aids in nesting
networks, as described below.

The third concept concerns nesting of 'networks'.  This is what Jeff refers to as a 'composite
locus'.  Essentially the idea is that once a network is created, it would be nice to treat the
network as a 'node', considering that once the options have been set, so it is essentially
'static' at this point: it need only have its input and output defined.  The 'network' can be
encapsulated, and treated like an elementary node. The user can create place this (sub)network
into a new new network. The icon that represents this network can be double clicked to reveal
the subnetwork.  If we used strategy b) to store our node relationships, then the XML that
represents this network would be contained in this parent file. Incidentally X,Y coordinates
for the icons/nodes could also be stored in this parent file, thus preserving the original
layout of the network. It would be faster, too, because you wouldn't have to query the XML
file associated with each node in the network for connectivity information.

In this fashion, networks can be infinitely nested, and the XML representation of the network
would be fast, manageable, and easy to parse by the DL for execution.



At least, that's how I understand it. Jeff?


I understand that these concepts are heavily user-and-XML-representation-oriented, and may not
jive exactly with the architectures of Overflow or GMS. But I hope it clarifies a bit the
Loci 'grand scheme'. 

Regards,

g.
--
                              Gary Van Domselaar
                           gary@bioinformatics.org
                      http://www.bioinformatics.org/~gary
            ----------------------------------------------------              
                       bioinformatics.org: The Open Lab
                        http://www.bioinformatics.org/



From bizzaro at geoserve.net  Tue Mar 28 16:08:55 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] [Fwd: BOND. The Distributed Object Multi-Agent System.]
Message-ID: <38E11F67.F5C303C6@geoserve.net>

 
-------------- next part --------------
An embedded message was scrubbed...
From: Jennifer Steinbachs <stein@fmnh.org>
Subject: [Pipet Devel] BOND. The Distributed Object Multi-Agent System.
Date: Tue, 28 Mar 2000 09:57:28 -0600 (CST)
Size: 2140
Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000328/c84612b6/attachment.mht
From bizzaro at geoserve.net  Tue Mar 28 16:43:37 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:40 2006
Subject: [Pipet Devel] Re: overflow<->loci translation and 3 basic gui objects
References: <200003282049.NAA32328@martin.pharmacy.ualberta.ca>
Message-ID: <38E12789.EC9F05F5@geoserve.net>

Gary Van Domselaar wrote:
> 
> The second concept concerns the connections/links made from one application/node/icon to
> another.
...
> The third concept concerns nesting of 'networks'.
...

Thanks Gary.  Basically, XML representations of the GUI are concatonated
(placed end-to-end) when the nodes are related by simple link:

    <NodeGUI 1>
    </NodeGUI 1>
    <NodeGUI 2>
    </NodeGUI 2>

But when the nodes are related by nesting (a node contains a network of
nodes), XML representations are nested:

    <NodeGUI 1>
        <NodeGUI 2>
        </NodeGUI 2>
    </NodeGUI 1>

And both formats can be translated directly to Gtk: The first format places
widgets/GUIs in adjacent cells in a table, and the second format places one
widget/GUI inside of another (NodeGUI 1 is the container for NodeGUI 2). 
(Remember, the widget/GUI appears in the window of the node that is parent to
the network.)

Now, you may be thinking 'Hey, this is going to produce one very big GUI, if
the parent has a network containing 100s of nodes!'  But the trick (and a very
neat one at that) is that the GUIs from individual nodes can be switched on
and off.  Recall that I mentioned you can switch between network and GUI views
in each window.  Well, switch a node's window view to network (and not GUI),
and its GUI won't appear in its parent's GUI.  And this can all be saved!  The
result is, you have custom-taylored GUIs for each node.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From bizzaro at geoserve.net  Wed Mar 29 01:01:06 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] summary for web
Message-ID: <38E19C22.11E00BE9@geoserve.net>

Brad or Jarl,

Could I get a quick description of how all three projects will be tied
together and ordered?  If you give me the text, I can make a Dia flowchart and
put it on the web page.

You should probably organize the summary into 5 sections, according to the
layers or parts of VSh:

  Front
      Gnome Interface (Loci)

  Middle
      Definitions Layer (Loci)
      Brokering Layer (GMS)
      Processing Layer (Overflow)

  Back
      Foreign Applications


Thanks.
Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Wed Mar 29 00:34:55 2000
From: jarl at casema.net (jarl van katwijk)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] summary for web
References: <38E19C22.11E00BE9@geoserve.net>
Message-ID: <38E195FF.95EA974A@casema.net>

> Could I get a quick description of how all three projects will be tied
> together and ordered?  If you give me the text, I can make a Dia flowchart and
> put it on the web page.

Sure, but first this:
Brad and I are allmost done with the DL2BL idl, but there's one thing we need you

people to think about, this name:
    "interface RetrieveInfo"
which holds a set of functions that indeed retreive info, but I think this name
is no good.
Anyone has a better, single, word that has the same meening ("being informed")?

When we got this right the idl is final and we can start implementing the corba
functions.
(and I'll give the VSH overview a try)

bye,
jarl



From jarl at casema.net  Thu Mar 30 02:09:40 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] summary for web
References: <38E19C22.11E00BE9@geoserve.net>
Message-ID: <38E2FDB3.FE0BC7DB@casema.net>

>

OK, here comes MY partial view on vsh. Jean & Brad & Jeff: please fix and expand
this!

-------------------

Interface internal:
Jeff: This one is for you?

Interface <-> DL:
Brad: This one is for you?

DL internal:
Brad: This one is for you?

DL <-> BL:
There will be three lines of CORBA communication, or ORBs, of which 2 are used
during runtime, 1 is for authentication only. The DL will 'login' to the BL, and
when succesfull the Representation- and the RetreiveInfo communications are
brought up. 'Representation' handles the XML stream ('upload', 'download',
'update'), and 'RetreiveInfo' (until we got a better name) will supply various
information about the 'core'. The XML data is communicated by upload(),
download() and update(), the <NETWORK> tags are filtered out and used as a basis
for brokering.

BL internal:
Runtime optimalisation like internet traffic, cpu load balansing and
representation sharing.

BL <-> PL:
Communications:The PL will be a library, or a 'dynamic loadable binairy', or
simply a '.so' file. So after the initialisation both the BL and PL are merged
into one process\application. They both have the same unix uid, same address
space, etc. The runtime loop is inside the BL, the PL is event based.
(Technically this aint true, the BL wil be attached to a glib-timer and to the
ORbit main loop, which is a glib loop to.. but forget this )

PL internal:
Jean: This one is for you?

PL <-> FA:
Jean: This one is for you?





From bizzaro at geoserve.net  Thu Mar 30 23:38:14 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] interface RetrieveInfo
References: <38E19C22.11E00BE9@geoserve.net> <38E195FF.95EA974A@casema.net>
Message-ID: <38E42BB6.D687E359@geoserve.net>

jarl van katwijk wrote:
> 
> Brad and I are allmost done with the DL2BL idl, but there's one thing we need you
> people to think about, this name:
>     "interface RetrieveInfo"
> which holds a set of functions that indeed retreive info, but I think this name
> is no good.

Why is it no good?

> Anyone has a better, single, word that has the same meening ("being informed")?

Exactly what does this function do?

How about...

    Get
    Learn
    Acquire
    Obtain
    Discover
    Ascertain
    Determine
    Memorize

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From chapmanb at arches.uga.edu  Fri Mar 31 03:01:50 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] summary for web
Message-ID: <200003310804.DAA130078@archa10.cc.uga.edu>

Jarl wrote: 
> OK, here comes MY partial view on vsh. Jean & Brad & Jeff: please 
fix and
> expand
> this!

I took a stab at my two assigned parts. Comments are very welcome :-)

Interface <-> DL:
	The user interface communicates with the functional parts of vsh 
through a streaming XML dialog with the definition layer. This dialog 
occurs in the form of a request and response interchange between the 
user interface and the definition layer through a local socket. For 
instance, to request the addition of a processor node (program) to 
the workspace, the user interface would send the following XML request 
to the definition layer:

<ui>
  <add>
    <workspace id = 'workspace1/workspace2'/>
    <node type = 'processor'>
  </add>
</ui>

The definition layer would then process this message, perform the 
requested operation, and if the operation was successful, return 
information to the user interface in the response:

<dl>
 <success>
   <node id = 'newly assigned node id'
         num_inputs = 'number of initial inputs'
         num_outputs = 'number of initial outputs'
 	     ui_name = 'name of the UI type' />
 </success>
</dl>

or return an error message if there were problems.
	The design of this interface to definition layer communication allows 
multiple user interfaces to be created on the same definition and 
processing layer internals. In addition, these user interfaces can be 
created in any programming language supporting XML manipulation tools 
and sockets. 

DL internal:
	Thw definition layer provides the glue which connects the user 
interfaces with the processing internals of vsh. It is responsible for 
a number of functions:

1. Maintaining communication with multiple user interfaces through a 
streaming XML dialog communication protocol.

2. Storing of work-flow diagrams from the user interfaces in an XML 
format. This XML models the nodes present in the user interface and 
the 
connections/interrelationships between them.

3. Maintaining a permanent storage location for specially designed 
nodes 
that represent specific programs and libraries. The information for 
these nodes will be stored as XML and be available directly available 
to the user interface through the streaming XML dialog.

4. Communication with the processing portion of the program be 
implementing a CORBA client to a server in the brokering layer. 
This communication allows work flow diagrams to be passed to the 
processing layers for implementation, and allows a process to be 
queried while running.

5. Communication with remote vsh programs as both a client and a 
server 
using corba (through an idl that still has to be defined). This allows 
a 
specific implementation of vsh to serve out a specialized node or set 
of 
nodes to be utilized by other vsh implementations at remote locations. 
The processing layer will handle the specifics of this communication, 
but the middle will define a protocol to authenticate remote vsh 
implementations and provide access only to available nodes.

	The definition layer is implemented in the python programming 
language and relies on freely available XML and DOM libraries. CORBA 
communication will occur using the Fnorb ORB implementation, but will 
eventually move to ORBit (using ORBit-python) as work on python 
bindings 
for ORBit continues.





From jarl at casema.net  Fri Mar 31 03:27:06 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] interface RetrieveInfo
References: <38E19C22.11E00BE9@geoserve.net> <38E195FF.95EA974A@casema.net> <38E42BB6.D687E359@geoserve.net>
Message-ID: <38E4615A.B735F06A@casema.net>

>     Obtain

Yes, this one, thnx!
Brad, you're ok 2?




From jarl at casema.net  Fri Mar 31 04:31:08 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] summary for web
References: <200003310804.DAA130078@archa10.cc.uga.edu>
Message-ID: <38E4705C.C1B703C2@casema.net>

>
> 5. Communication with remote vsh programs as both a client and a
> server
> using corba (through an idl that still has to be defined). This allows
> a
> specific implementation of vsh to serve out a specialized node or set
> of
> nodes to be utilized by other vsh implementations at remote locations.
> The processing layer will handle the specifics of this communication,
> but the middle will define a protocol to authenticate remote vsh
> implementations and provide access only to available nodes.

When the DL is authenticated by the BL, shoudn't 5) also be located inside the
BL?





From chapmanb at arches.uga.edu  Fri Mar 31 07:02:02 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] summary for web
Message-ID: <200003311204.HAA147434@archa10.cc.uga.edu>

>>
>> 5. Communication with remote vsh programs as both a client and a
>> server
>> using corba (through an idl that still has to be defined). This 
allows
>> a
>> specific implementation of vsh to serve out a specialized node or 
set
>> of
>> nodes to be utilized by other vsh implementations at remote 
locations.
>> The processing layer will handle the specifics of this 
communication,
>> but the middle will define a protocol to authenticate remote vsh
>> implementations and provide access only to available nodes.
> 
> When the DL is authenticated by the BL, shoudn't 5) also be located 
inside
> the
> BL?

Yes, I think so. I'm not at all clear on how the specifics of the node 
sharing are going to work eventually, although this is a very 
important part of vsh.
    I'll try and describe my ideas on this by an example. Lets say I'm 
running a vsh implementation at home, and I want to connect with a 
high speed machine of Jeff's running vsh at a remote location. My 
rough idea was that things would work as follows:

1. From within the user interface, I would request to see the 
available nodes at Jeff's computer, and enter a password for 
connection.

2. My definition layer would get access to Jeff's machine via some 
repository of IORs and machines, and connect with his corba server.

3. My dl would authenticate with his dl, and then would send a request 
of all nodes that I have permissions to use.

4. My definition layer would recieve this list and make these nodes 
known to the user interface. I would then use one of them in a 
work-flow diagram.

5. I would send this work-flow diagram with the remote node (as XML) 
to the brokering layer in a processing request.

(okay, this is where things get pretty hazy).

6. The brokering/processing layer would set up its path of processing 
(by whatever method) and start processing.

7. When it needed the info from that node, it would need to connect 
with Jeff's computer remotely (this connection would occur somewhere 
inside the brokering/processing layers), send the node the info it 
needs, let it process it, and then get the results back.

8. Then the results can be fetched from the brokering layer on my 
computer in the same way as for the execution of a local node.

Okay, that my cloudy idea of how things would work. Is this what you 
all were thinking? Is this at all feasible?

Brad




From jarl at casema.net  Fri Mar 31 07:38:11 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] summary for web
References: <200003311204.HAA147434@archa10.cc.uga.edu>
Message-ID: <38E49C33.EC2A67FA@casema.net>

Brad Chapman wrote:

> >>
> >> 5. Communication with remote vsh programs as both a client and a
> >> server
> >> using corba (through an idl that still has to be defined). This
> allows
> >> a
> >> specific implementation of vsh to serve out a specialized node or
> set
> >> of
> >> nodes to be utilized by other vsh implementations at remote
> locations.
> >> The processing layer will handle the specifics of this
> communication,
> >> but the middle will define a protocol to authenticate remote vsh
> >> implementations and provide access only to available nodes.
> >
> > When the DL is authenticated by the BL, shoudn't 5) also be located
> inside
> > the
> > BL?
>
> Yes, I think so. I'm not at all clear on how the specifics of the node
> sharing are going to work eventually, although this is a very
> important part of vsh.
>     I'll try and describe my ideas on this by an example. Lets say I'm
> running a vsh implementation at home, and I want to connect with a
> high speed machine of Jeff's running vsh at a remote location. My
> rough idea was that things would work as follows:
>
> 1. From within the user interface, I would request to see the
> available nodes at Jeff's computer, and enter a password for
> connection.
>
> 2. My definition layer would get access to Jeff's machine via some
> repository of IORs and machines, and connect with his corba server.
>
> 3. My dl would authenticate with his dl, and then would send a request
> of all nodes that I have permissions to use.
>
> 4. My definition layer would recieve this list and make these nodes
> known to the user interface. I would then use one of them in a
> work-flow diagram.
>
> 5. I would send this work-flow diagram with the remote node (as XML)
> to the brokering layer in a processing request.
>
> (okay, this is where things get pretty hazy).
>
> 6. The brokering/processing layer would set up its path of processing
> (by whatever method) and start processing.
>
> 7. When it needed the info from that node, it would need to connect
> with Jeff's computer remotely (this connection would occur somewhere
> inside the brokering/processing layers), send the node the info it
> needs, let it process it, and then get the results back.
>
> 8. Then the results can be fetched from the brokering layer on my
> computer in the same way as for the execution of a local node.
>
> Okay, that my cloudy idea of how things would work. Is this what you
> all were thinking? Is this at all feasible?
>
> Brad

If you want to accomplise 1) you do a ReteiveInfo::uriStatus() on the BL.

5) will work by Representation::upload()

Look again at the uriS structure.

What we'll need ain't a DL <-> DL communication line, but a BL -> DL one.

(opposing the DL -> BL commmunication we've described by the
vsh-pilot.idl)

This way all authentication will remain at one layer, otherwise we'll get
some
serious syncronising problems.

Please do doubt & discus my opinion!

bye,
jarl



From chapmanb at arches.uga.edu  Fri Mar 31 10:06:08 2000
From: chapmanb at arches.uga.edu (Brad Chapman)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] locus vs. node naming
Message-ID: <200003311508.KAA15338@archa10.cc.uga.edu>

Hello all;
    I know before you all kind of decided that it would be best to 
refer to individual elements as nodes, as opposed to loci. And this 
was fine with, but I started thinking about it and I see one big 
problem with this: DOM. In DOM, the base "thing" is a node.
    So, if I start referring to everything as a node, this is going to 
make things really confusing really fast, because the definition layer 
code uses *a lot* of DOM.
    Can we either use "locus/loci" in the definition layer, or think 
up a new term that won't clash with DOM so much? 

Brad



From bizzaro at geoserve.net  Fri Mar 31 18:36:50 2000
From: bizzaro at geoserve.net (J.W. Bizzaro)
Date: Fri Feb 10 19:19:41 2006
Subject: [Pipet Devel] locus vs. node naming
References: <200003311508.KAA15338@archa10.cc.uga.edu>
Message-ID: <38E53692.AA350910@geoserve.net>

Brad Chapman wrote:
> 
>     I know before you all kind of decided that it would be best to
> refer to individual elements as nodes, as opposed to loci. And this
> was fine with, but I started thinking about it and I see one big
> problem with this: DOM. In DOM, the base "thing" is a node.
>     So, if I start referring to everything as a node, this is going to
> make things really confusing really fast, because the definition layer
> code uses *a lot* of DOM.
>     Can we either use "locus/loci" in the definition layer, or think
> up a new term that won't clash with DOM so much?

This is a 'simple' namespace problem, which can usually be solved by appending
the application's name.  So, instead of 'node', you can use...

    vshnode

or if we want to distinguish between layers...

    blnode
    plnode

BTW, about the BL nodes, I was thinking that since these always have an
Internet location, they could be referred to as a node of type 'locus'.

Jeff
-- 
                      +----------------------------------+
                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |        BIOINFORMATICS.ORG        |
                      |           The Open Lab           |
                      |                                  |
                      |    http://bioinformatics.org/    |
                      +----------------------------------+


From jarl at casema.net  Wed Mar 15 13:00:10 2000
From: jarl at casema.net (Jarl van Katwijk)
Date: Fri Feb 10 19:20:14 2006
Subject: [Pipet Devel] TODO
References: <39563C56.5C93A748@geoserve.net>
Message-ID: <38CFCFA9.800E15A3@casema.net>

Both my laptop and smtp relay seem to work now, jippy :)

I forgot one item for the BL TODO list : "Rewrite the push methode into a pull
one". Maybe Jeff can add this to the 'official' list ?

bye,
jarl