From jeff at bioinformatics.org Fri Sep 1 21:35:19 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:26 2006 Subject: [Pipet Devel] [Fwd: Is Piper for real?] Message-ID: <39B05957.6E5B2C54@bioinformatics.org> This is an e-mail Gary and I (any others?) got today about Piper. Our responses follow. Jeff -------------- next part -------------- An embedded message was scrubbed... From: Cameron Laird Subject: Is Piper for real? Date: Fri, 1 Sep 2000 05:52:44 -0500 (CDT) Size: 1176 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000902/ba48937a/attachment.mht From jeff at bioinformatics.org Fri Sep 1 21:35:45 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:27 2006 Subject: [Pipet Devel] [Fwd: Re: Is Piper for real?] Message-ID: <39B05971.1800CB7F@bioinformatics.org> -------------- next part -------------- An embedded message was scrubbed... From: Gary Van Domselaar Subject: Re: Is Piper for real? Date: Fri, 1 Sep 2000 08:24:03 -0600 (MDT) Size: 2806 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000902/13468764/attachment.mht From jeff at bioinformatics.org Fri Sep 1 21:36:08 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:27 2006 Subject: [Pipet Devel] [Fwd: Re: Is Piper for real?] Message-ID: <39B05988.EEC91733@bioinformatics.org> -------------- next part -------------- An embedded message was scrubbed... From: "J.W. Bizzaro" Subject: [tol-advisors] Re: Is Piper for real? Date: Fri, 01 Sep 2000 22:42:35 +0000 Size: 10290 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000902/119a0fbc/attachment.mht From jeff at bioinformatics.org Mon Sep 4 12:06:05 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:27 2006 Subject: [Pipet Devel] [Fwd: Is Piper for real?] Message-ID: <39B3C86D.204F5DD8@bioinformatics.org> -------------- next part -------------- An embedded message was scrubbed... From: Cameron Laird Subject: Re: Is Piper for real? Date: Mon, 4 Sep 2000 11:02:24 -0500 (CDT) Size: 4422 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000904/01b28be3/attachment.mht From jeff at bioinformatics.org Mon Sep 4 12:23:21 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:27 2006 Subject: [Pipet Devel] [Fwd: Is Piper for real?] References: <39B3C86D.204F5DD8@bioinformatics.org> Message-ID: <39B3CC79.4AED2B41@bioinformatics.org> I wrote back to Cameron and told him that his description is accurate. If you have any comments, please mail them to Cameron and CC to the list. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Mon Sep 4 12:38:04 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:27 2006 Subject: [Pipet Devel] BlueBox Message-ID: <39B3CFEC.6115D0B8@bioinformatics.org> Greetings. Before Brad and I found out about ISYS and OpenBSA, we planned on a collaboration with the developers of BlueBox (developed by dLoo Software and licensed under the LGPL): http://www.dloo.com/products/index.html I'm trying to get Nile Geisinger (head developer of BlueBox) on this list. In the meantime, I think you might want to take a close look at BlueBox. I believe OpenBSA, ISYS, and BlueBox have a great deal in common. Also, the thought has been brought up more than once (even with Nile) that Piper may be best used as a pipeline infrastructure for systems like BlueBox, OpenBSA, and ISYS, and that Piper should generally leave the user interface end of operations to those systems. I think this may be the working ground for DNP API specifications. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From mangalam at home.com Mon Sep 4 11:52:57 2000 From: mangalam at home.com (Harry Mangalam) Date: Fri Feb 10 19:20:27 2006 Subject: [Pipet Devel] Re: [dnp-api] BlueBox References: <39B3CFEC.6115D0B8@bioinformatics.org> Message-ID: <39B3C559.FD8E96DF@home.com> Hi Jeff (and good to meet you at ISMB), It sounds pretty interesting, but just how do we find out anything substantive about it? There's nothing but a static page with sales gibberish with no useful links at the URL you supplied, and most of the google-derived web pages that might have pointed to something informative have been 404'ed.. Being a recent (re)subscriber I may have missed a previous substantial thread - Do you have a summary or pointers to what BB actually is? Harry "J.W. Bizzaro" wrote: > > Greetings. > > Before Brad and I found out about ISYS and OpenBSA, we planned on a > collaboration with the developers of BlueBox (developed by dLoo Software and > licensed under the LGPL): > > http://www.dloo.com/products/index.html > > I'm trying to get Nile Geisinger (head developer of BlueBox) on this list. In > the meantime, I think you might want to take a close look at BlueBox. I > believe OpenBSA, ISYS, and BlueBox have a great deal in common. > > Also, the thought has been brought up more than once (even with Nile) that > Piper may be best used as a pipeline infrastructure for systems like BlueBox, > OpenBSA, and ISYS, and that Piper should generally leave the user interface > end of operations to those systems. I think this may be the working ground > for DNP API specifications. > > Cheers. > Jeff > -- > J.W. Bizzaro jeff@bioinformatics.org > Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff > "Injustice anywhere is a threat to justice everywhere." > -- Martin Luther King, Jr. > -- > > _______________________________________________ > dnp-api maillist - dnp-api@bioinformatics.org > http://bioinformatics.org/mailman/listinfo/dnp-api -- Cheers, Harry Harry J Mangalam -- (949) 856 2847 (v&f) -- hjm@ncgr.org || mangalam@home.com From jeff at bioinformatics.org Mon Sep 4 13:18:26 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Re: BlueBox Message-ID: <39B3D962.B22BD7C0@bioinformatics.org> Harry Mangalam wrote: > > Hi Jeff (and good to meet you at ISMB), It was good to meet you too :-) > It sounds pretty interesting, but just how do we find out anything > substantive about it? There's nothing but a static page with sales > gibberish with no useful links at the URL you supplied, and most of the > google-derived web pages that might have pointed to something informative > have been 404'ed.. > > Being a recent (re)subscriber I may have missed a previous substantial > thread - Do you have a summary or pointers to what BB actually is? Well, how about that! They used to have lots of links to details and the source code. I found that dLoo still has some pages up about Piper and development plans: http://dloo.com/piper/index.html I also attached some e-mail messages that may be enlightening. But, as far as /real technical specifications/ are concerned, we haven't seen any. BlueBox has been in this pre-alpha/alpha stage, along with Piper, for some time. I hope this standardization project will bring out all the specs. Cheers. Jeff -------------- next part -------------- An embedded message was scrubbed... From: "J.W. Bizzaro" Subject: [Pipet Devel] dLoo BlueBox Date: Sun, 11 Jun 2000 04:52:34 +0000 Size: 4183 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000904/2235fd33/attachment.mht -------------- next part -------------- An embedded message was scrubbed... From: "J.W. Bizzaro" Subject: [Pipet Devel] [Fwd: GNU Piper] Date: Mon, 12 Jun 2000 02:25:40 +0000 Size: 9296 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000904/2235fd33/attachment-0001.mht -------------- next part -------------- An embedded message was scrubbed... From: "J.W. Bizzaro" Subject: [Pipet Devel] [Fwd: GNU Piper] Date: Tue, 13 Jun 2000 01:10:35 +0000 Size: 4358 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000904/2235fd33/attachment-0002.mht -------------- next part -------------- An embedded message was scrubbed... From: Nile Geisinger Subject: [Pipet Devel] Greetings from dLoo Date: Wed, 14 Jun 2000 02:35:14 +0100 Size: 3732 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000904/2235fd33/attachment-0003.mht -------------- next part -------------- An embedded message was scrubbed... From: Brad Chapman Subject: Re: [Pipet Devel] Greetings from dLoo + answers to questions Date: Thu, 15 Jun 2000 05:11:01 -0400 Size: 12101 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000904/2235fd33/attachment-0004.mht -------------- next part -------------- An embedded message was scrubbed... From: Nile Geisinger Subject: [Pipet Devel] Piper's Importance and Resources Date: Fri, 16 Jun 2000 12:23:13 +0100 Size: 3977 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000904/2235fd33/attachment-0005.mht -------------- next part -------------- An embedded message was scrubbed... From: Brad Chapman Subject: Re: [Pipet Devel] BlueBox source code Date: Tue, 25 Jul 2000 09:13:30 -0400 Size: 3215 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000904/2235fd33/attachment-0006.mht From jean-marc.valin at hermes.usherb.ca Mon Sep 4 13:33:45 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] BlueBox References: <39B3CFEC.6115D0B8@bioinformatics.org> Message-ID: <39B3DCF9.3BD10902@hermes.usherb.ca> > Also, the thought has been brought up more than once (even with Nile) that > Piper may be best used as a pipeline infrastructure for systems like BlueBox, > OpenBSA, and ISYS, and that Piper should generally leave the user interface > end of operations to those systems. I actually think Piper will require many user interfaces, since users will be doing things so different with it. For example, you can use Piper just to replace the pipes on the command-line (for non-technical users), while you can use it to do distributed parallel calculations for [signal processing/bioinformatics/artificial intelligence], or for fast real-time sound effect processing. I don't think one (or even a few) UI can fit all needs. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Mon Sep 4 13:51:10 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Re: BlueBox References: <39B3CFEC.6115D0B8@bioinformatics.org> <39B3DCF9.3BD10902@hermes.usherb.ca> Message-ID: <39B3E10E.F07E101C@bioinformatics.org> Jean-Marc Valin wrote: > > I actually think Piper will require many user interfaces, since users will be > doing things so different with it. For example, you can use Piper just to > replace the pipes on the command-line (for non-technical users), while you can > use it to do distributed parallel calculations for [signal > processing/bioinformatics/artificial intelligence], or for fast real-time sound > effect processing. I don't think one (or even a few) UI can fit all needs. Very true, thank you. For those on the DNP-API list, you'll find more about Overflow (the native processing engine of Piper) at its website: http://freespeech.sourceforge.net/overflow.html Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jarl at casema.net Mon Sep 4 23:43:27 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Is this accurate? Message-ID: <39B46BDF.CA97AB9@casema.net> Hi Cameron, I'm a co-developer of Piper and the author of one the the projects namely GMS, or Generalized Messaging System. This name should be corrected in your text. Furthermore, I think one of the most important features Piper will offer is multiple path piping. On normal UNIX's you can pipe ("|") only from one source to another. Piper can merge or split streams. I think your article is great; it's easy to read and covers Piper very well, thank you for this. bye, jarl -- http://sunsite.auc.dk/gms From jean-marc.valin at hermes.usherb.ca Mon Sep 4 15:04:02 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Is this accurate? References: <39B46BDF.CA97AB9@casema.net> Message-ID: <39B3F222.4BF68C03@hermes.usherb.ca> > Workers in the life sciences seem to have the most enthusiasm for > Piper; genetics researchers, for instance, can easily see > the benefits of a system that makes it easy to build > filters and computations on a remote store for the human > genome or other massive datasets. I'd also like to add that not all Piper developers are working on Life sciences. I'm co-author of the Piper processing layer (the Overflow project), which was primarly developped for signal (speech, image) processing, from an electrical engineering point of view. I think Piper is more than a life science thing. Other than that, I agree with the article. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Mon Sep 4 17:19:12 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Is this accurate? References: <39B46BDF.CA97AB9@casema.net> Message-ID: <39B411CF.D53E5304@bioinformatics.org> jarl van katwijk wrote: > > I'm a co-developer of Piper and the author of one the the projects namely GMS, > or Generalized Messaging System. You mean I've been spelling it wrong all this time? :-) Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Mon Sep 4 17:41:05 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Is this accurate? References: <39B46BDF.CA97AB9@casema.net> <39B3F222.4BF68C03@hermes.usherb.ca> Message-ID: <39B416F1.D70C16F6@bioinformatics.org> Jean-Marc Valin wrote: > > I'd also like to add that not all Piper developers are working on Life sciences. > I'm co-author of the Piper processing layer (the Overflow project), which was > primarly developped for signal (speech, image) processing, from an electrical > engineering point of view. I think Piper is more than a life science thing. I second that notion. I have presented Piper to my colleagues in the life sciences as a system that can be very useful to them, but I am always sure to mention that Piper is general-purpose in design. I believe that this makes Piper even better for special-purpose use, since it can change along with standards. It also attracts a broader mindshare of users and developers, which only helps to improve the system for everyone. Perhaps we are not all scientists, but all of the developers appreciate the need for a system capable of very sophisticated operation, yet with a simple facade. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jarl at casema.net Tue Sep 5 11:34:08 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Is this accurate? References: <39B46BDF.CA97AB9@casema.net> <39B411CF.D53E5304@bioinformatics.org> Message-ID: <39B51270.3B321AB5@casema.net> > > I'm a co-developer of Piper and the author of one the the projects namely GMS, > > or Generalized Messaging System. > > You mean I've been spelling it wrong all this time? :-) Think you took the save way of using 'GMS' :) bye, jarl From chapmanb at arches.uga.edu Tue Sep 5 12:51:42 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] IRC on Bl/Pl organization Message-ID: <200009051651.MAA119342@archa10.cc.uga.edu> Hey all; Jean-Marc, Jarl and I have been trying to work on thinking up good plans for how to organize the BL and PL and get some good connectivity going. Yesterday there was a big ol' IRC chat about this, which is pasted below. It discusses the organization of the BL and how this will interact with the PL. This discussion introduces the ideas of central "mother BLs" (MBLs) which are central controlling BLs that will interact with the DL (but maybe not directly). The mother BL can then have "Baby BLs" (BBLs) which will directly interact with the PL. This is sort of a summary of what is going on, but the actual structure is still under heated discussion, as you can see from reading the log. Part 2 of this discussion will occur today at 4:30pm EST. Anyone who is interested is welcome to join in (or listen). The chats occur on ChatNet (see www.chatnet.org for a list of servers) on the #piper channel (naturally enough). See you there! Brad --> jm (~jm@cn-997.143-201-24.mtl.mc.videotron.net) has joined #piper hi jean-mark I'm here! Hi Jean-Marc! Have you been talking for a long time? got the email or the icq? e-mail... got no ICQ! almost an hour already :) icq sucks :( ..oops I actually had the ICQ! --- bstard gives channel operator status to jm talking <- mostly about compiling the BL and fun stuff like that :-) And how's the C++ compiling going? and I screatched a meganism for the work the BL should do Go ahead... C++ port <- went smoothly until I met the C++ Corba C++ corba is very diff. from C Corba. 0: DL logs into BL 1: DL uploads XML network to BL 3: BL checks XMS for resources 4: BL distributes nodes to BL instances agross internet 5: Acknowledges DL and PL that BL is ready and waiting 4 a go 6: {undefined yet} step 4 will be the one that will undergo big enchancements once we've the basics operational. stuff like resource management etc will be runtime some day in the future :) So wouldn't the BL involved in 3 and 4 be a "master" while the distributed ones would be "slave"? oops, forgot 2: BL checks XML for authorized nodes master-slave: ok, way of looking at thing. things like you view it: master = local instance, slave = remotely (on other machines) so a BL is master if it's distributing, and a slave once it's receaving XML's Sort of, what I meant by "master" (may not be the best term) is the part that has the "global view" while all the other BL instances ("slave") only see a small part of the problem. Yeah, that is the view I had of master/slave as well. But once the network is running, the BL instances are 'same level' instances communicating results etc to each other. The master-slave relation only exsists during the 'distribution phase' jm: jep, that's the way I view the situation too. No, but I think that things also need to be broken up so that you can return information back to the DL, like in the "viewer" nodes. slave = partially knowledge of the 'job', master has full knowledge Well, I think there should be a master that is the only one allowed to talk to the DL. brad: what do you mean by 'no' ? no <- I mean, doesn't the master-slave situation also need to exist on a local level? jm: the DL logs into a BL, so only that DL can receive, only that DL will be offered the results and can upload jobs Well, if one part of the processing is done locally, then there should be a slave running on the local machine. nah, why an extra thread\instance? the way I see things, only 1 BL will run on a machine. but try to convince me :) What I meant was that all when slaves want to communicate with the UI, they should all send the info the to master. Then the master is the only one allowed to talk to the DL. convince <- Jean-Marc is probably the best person to do this, since it is his idea :-) hehe I see the master as a library that's linked with the DL, so there's one process for the DL and the master. uhh, how do I put this. See the DL as the master BL, that;s probably the best way to look at the situation Also, it you have an SMP machine and want to run two part of the processing on it, you should have two BL slaves running on this machine. all the BL's do the work for THAT specific DL they also do work for other DL's jm: SMP will be covered by internal threading which is already implemented for 50%.. Why not just say one BL slave process for each part of the processing? multiple DLs <- Jarl, this is confusing me. There will only be one DL process per machine. A single process can handle multiple user interfaces. but.. I still can not see why you people advocate a master-slave model. Try me again please.. jm: each part of the processing will be done by the PL? So it seem more logical to thread multiple PL's brad: did I say "Multiple DL's" ?? If I did, a mistake, sorry Ofcourse we'll only need 1 DL/mache machine arf multiple dls <- Okay, sorry. I was just getting confused :-) I'm thinkint 1PL=1BL, so there's as much BLs as there are processing parts. brad: who aint :) jm : good point.. they are gonna be statically linked which kinda sucks regarding my views :-) What's going to be statically linked? Overlow as lib to the BL or will we use .so ? Well, it "has" to be dynamically linked ic why? Overflow is meant to work dynamically, since the toolboxes are dynamic libraries that are loaded on startup. ok, no problem and Overflow can only execute 1 network at a time? Basically, when Overflow starts, it looks for all the .so files in a certain path and loads them all. Each .so contains code for a bunch of nodes. That way, you can add nodes without recompiling. ic, that's the ways GMS works with plugins too Well, Overflow is just a library, so you could build 10 network and run them at the same time if you have 10 threads. And could you implement threading in the library after the RUN object? so that the BL could upload multiple XML's into the PL and give it a go ? Well, it would be equivalent to the BL starting multiple threads and starting a processing in each one. :) hehe so I'm buggered again? :-) dont get me wrong The reason I prefer one PL per BL is that sometimes the PL could crash for some reason and you want to be able to track it. Also, you could restart only this part instead of all the PL's running on that machine. I cant really oversee the situation because I dont really know how Overflow is implemented.. If you think the best way is to have the BL do the threading that's fine to me... crashing <- ic crash <- I think this is very important. People will really want to know why things crashed, and during development we will definately need to know this :-) dont expect the BL to be 100 stable.. it HAS been running for days under stress testing.. but I you never know "but I you never know" <- but you never know :) stable <- No problems, I don't expect anything to be stable yet :-) OK, so the design will be : 1 DL -> 1 BL with internal threading -> 1 PL / BL thread ?? I think in this case, mulpiple processes are better than multiple threads. You're losing me... aailks loosing where? @ the threading vrs. processes? I don't understand what you want to thread. In my mind we wouldn't need any thread at all (just separate processes). I still am in faivor of 1 BL process / machine.. that's why I'm thing about threading multiple BL's will be very hard to manage darn... if only we were sitting next to each other I could draw a design :) that would clear up thing\ s jm: you saw this flowchart on http://sunsite.auc.dk/gms ? The problem is that if the PL crashes, it will crash the BL and all the other PL's on the machine. hmm... I can have the BL run inside a debugger for as long as things are unstable a build-in debugger no gdb or so.. but the ONLY thing you have again a sigle process design is the instability? Or do you just dont feel confortable with it? it's more about PL instabilities and error recovery. hmm.,.. just a sec, I'll give somebody a call about we running the BL\PL set inside a debugger.. just to ask if it's possible. arf... if you need people :( no answer :) otherwise we can have the DL do the resurrecting? it's not only about debugging but also real-life. If the PL crashes in real-life, then you only need to restart one PL. the BL is capable of keeping track of a runtime history, so it can do a full restore after a crash no, not debugging the errors! DL resurrection <- I can detect a BL crash very easily, but I'm not sure how I'll be able to report back to the UI on what caused the crash... Not if the crashing PL crashes the BL and the other PL before they have time to terminate. a debuggers receives the crash signals and can respawn the BL\PL process but you loose all the computations from the other PL's DL resurrection <- the DL can restart the BL with a special parameter, and the BL will do the rest just like if we'd BL slaves restart a PL process. Dl resurrection <- What do you mean be special parameter? jm: lose other computations. again good point. like ./BL -restart :) ok, the PL can not do history tracking.. that would become too slow. shit! sorry I don't see what's so complicated about running two BL's on the same machine. ok. What about: the BL will start a child\slave process for all data streams coming out of the Pipeline? out of the BL pipeline that is. What do you mean by BL pipeline? all incoming data streams go thru the pipeline, that does the authentication etc. see : http://sunsite.auc.dk/gms/gms.gif other things like marshalling data and resource management are also located in the pipeline Sorry, I don't understand your flowchart... if we are to use multiple BL's on 1 machine that will become very hard look only at the middle part, with 'outside world' above it the 'sensors' are the data streams coming from external sources.. What's harder? For me it's just communicate with another BL, regardless of where it is (local or remote) the basic of gms is that there is only 1 pipeline on a machine. That's why I make the fuss about multiple BL's :-) so the plan of stripped (non-pipeline, no DL-communicating, non-external input) slave BL's sounds ok? or dont you get it? Sorry, I think I'm missing some basic information... to you ( the PL) there wont be a diff. Also to the DL there's no diff. and the BL design will remain intact jm: maybe we need to discus this on voice some day? brad: you're still here? :-) Well, you're the BL specialist after all... it's just that from my point of view, things would look so much cleaner if "one part of the processing" = 1 BL "slave" = 1 PL, regardless of where everything runs. Yup, just listening :-). This is your guys battle. ...Just thinking... is the multiple BL problem about the authentication and stuff like that? jep. about deadlocks and so on. and about resource management In this case, maybe there could be a permanent part of the BL on each machine, which would spawn (process) Baby-BL's linked with the PL. that's why I only wnat to have 1 process doing all that on 1 machine YEP! the BL ' master' will be permanent those 'slaves' will be spawned one there's something to do 1 spawned stripped-down BL / PL once? and die when the job is done That's what I'm thinking about... it's just that I had forgotten about all that authentication/resource thing. Can you wait a sec... ok, pfeeew we've settled something it seems :) Good stuff! Baby-BLs are the way to go :-) ofcourse I'll wait... /me having a thinking-smoke brad: lol hehe BBL's :) :-) I'm happy you are coming together on that. The Baby-BLs actually makes a lot of sense with what both of you were saying. Maybe we can have parent/baby relationship, instead of master/slave. Sounds a little nicer :-) brad: could you build into the DL a respawn check for the Mother BL ? the MBL so to say What do you mean by a respawn check? if the MBL dies, the DL should restart the thing like the BBL's do for the PL' s Yeah, that is no problem. We should add a little ping() function to the IDL, then when the ping fails, the DL will restart the BL. Well the BBL would link to the PL, so both die at the same time it would be the mother BL that would respawn a BBL/PL brad: ping <- yep, I'll put it into the IDL jm: jep! So what is our plan for actually implementing all of this? the MBL will do all the resource\AAA and (re)spawning work As for what I was calling the "master", it could belong to the DL or BL (I don't care), and that's the one I see responsible for splitting the processing, sending that to the MBL's and respawning MBL's Actual impl. <- 1stly the C++ Corba should be done. After that I'll be eager to do some coding of new features :) jm: I see what you mean now. I think you can see my initial opposition to this now too can you? opposition to what? to the master-slace design slave impl <- Well, how about if we split like this: I'll work on getting the CORBA C++ working and you and Jean-Marc can work on implementing this whole mother/baby thing.... if slave start to do authentication etc the system will become a big mess I'm not sure you understand what I mean. What I was calling "master" is more like a library that the DL would use to communicate with all the mother-BL's brad: I can not devel that without the sources compiling :( that darn corba must be running soon For me master BL != mother BL. jm: ?? master != mother? OK, let me say how I now see the whole thing: impl <- Well, lets extract the CORBA out into its own separate module. Can you just write a little C++ code that will run to test things, and then we'll make things be driven by the CORBA subsequent to that? I think the CORBA should come out of the messaging_db stuff for cleanliness maybe anyways... The DL links (as library) with the "master"-BL, which takes care of splitting the processing into small XML chunks that are sent to all the mother-BL's on the different machines. Then each mother-BL spawns a Baby-BL/PL for each part it receives. brad: let's not create too much chaosshall we? shall we 1st finish the corba, and once it's working I'll separate the corba and the database, ok? jm: three BL's? hmm Assuming the flow A->B->C where A and C are run on machine X, B is run on machine Y, and the DL is run on machine Z, here are the processes I see: 1 process on Z (the DL/master-BL) 3 processes on X (1 Mother-BL, 2 baby-BL's) 2 processes on Y (1 mother-BL, one baby-BL) Once again, the confusion probably arises from the fact that you probably don't consider my "master-BL" to be part of the BL... impl <- I'll put this on hold until you and Jean-Marc are done... brad: k :) jm: ic what you mean I see this master BL and mother BL as being the same process the motherBL gets a chunk of XML from the DL, splits it, sends it to other motherBL's and the motherBL's start producing baby's I see all the mother-BL's being equal -> no reason why the splitting would be done by one particular instance of mother-BL. and maybe the 'other' motherBL's will also split the XML once more if they see fit hehe jm: still see the need for a 3th BL type, the master BL? ...or you can call it master-DL if you like, since I don't see where it belongs. I think it's not the mother-BL's job to split the processing in chunks. ...if you don't do processing on the local (where the DL runs) machine, you don't need a mother-BL there. but the BL will manage the resource management, this can not be done without doing the xml splitting Sure, but since there are many BL's, the splitting decision has to be split somewhere. There should be a part that only does that. there will be only 1 MBL on a machine. So it's capable of desiding the splitting why introduce a new process for just doing the splitting? the MBL will do 1: authenticate, 2: recouremanagement\splitting 3: spawning BBL's Yes, but how do you decide how to split between different machines. What I'm saying is that there should be a part that's in the middle of the DL and all the mother-BL's there are many MBL's, right? one on each machine I think a MBL should "take what it want" and "pass on what it doesn't like to other MBL's" no, just ONE MBL / machine many BBL\PL sets ...OK, let's go from the start. ok :-) If we have 10 machines, we have 10 MBL's (one on each machine). Let's assume the DL runs on a 11'th machine. ok.. The DL gets from the GUI a complete (unsplit XML network). (remember, there's no MBL running where the DL is) There's no reason the DL would ask one particular (of the 10) BL's to do the splitting. there is the DL doesn't have login access to every MBL but the DL COULD have it.. very strange situation though I think there should be "something" on the local machine that (maybe after communicating with all the MBL's) decides how the program should be split and then sends each part to the right MBL. Why not have a special part of the BL that would do the interface between the DL and all the MBL's (on all the machines) what about making this "something" this logic: "take what it want" and "pass on what it doesn't like to other MBL's" that way the splitting can start on every system on the net in theory.. I think we disagree on how the processing will be split... From what I understand, you say that only the BL will decide about the splitting. Am I right? you say splitting should be done by a splitting master, I think we can rely upon smart algorithms that will do the job without any 'mastering' I'm more thinking that the user will say "This runs here, that runs there, ..." the user CAN say that Only, sometimes, the user want something to run somewhere... and nobody's going to decide something else. that way the splitting will be very easy but what if the user says 'just runs it somewhere where it will be done fastest' ? In this case the local "splitter" would ask all the MBL's about their resources, but I think most of the time the decision will not be made by the MBL's. Let's say I have a sound acquisition node, I want it to record from a certain microphone that's on a certain machine... not to record on any microphone, where the BL like to. I think that way will build a very static system. You should have more faith in having the MBL's fighting about getting the optimal resource load microphone: ok, in that case you define to use THAT specific mic. Splitting highly defined xml will be easy, we'll just pass the xml on to where the user wants it to be I'm more interested into xml that does NOT contain any localisation defines I think the user needs to have a word to say about the splitting. sure he\she does! the user work with 'that file' and 'this soundcard' but also with 'some computation power, where the heck it might be' or 'information about this' What not have all the MBL's send their resource info at the same place and then combine that with what the user likes and take the decision *in one place*. how will this be done once Piper runs on 103430943 machine some day in the future? machineS arf There are always things the user will know that the BL won't know... like "if I run this part that lauches this process on this machine, than this user will be mad at me because of this side effect". There's no way the BL can handle that! ok, that's why the MBL will get strict resource limits maxcpu = xx% maxmem = xx mb etc but ok, you're a tiny bit right :) sometimes, it's just that a certain node needs more io or does something else that disturbs... the BL wont know everything.. so wont the 'splitter BL' Yes, but the user can speak to the splitter BL and help it do the splitting. that's why I want to implement a 'prediction' neural net once we've Piper running.. to predict availeble resources hmm.. you think the user should guide the splitting? I think the user can define anything he wants, after which it's up to the BL to fill in the gaps Anyway, I don't see how distributing the splitting would be better than making a centralized decision taken from the data (resources) of all the BL's. Note that I don't see how the BL will know what resources (CPUt time) are going to be required for each node, since it will not know what the nodes do. 1st because a centralized meganism wont be possible once there are a lot of Piper's running, and 2ndly because I know prediction algoritms are much more effective than centralized algoritms When I write new nodes, I don't want to have to tell the BL how long it will take to run, depending on all the variables (size of the problem, ...). not know how much it will use in advance <- ok, not the 1st time the node runs, but here the history logs and prediction algoritms come into play but I hope you'll reuse those nodes a next time you build a network so the MBL's will know upon hostory what that nodes consumed the previous time not exactly... there are some things you cannot predict... like how long it will take that algorithm to converge according to the caracteristics of the data. ok, such nodes will exsist. Those nodes will get the label 'unpredictable'. :) Let's say my neural network will take twice longer to converge if I train it with speech than if I train it with music... how will the BL know what files I'm senting to it... also, even the idea of how long it will take to converge is very intuitive. The problem is that these "unpredictable" nodes will often be the most resource consuming... in my case at least. ok, but nodes will consume resources in more-or-less the same order of magnitude maybe this is something that needs experimenting.. To me the design of having 1 central boss BL for each network will be limiting. Also, even with many Piper's running, I don't see the problem of centralizing decisions... Each MBL tells the splitter how much resources it has left, the splitter receives all that info and takes the decision. hmm... shall we continue this discussion tomorrow, it's 23:50 here. I'm a bit tired. maybe I see things much clearer tomorrow :) I'll give your point of view some brainpower tomorrow... let you know more next time, ok? brad: bye, you're around tomorrow too? Sure, yeah, I can be really quiet tommorrow as well :-) From chapmanb at arches.uga.edu Tue Sep 5 18:38:13 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] IRC log about general DL/BL/PL design Message-ID: <200009052238.SAA73240@archa12.cc.uga.edu> Hey all; The IRC chat today didn't happen as plan due to Jarl's sickness :-<. So there wasn't any time to do any arguing about design. Instead, I asked a lot of questions of Jean-Marc about the design to get myself more in touch with what he was thinking. The biggest change since yesterday is the idea of a Grandma-BL. This is a since high level BL process which communicates with the multiple mother-BLs. Jean-Marc and I talked about possibly implementing this in python in the DL and thus changing the interface that the DL uses to communicate with the BL. Anyways, the log is below for your perusal, if interested. Comments are, as always, quite welcome. Brad [Snip...talking about Jarl being sick...] I'm coding the C++ CORBA stuff for the DL/BL communication. Fun with C++ :-) BTW, any idea what's the best way to discuss about this splitting thing with everybody? Well, here is an idea. Why don't you write something about it on the piper list... Well, if we have a grandma-BL, there'll be no DL/BL Corba communication. The grandma-BL will be linked against the DL and the CORBA communication would be between the grandma-BL and the mother-BL's then start a page on the recently set up Piper Wiki (http://www.bioinformatics.org/piperwiki/)... I can do that. then we could discuss it there and people could go through it. I could set up a page for it. Then we'll have some kind of ready-made documentation on it -- or at least something to start with... grandma-BL <- So you think the GBL will be in python? I'm confused... Can't the GBL be done just like libflowui: a C/C++ dynamic library, that the python DL could "use"/talk to? I guess so, although I really prefer CORBA over simply wrapping things, since this helps keep some language independence. Additionally, I'm not sure how I would know if the GBL dies. With CORBA it shouldn't be too difficult to know it died and restart, but I don't know how this will work if I just wrap it. Well, if it's linked against the DL, then when it dies, the DL dies too! Everything would be exactly the same way you use libflowui. Plus, I don't see why use CORBA when you are on the same machine... ...If you prefer, then you could write the GBL in python and put it in the DL (the GBL would become the DL) dying <- Well, that is really bad, right? IF the DL dies then the UI will just be left hanging. I'd like the DL never to die... There's no way you could be sure the DL won't die... except if you don't run a DL at all :-) CORBA <- Well, I like CORBA a lot more than wrapping directly. I don't think the slowdown over direct wrapping is that big of a deal, but CORBA provides a nice interface to work off of, and would also make it possible to rewrite the DL (if someone wanted to) in any language (ie. java, if someone digs that). die <- Well, of course :-). But I don't want to the DL to die just because the BL dies during a run! With a CORBA interface we should be able to detect the death, and restart the BL as a separate process. Or what would you think about saying that the GBL and DL is one thing? All written in python. Then all the MBL would communicate with the DL. GBL in python <- Well, I don't know how much of the GBL functionality Jarl already has in GMS. But I wouldn't mind writing it in python at all. This is all provided that what Jarl has right now is mostly a bunch of MBLs. Do you have any kind of vision for what the GBL/MBL interface would look like? Sounds fine... but I doubt Jarl wrote the GBL, since we just "decided" there would be a GBL. The functionnalities are for viewers/probes and (maybe) splitting, none of which Jarl started coding... Not at all. I have never touched (or even approached) CORBA... I have an idea about the BBL/PL interface, but that's all! The GBL in python sounds great by me (one more reason why the dl is important :-). I have a slight idea of what the communications between the PL and the DL should look like with no idea how it will get through the BL. Well, the BBL to PL interface is definately a good start :-) ...and believe me, it won't be CORBA! I'd like to kind of see if we could at least hash out the rough ideas of how the communication will look. I'm kind of at a loss about how to proceed next with this, although I'm ready to code... no CORBA <- :-). No problems. I like CORBA a lot, but that doesn't mean everyone has to... ...not that I don't like CORBA (I don't know it enough to like it or not). I just see the BBL linked to the PL. Yeah, I agree, this is also I have been thinking about it thus far. I think the communication will be message/command based. The DL will be able to send a PL node commands like "Next", "Continue", "Stop", .. But I have also been thinking about the BL/DL communication (or GBL to BBL) as CORBA. A PL node will the the DL: "plot this data", ... The message based system makes a lot of sense with what we were talking about in terms of that iostream stuff and having the error logging system. Well, the GBL will not talk directly to the BBL's, there's MBL watching ;-) I don't have a very concrete idea of how the BBL/PL stuff will look, but what you say makes sense. MBL <- :-) Whoops sorry, I meant GBL to MBL. Still getting the new acronyms straight. the BBL will just call getOutput() on its subnetwork. If at some point a PL node needs to do a getOutput() on a remove machine, it will ask the BBL for that data. How the BBL gets it is not the PL's problem. In fact, it will end up as another BBL calling getOutput on its subnetwork. ...oops... I can't help on that! Okay, but how will the PL callback to the BBL to let it know it needs info from a remote machine? let's say we have A -> (B->C) -> D, where B->C is on a certain machine... Okay, I recognize this example :-) When breaking the network, B's input will be connected to a special communication node. This node will call a BBL function to get this data. the BBL will manage to get the data from A by talking (indirectly) to another BBL and will return it to the special node, which will return it to node B. Okay, so everything happens through the special communication node. Is your idea for the design of this similar to the interface we were kind of talking about with the iostream overriding class? no, this node will just cal a function. The only reason for the iostream class is that I would like to be able to make that stream equal to cerr (stderr) if I like without changing the code in the nodes. Okay, I'm with you. What do you imagine the function calls the node will make looking like? Would this be similar to the interface type that I sort of set up in the IDL file I sent a while back, or something completely different? It would be sonething like: theBBL->getMeMyInput(this, inputID, count); Okay, gotcha. BTW, what exactly is count? I saw it in all the getOutputs() but I never exactly got it. ("this" is a pointer to the current node, in case you're not familiar with this part of C++) Okay what about errors generated during processing a getOutput() in a node. How would those get returned to the BBL? this <- Yup. I'm with you on that :-) Thanks, tho! There's a count when loops are involved, it's used to select to output number. it depends on what kind of error... Okay, so count is only relevant when in a loop. error <- Well, what different kinds do you imagine? some errors are "fatal": they throw an exception which will be caught my the BBL. When there's an exception, it means that the task simply cannot be performed (eg. a parameter is missing to a certain node). Okay, that makes sense. What about non-fatal errors? Warnings through the streams (cerr << "something strange here" << endl;) will be sent to the DL (throught the BBL) and everything will continue. when not in a loop, the count asked will be zero; What about an error which might cause the user to want to choose between stopping and continuing (some kind of processing error)? The best example would be when treating video... if a node asks another getOutput(outputID, n), it means I want to have the n'th frame. Or how about your debugging/probe stuff? We will need to define a special interface for those... but I haven't found a use for that so far (though not saying it doesn't exist) debugging will be handled through display/next/continue commands Okay, but we should just stick with simple cases first and see when the other cases come up, you're right. I think this makes sense how the BBL/PL will happen. I guess we need to talk through everything with other people to see how the GBL/MBL should look... basically it'll be about commands (sometimes with data associated to it)... The address of a node will me something like MBL:BBL:ptr (the address of the MBL, the address of the BBL, the (C++) pointer to the node) Have you had a look at the dl2bl.idl interface Jarl and I wrote? nope... I know nothing about idl stuff, so I couldn't comment on it anyway. commands <- I think (maybe) I am with you on this. I basically understand the concept, although the implementation is foggy in my head. idl <- Well, it is very similar to C++ syntax, so it should be hard to follow. I just thought maybe this might give you ideas about the interfaces (ie. classes) we were thinking about for communicating between the DL and PL. This was written a while back, but I think parts may still be relevant to where we are at now. Ooops, that's 'shouldn't be too hard to follow' :-) Don't worry, the implementation is foggy for me too! I think the BL shouldn't need to know about the messages... The BL will just transmit "data" implementation -> Well, as we go, can we sketch it out in C++ header file syntax or pseudocode or something? This would help me follow a lot better, since I have some trouble thinking really abstractly :-). We could use the wiki for this, again. messages <- I'm a bit confused about these messages all of a sudden. Are these messages utilized by the PL? Well, there's no "detail" right now, the details will depend on the type of node (viewer/probe/...) When the DL wants to say "Next" to a PL probe, it sends the command through the BL, tough the BL never knows what "Next" means. Okay, so these are messages for DL to PL communication (and vice versa). I'm with you. Will these messages be defined in some kind of XML syntax, in your mind? Probably not... It will probably look a bit like a pseudo-packet... There will ba an address (MBL:BBL:ptr), a command and some data. The data will be a serialized Object (deriving from my Object class) Okay, but the only think I don't understand is how the ptr (the most important part) will be transmitted about, especially between python and C++. Or maybe, the command and data will be packed in an Object, which will be the data in the pseudo-packet. serialized <- In what way? Well, a DL->PL communication will always be some kind of answer to a PL->DL communication. The talking node will tell the DL its pointer. The DL won't know what to do with it, but the BBL, which is in the same address space (because it links to the PL) will. serialized: The whole Object (sometimes containing other Objects) will be changed in a string of bytes. serialized <- This is my sticking point since I don't really understand serialization :-< Sorry, I'll have to try and bone up on it. So far, I only support ASCII serialization: serializing a vector of float looks like If I have a vector of vectors, it will look like > >> The idea is just that an Object can be sent through a network or saved to a file. Okay, so this is just C++ specific serialization right? Maybe I do understand, since python has a pickle module which provides persistance like this (ie. you can 'pickle' a data structure, and then load it up from another program instance). I just don't understand it in C++ yet, but I get the idea now! It's not a C++ feature. it's just what you have to do to transmit an object... maybe python does it for you, but it still needs to do it. Right, but there is a specific syntax that you need to spit it out in so C++ can reload it (or transmit it across a network). What I mean is that I don't yet know this syntax, but I can understand what is it doing :-). How do pointers get serialized via this mechanism? I decided on the syntax... I wrote the save and the load. It's like for a UIDocument that has contains a bunch of UINetworks that in turn contain UILinks and UINodes, which contain UINodeParameters... You manage to transform that to a bunch of bytes, which to save (the .n files) For my Objects, instead of using libXML to serialize, I wrote my own serializer, which is cleaner for what I do. Okay, understood completely. Sorry to be so confused on that! I think I sort of see where we are at, now we just need to move forward and figure out the implementation stuff and see what other people think about things... but that is probably for another chat session :-) Yep! In the mean time, I'll try to expose my ideas on splitting on the list. Makes sense. I've been logging this, so I'll send it to the list as well. Thanks for talking this over with me! Ttyl. ttyl. Have a nice night! From chapmanb at arches.uga.edu Tue Sep 5 19:00:07 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Piper Wiki is up and running! Message-ID: <200009052300.TAA146370@archa11.cc.uga.edu> Hello all; Thanks to Gary's help, the Piper wiki is up and running on bioinformatics.org. You can check it out at: http://www.bioinformatics.org/piperwiki/moin.cgi It is also linked to from the mail Piper page. I've started working on adding information and will expand on this over the next few days. Since the point of wiki is to allow collaborative documentation, everyone is highly encouraged to check it out and add info/comments etc. If you have never used wiki before, the best place to start is the EditingTips page at: http://www.bioinformatics.org/piperwiki/moin.cgi/EditingTips This will give you the idea of the type of markup you can use when editing Wiki pages. Once you feel ready to try it out, head over to the WikiSandBox (linked off of the EditingTips page) where you can play around without any fear of messing things up. Enjoy! Comments are very welcome. Brad From jean-marc.valin at hermes.usherb.ca Wed Sep 6 21:03:18 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:28 2006 Subject: [Pipet Devel] Licensing issues Message-ID: <39B6E956.344C767@hermes.usherb.ca> With the latest issues/wars (depends on how you see it!) with the KDE/Qt licensing, I think it's now time to make sure not to end up with the same mess (no matter which side you are, the result is that it has hurt KDE a lot) with Piper. This is mostly important for the PL (Overflow), because CORBA linking is not covered by the (L)GPL. Also, now's the time to change the Overflow (PL) license, because all the copyrights are owned by three people: Me, Dominic, and Brad. The three need to agree to change the license (should not be too hard!). If we wait too long and too many people contribute, it'll be too late (there are hundreds of copyright owners in the Linux kernel, changing license is now impossible). For now, the license is GPL, since at first, it was not meant to be a used as library. I think we all agreed (do we?) that the basis sould be around the GPL/LGPL. Before I continue, here's my simple explanations on each. You may link any software (be it a program or a library, OSS or not) to an LGPL library, while to may not link to a GPL library. (this is commonly known). You CANNOT link either LGPL or GPL software (library or program) to a library that is not GPL-compatible (LGPL, GPL, and BSD are GPL-compatible, while the QPL is not). This is the clause that caused so much problem with KDE and that made it illegal, since libkdeui (which is LGPL) could not legally be linked to Qt, which was not GPL-compatible. Now this is what I'd like to discuss: 1. Should we allow non GPL-compatible nodes to be used (my opinion is yes)? If we want that, we need to provide with an exception to the GPL/LGPL that explicitly allows it (like KDE needed to have so it could be linked with Qt). The unclear part of this is would we be allowed to use *at the same time* non-GPL compatible nodes AND GPL nodes that don't provide the exception. 2. (apart from clause 1) Should we release the PL as GPL or LGPL? The LGPL would be less restrictive, but would also allow any proprietary Piper-like system to use Overflow as it's processing layer. What do you think? We'd also have to choose a license for the DL and BL. Right now, the DL has to be GPL, since it links with GPL code (libflowui is currently GPL) and it would be illegal to release it as LGPL. So I hope we can get the situation straight will less pain than KDE. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Thu Sep 7 11:27:32 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> Message-ID: <39B7B3E4.2DBB52F7@bioinformatics.org> In my opinion, we could license all of the parts to Piper under either the GPL OR LGPL, providing we clarify what is meant by "linked". GPL vs. LGPL: As Jean-Marc mentioned, the LGPL ("lesser" or "library" GPL) is meant to allow closed-source programs to "link" TO LGPL-licensed code. The GTK+ and GNOME libraries are both LGPL'd, which means a company like Applix can use them for Applixware with no legal problems. (The Qt library, however, has recently been re-licensed under the GPL, which would force such a company to use the alternative QPL and pay for it.) Is Piper a library? Well, sort of. I believe applications like BlueBox, ISYS, and OpenBSA will make use of Piper in such a way. And, ISYS is closed-source, which means the GPL might not work for them. But, the fact that the use of Piper by ISYS would take place through CORBA, is an issue. Is the use of CORBA considered "linking"? Jean-Marc says no. If not, GPL vs. LGPL is irrelevant to us. Neither the GPL or LGPL provides us with a good definition of "linking". I used to find this frustrating, since so much hinges on the definition. However, I now think this was intentional, because there is an enormous continuum of "linking" examples out there. It should thus be up to the author to define or clarify what is meant by "linking" and how it applies to their code. The LGPL would cover "one end" for us, without having to specify that "CORBA is not linking": It says that pretty much anything can link TO Piper. But, we would nonetheless be left to define what can be linked FROM Piper. I say we should put an exception in the license (either GPL or LGPL) for all programs, etc., that are "linked" as "nodes". The Linux kernel has the same exception for device drivers. Maybe I didn't answer the question that Jean-Marc proposed: GPL or LGPL? I just rephrased it :-) If we all agree that the ONLY uses of Piper would be via CORBA and nodes, then we should choose the ordinary GPL. Vote? Cheers. Jeff Jean-Marc Valin wrote: > > With the latest issues/wars (depends on how you see it!) with the KDE/Qt > licensing, I think it's now time to make sure not to end up with the same mess > (no matter which side you are, the result is that it has hurt KDE a lot) with > Piper. > > This is mostly important for the PL (Overflow), because CORBA linking is not > covered by the (L)GPL. Also, now's the time to change the Overflow (PL) license, > because all the copyrights are owned by three people: Me, Dominic, and Brad. The > three need to agree to change the license (should not be too hard!). If we wait > too long and too many people contribute, it'll be too late (there are hundreds > of copyright owners in the Linux kernel, changing license is now impossible). > For now, the license is GPL, since at first, it was not meant to be a used as > library. > > I think we all agreed (do we?) that the basis sould be around the GPL/LGPL. > Before I continue, here's my simple explanations on each. > > You may link any software (be it a program or a library, OSS or not) to an LGPL > library, while to may not link to a GPL library. (this is commonly known). > > You CANNOT link either LGPL or GPL software (library or program) to a library > that is not GPL-compatible (LGPL, GPL, and BSD are GPL-compatible, while the QPL > is not). This is the clause that caused so much problem with KDE and that made > it illegal, since libkdeui (which is LGPL) could not legally be linked to Qt, > which was not GPL-compatible. > > Now this is what I'd like to discuss: > > 1. Should we allow non GPL-compatible nodes to be used (my opinion is yes)? If > we want that, we need to provide with an exception to the GPL/LGPL that > explicitly allows it (like KDE needed to have so it could be linked with Qt). > The unclear part of this is would we be allowed to use *at the same time* > non-GPL compatible nodes AND GPL nodes that don't provide the exception. > > 2. (apart from clause 1) Should we release the PL as GPL or LGPL? The LGPL would > be less restrictive, but would also allow any proprietary Piper-like system to > use Overflow as it's processing layer. What do you think? > > We'd also have to choose a license for the DL and BL. Right now, the DL has to > be GPL, since it links with GPL code (libflowui is currently GPL) and it would > be illegal to release it as LGPL. > > So I hope we can get the situation straight will less pain than KDE. > > Jean-Marc -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Thu Sep 7 11:51:38 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> <39B7B3E4.2DBB52F7@bioinformatics.org> Message-ID: <39B7B98A.E57131C2@hermes.usherb.ca> > Is Piper a library? Well, sort of. I believe applications like BlueBox, > ISYS, and OpenBSA will make use of Piper in such a way. And, ISYS is > closed-source, which means the GPL might not work for them. But, the fact > that the use of Piper by ISYS would take place through CORBA, is an issue. Is > the use of CORBA considered "linking"? Jean-Marc says no. If not, GPL vs. > LGPL is irrelevant to us. If I understood what Richard Stallman had to say on the subject, linking to GPL code through CORBA (from closed-source) is legal, because it's a hole in the GPL. CORBA didn't existed (or wasn't widely used) when GPL version 2 was written. Since future versions of the GPL (remember we can link with any version newer that 2 with the current license) may patch the CORBA hole, I say: "if you want to be allowed to link with CORBA, release LGPL". > Neither the GPL or LGPL provides us with a good definition of "linking". I > used to find this frustrating, since so much hinges on the definition. > However, I now think this was intentional, because there is an enormous > continuum of "linking" examples out there. It should thus be up to the author > to define or clarify what is meant by "linking" and how it applies to their > code. Right now, linking means "static linking" and even the issue about dynamic linking isn't clean (let's assume that dynamic linking is also covered). > > The LGPL would cover "one end" for us, without having to specify that "CORBA > is not linking": It says that pretty much anything can link TO Piper. But, we > would nonetheless be left to define what can be linked FROM Piper. I say we > should put an exception in the license (either GPL or LGPL) for all programs, > etc., that are "linked" as "nodes". The Linux kernel has the same exception > for device drivers. Calling a program from inside a node is not linking. Only static/dynamic linking of code is. But the question still raises much more complex issues. For instance, can an LGPL library link to (not "be linked to") a GPL library. Let's say you have library Gtk+ was GPL instead of LGPL. Could libgnumoui (that links/uses Gtk+) be LGPL. One thing is sure is that it would be illegal to write a closed-source program that links with libgnomeui, since it would violate the GPL on Gtk+, which you need to link. Remember that this is fiction, since Gtk+ is released under the LGPL. So the real issue is: if the PL is LGPL, can it link to GPL-only libraries? Just for those who aren't familiar with it... PL nodes are classes that are part of libraries (I call them toolboxes) that are dynamically linked at startup. Even more complex is the fact that it could happen that GPL software AND closed-source software be linked from the PL is you run uses both proprietary nodes and GPL nodes. I think it would be illegal (the guy who wrote the GPL code didn't want closed-source software to be linked to it). I think we should try to wrap all our questions/doubts/fears in an e-mail and send that to either Richard Stallman or Bruce Perens (or both). What do you think? Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Thu Sep 7 11:57:25 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> Message-ID: <39B7BAE5.CD53BF1B@bioinformatics.org> Jean-Marc Valin wrote: > > 1. Should we allow non GPL-compatible nodes to be used (my opinion is yes)? If > we want that, we need to provide with an exception to the GPL/LGPL that > explicitly allows it (like KDE needed to have so it could be linked with Qt). > The unclear part of this is would we be allowed to use *at the same time* > non-GPL compatible nodes AND GPL nodes that don't provide the exception. Hmmm. I don't believe each node would need to provide the exception to the "linking restriction". It should be just Piper that provides it. However, Piper does allow "one node to be linked to another", so what are the licensing implications of that? This is a good question. I recall Richard Stallman writing at one time about the implications of Free Software being used by Application Service Providers (ASP's). The question someone posed was, "Do ASP's need to provide the source code for the Free Software a user executes via their service?" Stallman's response was that the user is not effectively "linking" to software, but is instead using it as a service, so the source code does not have to be available. The same is true for Web servers. Think about it: If you wrote a Web server and licensed it under the GPL, must every browser that accesses the server be GPL'd? Would you be required to provide the source code to every surfer who visits your site? No, of course not. I think we can make the same argument for nodes connecting to Piper and to each other: They are only providing a service to each other. They are not being linked to form a single code base. In any case, I believe the spirit of the "linking clause" in the L/GPL is that, by "linking", you are creating one code base that would result in a non-Free component restricting the use of a Free component. Piper is usable without any particular node, so I think we are in jive with that spirit. We should perhaps put some of this reasoning in our license modification. > 2. (apart from clause 1) Should we release the PL as GPL or LGPL? The LGPL would > be less restrictive, but would also allow any proprietary Piper-like system to > use Overflow as it's processing layer. What do you think? I don't know. We'll still be include-ing Overflow as a library from the BL, right? I suppose that's up to you guys, if you think Overflow might be used outside of Piper. A single license might be neater though. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Thu Sep 7 12:12:32 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> <39B7B3E4.2DBB52F7@bioinformatics.org> <39B7B98A.E57131C2@hermes.usherb.ca> Message-ID: <39B7BE70.8B7CBEF1@bioinformatics.org> Jean-Marc Valin wrote: > > If I understood what Richard Stallman had to say on the subject, linking to GPL > code through CORBA (from closed-source) is legal, because it's a hole in the > GPL. CORBA didn't existed (or wasn't widely used) when GPL version 2 was > written. Since future versions of the GPL (remember we can link with any version > newer that 2 with the current license) may patch the CORBA hole, I say: "if you > want to be allowed to link with CORBA, release LGPL". Good point. > Calling a program from inside a node is not linking. Only static/dynamic linking > of code is. But the question still raises much more complex issues. For > instance, can an LGPL library link to (not "be linked to") a GPL library. Let's > say you have library Gtk+ was GPL instead of LGPL. Could libgnumoui (that > links/uses Gtk+) be LGPL. One thing is sure is that it would be illegal to write > a closed-source program that links with libgnomeui, since it would violate the > GPL on Gtk+, which you need to link. Remember that this is fiction, since Gtk+ > is released under the LGPL. I recognized that problem when first reading the GPL and LGPL. Let me phrase it another way to see if we're talking about the same thing: The LGPL specifically allows something (linking) that is not allowed by the GPL. Would merging LGPL + GPL code bases then violate the GPL, or at least be very confusing? I think the intention of the LGPL is to be a "lesser" license, and to thus be superseded by the GPL when the two are combined. A clause in the new version of the LGPL actually permits an "upgrading" to the GPL (I think by anyone). So, maybe when the two are combined, the LGPL automatically defers to the GPL? > So the real issue is: if the PL is LGPL, can it link to GPL-only libraries? Just > for those who aren't familiar with it... PL nodes are classes that are part of > libraries (I call them toolboxes) that are dynamically linked at startup. Even > more complex is the fact that it could happen that GPL software AND > closed-source software be linked from the PL is you run uses both proprietary > nodes and GPL nodes. I think it would be illegal (the guy who wrote the GPL code > didn't want closed-source software to be linked to it). Oh, I see what you're talking about: Overflow nodes. Would you say that they are statically/dynamically linked at compile time? > I think we should try to wrap all our questions/doubts/fears in an e-mail and > send that to either Richard Stallman or Bruce Perens (or both). What do you > think? Heh. Stallman is actually really nice about replying to e-mails. He replied to more than a couple of mine. I don't know about Perens. Eric Raymond never replied to my e-mails, even when Stallman CC'd them to him. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jarl at casema.net Fri Sep 8 01:56:57 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> <39B7B3E4.2DBB52F7@bioinformatics.org> <39B7B98A.E57131C2@hermes.usherb.ca> Message-ID: <39B87FA9.8810BC4B@casema.net> > If I understood what Richard Stallman had to say on the subject, linking to GPL > code through CORBA (from closed-source) is legal, because it's a hole in the > GPL. CORBA didn't existed (or wasn't widely used) when GPL version 2 was > written. Since future versions of the GPL (remember we can link with any version > newer that 2 with the current license) may patch the CORBA hole, I say: "if you > want to be allowed to link with CORBA, release LGPL". The only place we've this Corba 'linking' problem is with 3th party UI's, isn't it? If we want people to be 100% free to build UI's we should go for LGPL. > > Right now, linking means "static linking" and even the issue about dynamic > linking isn't clean (let's assume that dynamic linking is also covered). Linking seems to be a though problem to us I think. Imaging this: Piper UI -> Piper DL -> '3th party BL' -> PL What if this 3th party BL registers to our central coordination database, or what if it screws to authentication meganism? It can be done very easily if one were to recode some core parts of the BL. Like has happend to icq. Clones line gnomeicu 'parasite' upon mirabilis. gnapser this to Napster. etc. I whould therefore see we licence our communication formats (like the XML format, corba API) very strict! Something that would allow people to build UI's, link TO Piper (not link Piper INTO some other app), and 'hook' apps into the PL. I see much chaos wonce parts of piper are to be 'emulated' by some 3th party. bye, jarl From jean-marc.valin at hermes.usherb.ca Thu Sep 7 17:28:04 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> <39B7B3E4.2DBB52F7@bioinformatics.org> <39B7B98A.E57131C2@hermes.usherb.ca> <39B87FA9.8810BC4B@casema.net> Message-ID: <39B80864.E38AE131@hermes.usherb.ca> > The only place we've this Corba 'linking' problem is with 3th party UI's, isn't it? There might be some other cases, but I think 3rd part UI's would be the most important. > > If we want people to be 100% free to build UI's we should go for LGPL. This means the DL should be LGPL, but the BL's than run on each machine could be GPL. ...it all depends on how you see the linking... with remote comunications, this becomes very blured. For instance, you can't ask that the user of a GPL'd ftp client only connects to GPL'd ftp servers... of that mozilla (which is GPL) only goes to sites that run apache. The same way, I don't think we can forbid non-free BL's that can be connected to the rest of the system (even if everything else is GPL'ed). > Piper UI -> Piper DL -> '3th party BL' -> PL > > What if this 3th party BL registers to our central coordination database, or what > if it screws > to authentication meganism? It can be done very easily if one were to recode some > core > parts of the BL. Like has happend to icq. Clones line gnomeicu 'parasite' upon > mirabilis. > gnapser this to Napster. etc. > > I whould therefore see we licence our communication formats (like the XML format, > corba API) very strict! > Something that would allow people to build UI's, link TO Piper (not link Piper INTO > some other app), and > 'hook' apps into the PL. I see much chaos wonce parts of piper are to be 'emulated' > by some 3th party. We cannot (and shouldn't) prevent 3rd party BL's. The only way to do that is through proprietary (patented) protocols and we don't want to do that. As for the authentication mecanism, it has to work for any (even hostile BL's). Napster had problems because it's assumed all its clients were the official clients. This kind of security is flawed, even more in OSS. If a 3rd party BL can screw up the authentication, then nothing prevents a cracker to modify your BL source to screw it up. This means that your BL is not secure. This is why we sould not try to prevent 3rd-party BL's (that's the whole point of open-source: someone can modify your work). However, if the BL links with the PL, then depending on the PL license, a closed-source BL may or may not be legal... but then they could provide their own PL and there'd be nothing we can do. Before we start going into license (in)compatibilities, let's start by deciding what kind of things we want to prevent. We know we don't want people to take our code and make it proprietary. That leaves us with GPL and LGPL. Now is there any kind of "linking" we want to prevent? If not, then we can almost LGPL everything. If not, we need to see what needs to be GPL and what needs to be LGPL. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jarl at casema.net Fri Sep 8 02:49:59 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> <39B7B3E4.2DBB52F7@bioinformatics.org> <39B7B98A.E57131C2@hermes.usherb.ca> <39B87FA9.8810BC4B@casema.net> <39B80864.E38AE131@hermes.usherb.ca> Message-ID: <39B88C17.BBBCC457@casema.net> > We cannot (and shouldn't) prevent 3rd party BL's. The only way to do that is > through proprietary (patented) protocols and we don't want to do that. As for > the authentication mecanism, it has to work for any (even hostile BL's). Napster > had problems because it's assumed all its clients were the official clients. > This kind of security is flawed, even more in OSS. If a 3rd party BL can screw > up the authentication, then nothing prevents a cracker to modify your BL source > to screw it up. Do you happen to know if some firm that has this problem somehow took legal action against this. Or is this not possible? (I remember this aol vrs M$ struggle about the M$ messenger and Icq. How did thius end, or is this still going on?) > > This means that your BL is not secure. This is why we sould not > try to prevent 3rd-party BL's (that's the whole point of open-source: someone > can modify your work). However, if the BL links with the PL, then depending on > the PL license, a closed-source BL may or may not be legal... but then they > could provide their own PL and there'd be nothing we can do. Ok, I see the problem. Legal stuff gives me headaches :) You're right about us fixing our own problems. bye, jarl From jeff at bioinformatics.org Thu Sep 7 18:12:32 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> <39B7B3E4.2DBB52F7@bioinformatics.org> <39B7B98A.E57131C2@hermes.usherb.ca> <39B87FA9.8810BC4B@casema.net> <39B80864.E38AE131@hermes.usherb.ca> Message-ID: <39B812D0.60115D17@bioinformatics.org> Jean-Marc Valin wrote: > > > The only place we've this Corba 'linking' problem is with 3th party UI's, isn't it? > > There might be some other cases, but I think 3rd part UI's would be the most > important. Regardless of whether we choose the LGPL or GPL, we can always make a modification that explicitly permits CORBA linking or any other kind of linking. That's something we should keep in mind. > Before we start going into license (in)compatibilities, let's start by deciding > what kind of things we want to prevent. We know we don't want people to take our > code and make it proprietary. That leaves us with GPL and LGPL. Now is there any > kind of "linking" we want to prevent? If not, then we can almost LGPL > everything. If not, we need to see what needs to be GPL and what needs to be > LGPL. Jarl has a good point in that we may want to prevent one of the layers from being substituted with a proprietary program. This may be more of an advocacy issue than a security issue, since, as Jean-Marc stated, an Open Source layer can be just as malicious. On the other hand, there may be some "control" issues here. Let's pretend that Micro$oft (perhaps we are having some delusions of grandure) wants to "Embrace, Extend, and Extinguish" Piper. It could do so by creating its own BL (or another layer) that uses a SLIGHTLY different protocol that only works with their UI. They've done this before: Look at how many Web pages work only with IE. But, if we are to make everything GPL, and say that CORBA linking from a non GPL'd layer is illegal for DL <-> BL <-> PL linking, then it has to be illegal for UIL <-> DL linking too. It's a tough choice. We (at least Jarl and I) want the former to be illegal but the latter to not be. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Thu Sep 7 20:47:38 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> <39B7B3E4.2DBB52F7@bioinformatics.org> <39B7B98A.E57131C2@hermes.usherb.ca> <39B87FA9.8810BC4B@casema.net> <39B80864.E38AE131@hermes.usherb.ca> <39B812D0.60115D17@bioinformatics.org> Message-ID: <39B8372A.B95ACFE7@hermes.usherb.ca> > Regardless of whether we choose the LGPL or GPL, we can always make a > modification that explicitly permits CORBA linking or any other kind of > linking. That's something we should keep in mind. In their current form, the GPL and LGPL allow linking with CORBA. > > > Before we start going into license (in)compatibilities, let's start by deciding > > what kind of things we want to prevent. We know we don't want people to take our > > code and make it proprietary. That leaves us with GPL and LGPL. Now is there any > > kind of "linking" we want to prevent? If not, then we can almost LGPL > > everything. If not, we need to see what needs to be GPL and what needs to be > > LGPL. > > Jarl has a good point in that we may want to prevent one of the layers from > being substituted with a proprietary program. This may be more of an advocacy > issue than a security issue, since, as Jean-Marc stated, an Open Source layer > can be just as malicious. Well, if you have a (static or dynamic) linking, then you can prevent that. Otherwise, you can't. For instance, You cannot release under the GPL an ftp client for which you forbid to connect to a closed-source ftp server. I would go as far as saying that such a restriction would make your license non GPL-compatible. Actually, locking such 3rd parties would be playing the very same game as all the big bad software vendors. ...It would be like having the gcc license saying you cannot compile closed-source programs with it! > On the other hand, there may be some "control" issues here. Let's pretend > that Micro$oft (perhaps we are having some delusions of grandure) wants to > "Embrace, Extend, and Extinguish" Piper. It could do so by creating its own > BL (or another layer) that uses a SLIGHTLY different protocol that only works > with their UI. They've done this before: Look at how many Web pages work only > with IE. Actually, locking out 3rd parties implementations would be playing the very same game as all the big bad software vendors. The only thing we could try doing is to patent the protocol and allow free use as long as you don't make a protocol modification proprietary. It would be like a GPL-protocol, but I'm really not sure we can (and want to) do that. > But, if we are to make everything GPL, and say that CORBA linking from a non > GPL'd layer is illegal for DL <-> BL <-> PL linking, then it has to be illegal > for UIL <-> DL linking too. It's a tough choice. We (at least Jarl and I) > want the former to be illegal but the latter to not be. In theory, you could allow CORBA linking for some parts, and not for others... However, as I stated above, the GPL and LGPL *allow* CORBA linking from and to closed source programs and I think adding a restriction would make Piper GPL-incompatible and we do not want to do that. You can allow more that stated in the (L)GPL, but you cannot add restriction. The "allow CORBA exception" was only to clarify the matter, but right now it is legal anyway. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Fri Sep 8 07:20:46 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] Licensing issues References: <39B6E956.344C767@hermes.usherb.ca> <39B7B3E4.2DBB52F7@bioinformatics.org> <39B7B98A.E57131C2@hermes.usherb.ca> <39B87FA9.8810BC4B@casema.net> <39B80864.E38AE131@hermes.usherb.ca> <39B812D0.60115D17@bioinformatics.org> <39B8372A.B95ACFE7@hermes.usherb.ca> Message-ID: <39B8CB8E.98624A1A@bioinformatics.org> Jean-Marc Valin wrote: > > You can allow more that stated in the (L)GPL, but you cannot add restriction. True, I did not consider that. However, we must keep in mind that at some point CORBA linking may become illegal for the GPL. If we choose the GPL, we can specifically permit CORBA linking for certain cases. Other cases could then be left illegal for non-GPL programs (3rd party BL, for example). But, do we want to wait? Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jarl at casema.net Sat Sep 9 07:16:42 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] BL->PL design Message-ID: <39BA1C1A.D3AD6394@casema.net> Hi All, here the irc log between Jean-Marc and me. It's about the BL->PL design. There's might be info missing because there's a lot discussed via email. JM offered to write a document about the design us two are happy about. ok, I think even on the bandwidth cq network delay issue we agree. My point is that the user will know where the network bottlenecks will be, the BL won't somewhere is a turning point though, when 200% ram is requested for say, the schedular tends to de-lacalisation What do you mean? localisation should be the 1th thing the schedular should distribute.. de-localisation = distribute node to remote machine but I would like to pute more logic into the schedular then just 'do what the xml says' Well, there are some issues that don't distribute well... if you consider a network which will be network-bound and not CPU(or RAM)-bound jep.. maybe we can weight total network execution time? Finding a split that minimizes the network links is a bit like solving the traveling salesman problem. traveling prob <- that's why I promote this non-localised methode It's already an NP problem... if you try to distribute it, it just won't work. but seems we're agreed on most... maybe only details now :) or do you see more issues not ok? Well, I think deciding on how the splitting will be done is an important issue. :) but I think you have want you need, or what you're missing? or see an obstacle? what do you mean? the monitoring, the naming, the corba stuff hmm.. what more had been discussed? the scheduling... it's being optional you still see a need for a 'central BL' meganism? By scheduling, you mean decide which node runs where, right (I called that splitting)? jep, splitting = scheduling Yes, I still think we really need a central scheduler. darn :) why? It will already be hard enough to have a good scheduler... I don't see have we can make a good, distributed scheduler ...sorry "HOW we can make..." hmm.. the optimization has to be done globally... You cannot just the BL's take what they want. I think it can be done oops... "just LET the BL's take..." not if you take into account the network. even if we assume all outputs have the same size This is equivalent to "solve the traveling salesman problem with distributed machines that don't talk to each other" kinda hard to explain why I think very simple rules will do the trick I'll link nodes (their execution binary or xml text) to execution times.. when the local machine is free it will go there.. such rules very simple rules that have local impact no 'all descibing' mathamatics Can you explain me how? forget my last line, it was supposed to go somewhere else! it wont be perfect, but it will fluctuate around the optimum wonce you introduce enough weighted values into the algorithm execution time ram load cpu load bandwidth all such issue are simply calculated with constanf factors and indicate where the node runs. local execution having the biggest multiplyer I don't understand you objection to running the scheduler locally... Or could we start with a local scheduler and when (or if) we need a distributed one, we can write it? sure but I think this is usage of piper, should be some options in the UI for defnining where and how you want to run This is how I see the optimal (though not necessarly practical, but still) scheduler: (I'm assuming that all required resources are known) You first need to have a function that, given the node districution(scheduling) returns how long it would take to run. given on history? Then you use an optimization technique (most likely genetic algorithms) to find out there the global minimum of the function is. ie what's the scheling that minimizes the total time it takes to run the "program" This can only be done if the "evaluation function" knows where each node is located. ok. so the BL that gets the xml data uploaded from the DL should do ALL the scheduling and then distribute? Yes. That's what I think. needs much cpu power.. needs uploading of recources useage back to central host? That's the only way to solve the problem. Not all problems distribute well... otherwise, all the super-computers would have ween replaced by Beowolf clusters... I think GP could do well, but not really real time. not the resource usage, but the resources available on each system (or maybe we're not talking about the same thing) Well, the genitic algorithm part could distribute some crunching to the other machines if it's that complicated... but the problem itself is resolved locally. Also, I think that if the system is too complex for a local scheduler to run fast, it will totally confuse a distributed scheduler anyway. I think the question is changed a bit: what will we do 1st. I must admit this GP distributor is quite nice, but I hope to see these rule-set based version running too I say we start with two modes for now: 1) the user does the scheduling and 2) local scheduling... later if we can try a distributed one if you really like. rule based scheduling... how? confuse a distributed scheduler? all confusion is taken care of in distribution of next network, so it will 'stabalize' around some situation Confuse in the sense that the "solution" found by a distributed scheduler will be so non-optimal... ...what you might as well distributed at random hehe speak again once it's running :) ok, but at first we wont have an GP schedular, so maybe we should 1st work on the 100% defnined version 100% defnined version? and when all the part commnunicate we'll work on a more sofistaced one I said the GP scheduler would be the optimal scheduler, but of course we shouldn't start with that. 100% definied <- all node have a fixed location\machine OK, 100%-defined first... I think the best system would be an interactive scheduler. One that talks to the UI and helps the user distributing. and also ok with this (very simple) PL main Corba thinggy? talks <- yer :) The UI could say: "I think you're putting too much stuff on machine XYZ". would be very very sweet is this voice regoc ready for such? This would give you the real optimal scheduler. (remember that the GP thing was optimal if the system knew everything about required resuirces... the user already has a better idea of what nodes do) ic ...talk in the weak sense... the interactive scheduler communicates with the UI Do we agree on most of the system? think so for a while at least :-) it's just about what goes 1st I think about the PL, did you agree if we called it Baby-BL and that it would link to the PL? (in my e-mail) the part I called PL main Corba thinggy? Something like that... ok, as long as you agree with using corba between BL and PL\BabyBL Sure... the way the BL communicates with it's other parts is up to you. I don't even know how to write an ftp client, so I'm not going to tell you what to do for this part! L) but I'll need you input for this baby BL. ...You may not like the naming... but the local scheduler is actually what I was calling grandma-BL (after all, it's responsible for the brokering) "I'll need you input for this baby BL" what do you mean? pointing me at the enrty point of the Overflow library how it can be wrapped best Sure, I'll help. but it's quite easy to understand. ok Do you have the Overflow (PL) code downloaded? it's something for later.. i'm currently buzy working on the C++ port of BL downloaded <- yes It's as simple as calling: UIDocument *doc = new UIDocument(argv[1]); doc->load(); doc->run(param); This is the actual code in batchflow.cc that starts the processing... this run, is this only called inside the last node? ir is a UIDocument a complete network? run calls getOutput() on the last node... and everything starts... a UIDocument is the equivalent of an .m file in matlab. However, when distributing, the big document will be split in several smaller UIDocument. Each one will be sent to a Baby-BL yes does each UIDocument need some special nodes? Not in overflow... but for Piper, we will need special nodes for all the inputs of a (splitted) UIDocument. This special node will call a Baby-BL function to get the result from another part of the program. as in my A->B->C example, when we decide that B->C goes on the different machine than A, we need to "rewrite" the part as S->B->C, where S is a special node that will ask the BL to fetch the result from A. ic routing the lose ends true networks ?? lose ends <- end of node chain on one machine, a splitted network that's located over multiple machines and the up- and downloading ofcourse hmm.. no, no uploading only result uplodaing darn sorry.. yes, the only "special node" required will be for result uploading no downloading extra nodes , but nodes that transport the results back yes ...well, I'm all mixed up with the up-down point of view... the special node is at the input of the network. o, why there? nah, never mind but anyways, the DL will be suppying this extra node also? the Baby-BL receives a getOutput CORBA "signal" (don't know how to call it). it calls getOutput() on its PL network... some computation is done and the special input node will call a (BBL) callback function, which will call getOutput() on another BL. ok get it Extra DL nodes... not really. There will be debugging/viewing nodes I don't know if we should say they are DL, GUI, or Pl nodes... uhhh I meant to say was if the DL add this extra node into the xml description? However (I don't know whether you were refering to that), there will be a communication system between the DL and the PL, so that nodes can send warnings/errors to the user. ...sorry, now I understand what you mean!... ok.. will this commenction go directly from DL to PL? conn... arf I'm not sure whether it'll be the DL job, the scheduler's, or the BL... it's 3:40 now :( I mean the DL/BL/... job to add the node to the document... I think it should be the scheduler... ok BL but I'll let you sleep now...... hehe nah good idear Do you mean you're still OK to discuss? ...or the "nah" was for something else? sure, I just make much more spelling errors and bollocks talk at there times :) "nah" was a out-lout thinking.. but not yet sleeping that is Anyway... who will add the "BL2PL" node is a detail. right I was talking to Brad about the DL-PL communication system. I'm just asking some to get the picture of what the babyBL will be like We though we could have "pseudo-packets" that could be transmitted by the BL. sure The Baby-BL will be quite simple. simple <- nice The DL address will be simple: "DL" since there's only one. A node address will be like "machine:BabyBL ID:node pointer" maybe we could use some sort of number system that's used Piper wide that defines locations of nodes\dl's\etc? the machine could be an IP of a name, the BabyBL ID is used to discriminate many BBL's on the same machine. Since the BBL lives in the same address space as the PL, node pointers are OK. right What sort of number system? something like this you're just describing.. DL->BL we introduced this URI numbering I described in my email... which e-mail? I dont really care how, as long as we use the same numbering uhh... mom..I'll copy paste #Define Piper URI id number (Univ. Relocation Identification, every Piper part can be access by this number: 342.0.0 = BL or UI instance, I'm not sure about this one. 342.168.0 = Subnet (or user space) 168 on machine 342 342.168.4281 = Node 4281 inside network 168 on machine 342. just a scretch, neven really critised Well, I think it's equivalent to what I was saying, right? BTW... I've just thought about something you might not like to hear... all this translation to C++ you've been doing is for what I call the mother-BL (the one that does the authentication), right? yes originaly because it was to link to Overflow :( ...if this part only communicates to the DL and the Baby-BL (thus the PL) though CORBA, maybe the translation to C++ wasn't necessary (or am I just completly lost)? hehe kinda sad :) Is the translation complete or are you still having trouble with CORBA? but it's much cleaner alfter all to have one Orb library used and so ...then you at least gain something. Brad helped my out with some examples I just finished the removal of the old corba code, and will do the new ones starting tomorrow ...so you're staying with C++? If so, have you started using the STL. If it's not that hard, it would be nice to slowly replace glib by it. C++ COrba is much more simple as C bindings luckily if I had to go the other way around it woud be much worst nah, I havent started using any C++ feature.. I just worked hard on letting the C++ compiler to shut up but I hope to learn while coding I wouldn't like to have to translate Overflow to C. I use (and need to) just about all the C++ features that C doesn't have (templaces, virtual functions, double inheritence, ...)! maybe a lot of old lines can be replaced with much better functions but for now I know C and related libraries... wont speed up things if I was to do everything in advanced C++ yet Sure... makes sense... when you know more and have some time it might be a good idea jer, slowly we learn When you're done with the C++ CORBA thing, we could start writing the BabyBL together. ok, maybe you can code the logic \ skeleton, and I can code the corba portals? Sure, as I said, the way I see it so far, the non-CORBA part will probably be around one hundred lines of code. bye, jarl From chapmanb at arches.uga.edu Sat Sep 9 12:47:06 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] BL->PL design In-Reply-To: <39BA1C1A.D3AD6394@casema.net> Message-ID: <200009091647.MAA89196@archa13.cc.uga.edu> Hey thanks for posting this Jarl! A couple of quick comments below: > JM offered to write a document about the design us > two are happy about. I would *very* much like to see this. Right now I am really lost as to where Piper is going next and what stuff we should be coding on. > I meant to say was if the DL add this extra node into the xml > description? I'm not exactly sure if this is what you guys were talking about here, but right now the XML description that the DL outputs should be immediately runnable by the PL. Recently I wrapped the UIDocument::run() function and ran a couple of XML descriptions produced by the DL through this and they ran without problems. This is definately not perfect, but should work for most cases. > A node address will be like "machine:BabyBL ID:node pointer" > maybe we could use some sort of number system that's used Piper > wide that defines locations of nodes\dl's\etc? I'd be really interested to hear what we are going to do for this point. Right now we've got a numbering system for the dl2bl communication (which Jarl talked about in the IRC) but I don't know if this is completely compatible with what Jean-Marc wants needs. In my mind it would be very useful to create a standard representation of a node that we could use throughout Piper and that could be serialized and unserialized by all of the different parts. Since my last IRC conversation with Jean-Marc I've been thinking a lot about this and how important it is to have a way to pass these nodes around between the different parts. I don't know if I really know the best way to do this, but I thought I would throw it out to see if anyone has any suggestions/thoughts on it.... Thanks again for posting this. I'm glad that Jarl and Jean-Marc are coming together on a design they both like :-). Brad From jeff at bioinformatics.org Sat Sep 9 13:02:16 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] BL->PL design References: <200009091647.MAA89196@archa13.cc.uga.edu> Message-ID: <39BA6D18.EC2EC834@bioinformatics.org> Brad Chapman wrote: > > > JM offered to write a document about the design us > > two are happy about. > > I would *very* much like to see this. Right now I am really lost as to > where Piper is going next and what stuff we should be coding on. Beyond the many goals of Piper that we have already discussed (on this list and in documents), at the lowest levels of detail, we'll have to do things democratically, with a lot of discussions between us. I think it is beyond any one of us to dictate the how every piece of code need be written. Even Linus Torvalds doesn't hand out coding assignments to contributors :-) I'm happy to see the IRC conversations and some of the details for DL <-> BL <-> PL interaction being worked out. This is surely how we should keep proceeding. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Sat Sep 9 13:09:05 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] BL->PL design References: <200009091647.MAA89196@archa13.cc.uga.edu> Message-ID: <39BA6EB1.BE9004CE@hermes.usherb.ca> > > JM offered to write a document about the design us > > two are happy about. > > I would *very* much like to see this. Right now I am really lost as to > where Piper is going next and what stuff we should be coding on. Oops.. perhaps I got misunderstood... I don't really have time for that... at least not until I come back from Chicago (Cpeech Coding Workshop... not everybody is doing bioinformatics :-) ) on the 20th. > > > I meant to say was if the DL add this extra node into the xml > > description? > > I'm not exactly sure if this is what you guys were talking about here, > but right now the XML description that the DL outputs should be > immediately runnable by the PL. Recently I wrapped the UIDocument::run() > function and ran a couple of XML descriptions produced by the DL through > this and they ran without problems. This is definately not perfect, but > should work for most cases. What Jarl means is that when we break up a network in parts, all the inputs that are not connected anymore will have to be connected a a special node that will ask for the input to the BL. What he asked was: sould the DL add this node before passing the sub-network to the PL of is it somebody else's job. (This is not a major issue right now... and we'll figure that out later) > > > A node address will be like "machine:BabyBL ID:node pointer" > > maybe we could use some sort of number system that's used Piper > > wide that defines locations of nodes\dl's\etc? > > I'd be really interested to hear what we are going to do for this point. > Right now we've got a numbering system for the dl2bl communication (which > Jarl talked about in the IRC) but I don't know if this is completely > compatible with what Jean-Marc wants needs. In my mind it would be very > useful to create a standard representation of a node that we could use > throughout Piper and that could be serialized and unserialized by all of > the different parts. Since my last IRC conversation with Jean-Marc I've > been thinking a lot about this and how important it is to have a way to > pass these nodes around between the different parts. I don't know if I > really know the best way to do this, but I thought I would throw it out > to see if anyone has any suggestions/thoughts on it.... I think Jarl's numbering scheme is roughly the same as what I am suggesting, but all with numbers. I think we agree on the principle of a uniform address scheme. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Sat Sep 9 13:58:48 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] where is piper going? References: <200009091647.MAA89196@archa13.cc.uga.edu> Message-ID: <39BA7A58.A0678E20@bioinformatics.org> Where is Piper going next? Who knows? Actually, we do have a pretty clear idea, but we want to keep our options open. Historically, Piper's design has always been in a state of flux, and I think that is a Good Thing. Piper started out (from the "Loci" end) as "The Lowell Package". It was supposed to be "the GNU equivalent of The Wisconsin Package (or GCG)", a commercial, mostly console-based, suite of sequence analysis tools. I don't even know if there is anyone on this list who was with me back then. And then about one and a half years ago, I discovered that Piper (under the new acronym "TULIP") had the same design goal as a project called "EMBOSS": http://www.uk.embnet.org/Software/EMBOSS/ I spent a bit of time afterward concerned that we would be duplicating the efforts of another GPL'd project. Anyone who knows me knows that that is what drives my decision making with respect to my own projects. So, I decided that we should not concentrate on tools, as EMBOSS did, but on an infrastructure for distributing tools. Shortly thereafter, I decided to not make Piper (which became "Loci" after TULIP) specifically for bioinformatics, for reasons some here have heard over and over: A general-purpose system would attract more users, and thus developers, which would be better for bioinformatics users in the long run. We would not have Jarl (GMS) and Jean-Marc (Overflow) collaborating with us had I not made that decision. And, just prior to GMS and Overflow merging with Loci, Piper was only going to provide interfaces between existing programs. Overflow introduced the whole concept of "small nodes" to us (and Loci would provide "nodes over the Internet" for Overflow). So, Piper (from any end) has been through a lot of changes. Had a design be written immutably in stone, none of us would be reading this e-mail right now. FLEXIBILITY, rather than a pre-defined path, has been, and always will be, most important. But, our objectives are becoming clearer. We know at this point that Piper is not a Post-It Notes program :-) We have written a number of documents on the overall objective. Check out the poster written by Brad: http://www.bioinformatics.org/bradstuff/ismb2.ps (warning: very large) Along with the e-mail I sent to Cameron Laird about Piper vs. .NET, we have 2 very recent and clearly thought out design documents. But things can still change. What should we be coding next? Actually, I should be used to these questions from Brad. He told me very early on that he's a code monkey and not a coordinator. What we should code on next is whatever anyone thinks will help us reach the most recently drafted design goals. I can't dictate the coding any closer than that, and I probably shouldn't. We've had some heated arguments at the times I told Brad what should be done with his code. Plus, I don't have the depth of knowledge that Jean-Marc and Jarl have regarding how their particular code bases can be developed. I hope everyone understands my position on the direction of Piper. I don't want anyone to get the impression at any time that I have abandoned the project or that it is in complete chaos. We're all too dedicated and smart for that. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jarl at casema.net Sun Sep 10 00:20:06 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] where is piper going? References: <200009091647.MAA89196@archa13.cc.uga.edu> <39BA7A58.A0678E20@bioinformatics.org> Message-ID: <39BB0BF6.8BAA2284@casema.net> > Where is Piper going next? Who knows? Actually, we do have a pretty clear > idea, but we want to keep our options open. I think there're are three objectives attracted to form what Piper will be: 1. The needs to have a tool that links application to each other 2. A network of nodes that are executed in a powerfull environment 3. A platform that offers a better way of building a system on top Think everybody can link these points to specific Piper participants :) > > What should we be coding next? Actually, I should be used to these questions > from Brad. He told me very early on that he's a code monkey and not a > coordinator. What we should code on next is whatever anyone thinks will help > us reach the most recently drafted design goals. I can't dictate the coding > any closer than that, and I probably shouldn't. We've had some heated > arguments at the times I told Brad what should be done with his code. Plus, I > don't have the depth of knowledge that Jean-Marc and Jarl have regarding how > their particular code bases can be developed. We still need to finish the 1st phase of creating Piper: getting a pilot to actually run :) bye, jarl From jarl at casema.net Sun Sep 10 00:23:06 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] BL->PL design References: <200009091647.MAA89196@archa13.cc.uga.edu> <39BA6EB1.BE9004CE@hermes.usherb.ca> Message-ID: <39BB0CAA.53E82455@casema.net> > > > > I would *very* much like to see this. Right now I am really lost as to > > where Piper is going next and what stuff we should be coding on. > > Oops.. perhaps I got misunderstood... I don't really have time for that... at > least not until I come back from Chicago (Cpeech Coding Workshop... not > everybody is doing bioinformatics :-) ) on the 20th. OK, that leaves this work to me :( I'll hope to have it done tomorrow.. > I think Jarl's numbering scheme is roughly the same as what I am suggesting, but > all with numbers. I think we agree on the principle of a uniform address scheme. Thinking same thing, JM and I (and Brad because he participated in the dl->bl scretch api) want to have the same thing: a Piper wide relocation id. I dont really care how it will be actually implemented, as long as it will be there. From jean-marc.valin at hermes.usherb.ca Sat Sep 9 15:28:55 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:29 2006 Subject: [Pipet Devel] where is piper going? References: <200009091647.MAA89196@archa13.cc.uga.edu> <39BA7A58.A0678E20@bioinformatics.org> Message-ID: <39BA8F77.BF778031@hermes.usherb.ca> I would just like to state my opinion on "where is piper going?". I think the answer will be much clearer once we have at least a basic integration of all the layers. When we have the GUI, DL, BL and PL talking together, we'll see what it can do and what would be nice to have. Until we reach that point, I don't think we should decide anything. I don't know how stated this law of development: "Make it work, make it right, make it fast". We're still in the make it work phase. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jean-marc.valin at hermes.usherb.ca Sat Sep 9 15:47:53 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] Baby-BL implementation Message-ID: <39BA93E9.507EDEC6@hermes.usherb.ca> This is mostly for Jarl and Brad, but I'd like to keep the others informed: I've attached how I see the Baby-BL (which links to the PL) being implemented. This is short and I also expect the full implementation to be fairly small too. ...at least before we start putting all kinds of features. (When some comments are replaced by actual code, it could be the first working version of the Baby-BL) Comments are welcome. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca -------------- next part -------------- #include #include int main() { //BL and PL initialization //get the network to run and the parameters from the mother-BL ParameterSet params = ...; UINetwork netToRun = ...; Network *net = netToRun->build("MAIN", p); net->initialize(); //Wait for corba events and call the corresponding methods } //This method is called remotly using CORBA ObjectRef getOutput(int output_id, int count) { return net->last_node->getOutput(output_name, count); } //This method will be called by the 'Special PL node' that will be automatically connected // to dangling inputs after the scheduling (network splitting) is done. ObjectRef fetchInput (URI node_location, int output_id, int count) { //Ask the BL to ask getOutput to the BL that owns the node specified by 'node_location' return result. } From jean-marc.valin at hermes.usherb.ca Sat Sep 9 15:53:15 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] oops... this one instead Message-ID: <39BA952B.AC723CDE@hermes.usherb.ca> -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca -------------- next part -------------- #include #include //This Baby-BL program will be called by the mother-BL that runs on each machine. //This executable will be linked against the PL libraries. int main() { //BL and PL initialization //get the network to run and the parameters from the mother-BL ParameterSet params = ...; UINetwork netToRun = ...; //Create a runnable network from the UINetwork. Network *net = netToRun->build("MAIN", p); //perform initialization net->initialize(); //Wait for corba events and call the corresponding methods } //This method is called remotly using CORBA ObjectRef getOutput(int output_id, int count) { return net->last_node->getOutput(output_name, count); } //This method will be called by the 'Special PL node' that will be automatically connected // to dangling inputs after the scheduling (network splitting) is done. ObjectRef fetchInput (URI node_location, int output_id, int count) { //Ask the BL to ask getOutput to the BL that owns the node specified by 'node_location' return result. } From jarl at casema.net Sun Sep 10 05:27:50 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] Baby-BL implementation References: <39BA93E9.507EDEC6@hermes.usherb.ca> Message-ID: <39BB5415.8F3AD17F@casema.net> > I've attached how I see the Baby-BL (which links to the PL) being implemented. > Comments are welcome. Thnx for this jm, it's a bit early for me to go too much into detail of the baby BL code. Looks good what you attached. I think the BBL is indeat goin to be a very small program. Some corba code, some small function set and we're done. From jean-marc.valin at hermes.usherb.ca Sun Sep 10 02:14:17 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] GPL/LGPL issues Message-ID: <39BB26B9.B2F08439@hermes.usherb.ca> Hi, I am a developer working on the Piper (http://bioinformatics.org/piper/) project. We are at the point where we should decide which of the Free licenses (GPL, LGPL) to use for the project. However, we find that there some unclear (to us) issues with the (L)GPL when it comes to what is defined as "linking". The first one involves a library, libdata-flow which uses plugins by opening other shared libraries using dlopen(). Is that considered linking? If libdata-flow's license is the LGPL, would it be allowed to open a GPL plugin? would it be allow to open a closed-source plugin? (or both at the same time?) Does the license of the program that links to libdata-flow change the answer to the previous questions? Our second issue has to do with CORBA. Is it allowed by the GPL to use CORBA to link a GPL'd program and a closed-source program? I hop you can help us resolve our licensing problems. Thanks, Jean-Marc Valin -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jarl at casema.net Mon Sep 11 03:09:14 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] BL PL design documentation Message-ID: <39BC851A.45A9BE44@casema.net> Hi All, As promised I've made a draft about the BL & PL design. Please supply me with anything you can think of that I missed, so I can finalize the document soon. bye, jarl -- http://sunsite.auc.dk/gms -------------- next part -------------- A non-text attachment was scrubbed... Name: BL_PL.html.tgz Type: application/octet-stream Size: 9817 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000910/af9bcf42/BL_PL.html.obj From jeff at bioinformatics.org Sun Sep 10 19:19:54 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] BL PL design documentation References: <39BC851A.45A9BE44@casema.net> Message-ID: <39BC171A.FE6CE4D2@bioinformatics.org> Question: What about the "Grandma BL"? Also, do you really want to call these components "Grandma, Mother, and Baby"? It's perhaps a little corny :-) I would prefer "Primary, Secondary, and Tertiary". Then you can abbreviate them "PBL, SBL, and TBL" or "BL1, BL2, and BL3" or whatever. Jeff jarl van katwijk wrote: > > Hi All, > > As promised I've made a draft about the BL & PL design. Please > > supply me with anything you can think of that I missed, so I can > > finalize the document soon. -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Mon Sep 11 00:46:29 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] BL PL design documentation References: <39BC851A.45A9BE44@casema.net> <39BC171A.FE6CE4D2@bioinformatics.org> Message-ID: <39BC63A5.FEC42336@hermes.usherb.ca> > Question: What about the "Grandma BL"? > > Also, do you really want to call these components "Grandma, Mother, and > Baby"? It's perhaps a little corny :-) I would prefer "Primary, Secondary, > and Tertiary". Then you can abbreviate them "PBL, SBL, and TBL" or "BL1, BL2, > and BL3" or whatever. Well, there is a real meaning between these grandma-mother-baby qualifiers, that would be lost with a BL1, BL2, BL3 scheme. You need only one grandma-BL (Jarl doesn't use the term, but in my mind it's the part responsible for "centralized distribution") to control a whole "piper program". You need many mother-BL's (one on each machine) for one grandma-BL. The same way, there are many baby-BL's for one mother-BL (many baby-BL's on each machine). If you don't like baby and grandma, what about super-BL (SBL), mother-BL (MBL) and child-BL (CBL)? BTW Jarl, your document really reflects what we agreed on friday! Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jarl at casema.net Mon Sep 11 11:22:57 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] BL PL design documentation References: <39BC851A.45A9BE44@casema.net> <39BC171A.FE6CE4D2@bioinformatics.org> Message-ID: <39BCF8D1.BA296384@casema.net> > Question: What about the "Grandma BL"? We only use the BL and the babyBL. bye, jarl From rms at gnu.org Mon Sep 11 05:35:53 2000 From: rms at gnu.org (Richard Stallman) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] Re: GPL/LGPL issues In-Reply-To: <39BB26B9.B2F08439@hermes.usherb.ca> (message from Jean-Marc Valin on Sun, 10 Sep 2000 02:14:17 -0400) References: <39BB26B9.B2F08439@hermes.usherb.ca> Message-ID: <200009110935.DAA12998@wijiji.santafe.edu> The first one involves a library, libdata-flow which uses plugins by opening other shared libraries using dlopen(). Is that considered linking? In general, I believe it makes a combined program. However, in some trivial cases maybe one could make the argument that they are two separate programs that just happen to run in the same address space. If libdata-flow's license is the LGPL, would it be allowed to open a GPL plugin? Certainly. You can link LGPL-covered code with GPL-covered code in any fashion. would it be allow to open a closed-source plugin? (or both at the same time?) I don't want to speak about the categories of "open source" and "closed source", because that would be letting people make a mistake. I disagree fundamentally with the Open Source Movement, and if people think I support it, my work will all promote the wrong message. If I use their terminology in conversation, I will be encouraging that problem. See http://www.gnu.org/philosophy/free-software-for-freedom.html for more explanation. In the Free Software Movement we distinguish between free software and non-free; these are not the same categories as open or closed source, because the standards they adopted in 1998 are less strict than the ones we use. An open-source program is probably free software, but you can't be certain without checking the license. So if I used the terms "open source" and "closed source", I would also be talking about the wrong things. I can give you advice about free software based on the views of the Free Software Movement; if you would like advice from the Open Source Movement, you have to ask them. What I can say about the question of non-free plug-ins is that using the LGPL would definitely permit them. Is that a good idea? It depends on details of the situation, but in general I think it is a bad idea. It is better to insist that all add-ons must be free software. Otherwise we will face a constant struggle to develop free add-ons to compete with the non-free ones. Does the license of the program that links to libdata-flow change the answer to the previous questions? Yes. Our second issue has to do with CORBA. Is it allowed by the GPL to use CORBA to link a GPL'd program and a closed-source program? I can only speak about non-free programs, not about closed-source. However, I think that *in general* two modules that communicate through CORBA are separate programs, so that the license of each one does not cover the other. From chapmanb at arches.uga.edu Mon Sep 11 17:08:17 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] Baby-BL implementation In-Reply-To: <39BA93E9.507EDEC6@hermes.usherb.ca> Message-ID: <200009112108.RAA32974@archa10.cc.uga.edu> Hey all; Apologies in advance for being late on responding to all of this. I have just now come to some understanding of what is going on, and didn't want to respond when I was so confused. Sorry for being so slow on coming to grips with this! Jean-Marc wrote: > I've attached how I see the Baby-BL (which links to the PL) being > implemented. > Comments are welcome. This makes a lot of sense and seems very practical: 1. Fire up a CORBA server in the BBL and wait for calls from the MBL. 2. Call getOutput upon demand from the MBL to run a network (or subsection of a network). 3. Call fetchInput upon demand from the PL to get input from a particular node that may be remotely located. So, if I am with you so far, this just leads to a couple of questions: 1. It seems like this requires that the MBL be smart enough to know that it needs to split a network at display nodes, so that it can have access to the results that need to be displayed. So if we we had Jean-Marc's famous A->(B->C)->D network and say C were a viewer that needed to get results, then the MBL would have to know to split the network here and call getOutput on C (before calling on D), so that is could have the output for display. Is this right, or am I way off base? 2. I'm not sure how the Overflow *Probe will work under this system. Am I missing something basic? Otherwise it looks great. You and Jarl seem to really have a good system down here. Please let me know how you'd like me to help with the implementation details of the BBL stuff (if at all!) and I'll be more than happy to do some coding. Brad From chapmanb at arches.uga.edu Mon Sep 11 17:28:48 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] BL PL design documentation In-Reply-To: <39BC851A.45A9BE44@casema.net> Message-ID: <200009112128.RAA152736@archa11.cc.uga.edu> Jarl wrote: > As promised I've made a draft about the BL & PL design. Please > supply me with anything you can think of that I missed, so I can > finalize the document soon. This looks really nice Jarl. Thanks for writing this up so clearly and concisely. I just have one question about it all: You only mention briefly the idea of the centralized name-server that Piper has to connect with to know about other Piper instances on the net. >From your listing of tasks, it appears as if you would like the BL to be the place where all of this communication occurs, and hence the place where the name-server stuff would be implemented. You only talk about KQML remote communication, so I'm not sure exactly how this implementation will work. What are your thoughts on this? I would like to argue that we give the DL the responsibility of finding other available Piper instances, and then returning this info to the BL (we could design an addition to the dl2bl.idl interface to handle the BL requesting the info from the DL). I think this is good for a couple of reasons: 1. This way we can write and prototype the whole thing in python, which, IMHO, is very nice for designing CORBA stuff in. I'm quite worried about fighting with C++ memory issues, especially for a long running name-server. 2. I'm a little worried about the BL becoming huge and unweildy if it gets assigned a ton of tasks (it already has communicating with the DL and BBL, doing node splitting, GA stuff, using KQML to communicate with remote BLs). 3. I think this is a nice separation between "high-level" remote communication (ie. finding out what other Piper programs are out there) versus "low-level" remote communication (ie. querying for getOutput() requests). What do you guys think? Am I completely off the wall, or does this make some sense? Jeff wrote: > Also, do you really want to call these components "Grandma, Mother, > and Baby"? It's perhaps a little corny :-) I would prefer "Primary, > Secondary, and Tertiary". Then you can abbreviate them "PBL, SBL, and > TBL" or "BL1, BL2, and BL3" or whatever. Nah, you can give my +1 to keeping the "family names." Not only is it more descriptive of what is going on, it is also a heck of a lot more interesting, even if it may be a bit corny :-). Man, if we can't be a little corny, what else do we have left? Brad From chapmanb at arches.uga.edu Mon Sep 11 17:44:49 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] GPL/LGPL issues In-Reply-To: <200009110935.DAA12998@wijiji.santafe.edu> Message-ID: <200009112144.RAA110552@archa13.cc.uga.edu> Hey guys; I have been following with interest the whole licensing discussion, although I really don't know much about it (since thinking about lawyer type things make my head hurt). Based on what Richard Stallman wrote (thanks for writing to him, Jean-Marc -- it really helped clarify things, at least for me). Here are my comments on the whole thing right now: 1. Programs communicating by CORBA are separate programs, and so the license won't matter. It doesn't seem like this will change soon. So people can write replacements for any of the Piper parts if they want. Oh well, that was a design choice, right? I still think it is a good one to allow people to rewrite sections in a language independent manner. So we shouldn't worry about this. 2. Then I guess all we have to worry about is free versus non-free plugins. If we want people to be able to write non-free plugins that link with Piper, we should go for the LGPL, if not the GPL is our choice. 3. I think we should have one license for every part (as long as everyone can agree). This will make things a whole lot clearer about this issue. So, based on all of this, I give my vote to an LGPL license. If someone at some company wants to use Piper, and then wants to write a proprietary plugin to deal with proprietary data, I don't see a big problem with that. Having people in companies be able to use Piper and extend it is a big plus to us, since then we will have more testers and users. Many of these people may work in corporate settings and write in-house only code, but may also contribute back to Piper. I don't want to lose these users. So anyways, those are my thoughts about licensing. What do other people think right now? Brad From jean-marc.valin at hermes.usherb.ca Mon Sep 11 18:20:52 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] Baby-BL implementation References: <200009112108.RAA32974@archa10.cc.uga.edu> Message-ID: <39BD5AC4.3613D019@hermes.usherb.ca> > This makes a lot of sense and seems very practical: > > 1. Fire up a CORBA server in the BBL and wait for calls from the MBL. > > 2. Call getOutput upon demand from the MBL to run a network (or > subsection of a network). > > 3. Call fetchInput upon demand from the PL to get input from a particular > node that may be remotely located. > > So, if I am with you so far, this just leads to a couple of questions: You're totally on the track so far. > 1. It seems like this requires that the MBL be smart enough to know that > it needs to split a network at display nodes, so that it can have access > to the results that need to be displayed. So if we we had Jean-Marc's > famous A->(B->C)->D network and say C were a viewer that needed to get > results, then the MBL would have to know to split the network here and > call getOutput on C (before calling on D), so that is could have the > output for display. Is this right, or am I way off base? If by "display nodes", you mean viewers, then I'll answer for the next question. > 2. I'm not sure how the Overflow *Probe will work under this system. Am I > missing something basic? You're right, they won't work! My "implementation" was only meant to include the basic functionalities, but for the probes/viewers, we'll need to add a sendMessage() and a distributeMessage() to the CORBA stuff in order to have communication between the DL and PL. BTW, there is no reason to do anything particular about the splitting for the probes. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Mon Sep 11 18:22:51 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] BL PL design documentation References: <39BC851A.45A9BE44@casema.net> <39BC171A.FE6CE4D2@bioinformatics.org> <39BC63A5.FEC42336@hermes.usherb.ca> Message-ID: <39BD5B3B.D39822B@bioinformatics.org> Jean-Marc Valin wrote: > > Well, there is a real meaning between these grandma-mother-baby qualifiers, that > would be lost with a BL1, BL2, BL3 scheme. You need only one grandma-BL (Jarl > doesn't use the term, but in my mind it's the part responsible for "centralized > distribution") to control a whole "piper program". You need many mother-BL's > (one on each machine) for one grandma-BL. The same way, there are many baby-BL's > for one mother-BL (many baby-BL's on each machine). If you don't like baby and > grandma, what about super-BL (SBL), mother-BL (MBL) and child-BL (CBL)? Then how about "1st Generation Broker" (G1B), "2nd Generation Broker" (G2B), and "3rd Generation Broker" (G3B)? "Generation" implies the same thing as "Parent/Child". Let me know if I've got this right: G1B (a.k.a. "Granny"): Part of the DL code base. Spawns or communicates with multiple G2B's G2B (a.k.a. "Mommy"): The BL code base written by Jarl. Spawns or communicates with multiple G3B's G3B (a.k.a. "Baby"): Part of the PL code base. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Mon Sep 11 18:30:09 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] GPL/LGPL issues References: <200009112144.RAA110552@archa13.cc.uga.edu> Message-ID: <39BD5CF1.8F6D70A6@hermes.usherb.ca> > 1. Programs communicating by CORBA are separate programs, and so the > license won't matter. It doesn't seem like this will change soon. So > people can write replacements for any of the Piper parts if they want. Oh > well, that was a design choice, right? I still think it is a good one to > allow people to rewrite sections in a language independent manner. So we > shouldn't worry about this. You're right, the only thing I was a bit worried was that it would change in later versions of the (L)GPL. > 2. Then I guess all we have to worry about is free versus non-free > plugins. If we want people to be able to write non-free plugins that link > with Piper, we should go for the LGPL, if not the GPL is our choice. It's not as simple as that. You can link a closed-source program with an LGPL library, but whether you can link the library (libdata-flow) to a closed source plugin (library) is unclear. Remember that although libkdeui was LGPL, you still couldn't link it to Qt when it was QPL (at least not within the KDE context). > 3. I think we should have one license for every part (as long as everyone > can agree). This will make things a whole lot clearer about this issue. Do you want "one license for every part" or the same license for all the parts, it's unclear to me? Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From chapmanb at arches.uga.edu Mon Sep 11 18:33:49 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] Baby-BL implementation In-Reply-To: <39BD5AC4.3613D019@hermes.usherb.ca> Message-ID: <200009112233.SAA118774@archa12.cc.uga.edu> Jean-Marc wrote: > If by "display nodes", you mean viewers, then I'll answer for the next > question. Yup, that's what I meant! > You're right, they won't work! My "implementation" was only meant to > include the basic functionalities, but for the probes/viewers, we'll need to add > a sendMessage() and a distributeMessage() to the CORBA stuff in order to > have communication between the DL and PL. Okay, this makes sense. Basics first :-). I just didn't know if what you sent already included the probe/viewer stuff in it, or if that was to come. Then I am completely in support of what you have -- let's go for it! We can add this other stuff on later. Thanks for clearing this up for me. > BTW, there is no reason to do anything > particular about the splitting for the probes. I think this makes sense since we'll have the sendMessage()/distributeMessage() stuff. But we can worry about this more later on when we start implementing it! After the basics are done. Brad From jean-marc.valin at hermes.usherb.ca Mon Sep 11 18:37:39 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] BL PL design documentation References: <39BC851A.45A9BE44@casema.net> <39BC171A.FE6CE4D2@bioinformatics.org> <39BC63A5.FEC42336@hermes.usherb.ca> <39BD5B3B.D39822B@bioinformatics.org> Message-ID: <39BD5EB3.3B60D4D8@hermes.usherb.ca> "J.W. Bizzaro" a ?crit : > > Then how about "1st Generation Broker" (G1B), "2nd Generation Broker" (G2B), > and "3rd Generation Broker" (G3B)? "Generation" implies the same thing as > "Parent/Child". I don't understand what you have against this mother-child (or maybe parent-child) thing. This is the standard terminology eg. when it comes to processes (child process, parent process). It's still clearer than talking about generation, which can mean more than one thing (eg. that we scrapped generation 1 and 2 before rewriting generation 3). As for grandma, it might not even contain BL at the end (it might just be called scheduler, DL2BL, ...) Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From chapmanb at arches.uga.edu Mon Sep 11 18:42:47 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:30 2006 Subject: [Pipet Devel] GPL/LGPL issues In-Reply-To: <39BD5CF1.8F6D70A6@hermes.usherb.ca> Message-ID: <200009112242.SAA92610@archa11.cc.uga.edu> Jean-Marc wrote: [CORBA stuff] > You're right, the only thing I was a bit worried was that it would > change in > later versions of the (L)GPL. Right, but I guess it doesn't sound like it from Richard Stallman's mail, so I guess we shouldn't speculate about it if the lawyers aren't! [...my simplistic view of linking...] > It's not as simple as that. You can link a closed-source program with > an LGPL library, but whether you can link the library (libdata-flow) to a > closed source plugin (library) is unclear. So what does this mean? That there is no way that anyone could "legally" write a non-free plugin, even if we use a LGPL license? Does this relate to just using a plugin or to writing and "distributing"/selling a plugin? Damn, licenses make my head hurt! > Do you want "one license for every part" or the same license for all > the parts, > it's unclear to me? I am for the same overall license for all of the parts. I would like Piper to just be one big program and all be covered together under the exact same license. At least, that is my vote :-). Brad From jeff at bioinformatics.org Mon Sep 11 18:49:52 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] BL PL design documentation References: <39BC851A.45A9BE44@casema.net> <39BC171A.FE6CE4D2@bioinformatics.org> <39BC63A5.FEC42336@hermes.usherb.ca> <39BD5B3B.D39822B@bioinformatics.org> <39BD5EB3.3B60D4D8@hermes.usherb.ca> Message-ID: <39BD6190.26F9433F@bioinformatics.org> Jean-Marc Valin wrote: > > I don't understand what you have against this mother-child (or maybe > parent-child) thing. This is the standard terminology eg. when it comes to > processes (child process, parent process). It's still clearer than talking about > generation, which can mean more than one thing (eg. that we scrapped generation > 1 and 2 before rewriting generation 3). As for grandma, it might not even > contain BL at the end (it might just be called scheduler, DL2BL, ...) I don't know. It just sounds corny to me. If you use "grandparent", "parent", "child" (no discrimination against grandpa's and fathers ;-)), it's fine with me. BTW, did I get this right?: Grandparent: Part of the DL code base. Spawns or communicates with multiple Parents Parent: The BL code base written by Jarl. Spawns or communicates with multiple Children. Child: Part of the PL code base. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Mon Sep 11 18:54:09 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] Re: GPL/LGPL issues References: <39BB26B9.B2F08439@hermes.usherb.ca> <200009110935.DAA12998@wijiji.santafe.edu> Message-ID: <39BD6291.4D4D669C@bioinformatics.org> Richard Stallman wrote: > > Our second issue has to do with CORBA. Is it allowed by the GPL to use CORBA to > link a GPL'd program and a closed-source program? > > I can only speak about non-free programs, not about closed-source. > However, I think that *in general* two modules that communicate > through CORBA are separate programs, so that the license of each one > does not cover the other. It doesn't sound like CORBA linking between a GPL'd program and a non-GPL'd program will become illegal then. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Mon Sep 11 18:58:40 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] GPL/LGPL issues References: <200009112144.RAA110552@archa13.cc.uga.edu> Message-ID: <39BD63A0.A65D7552@bioinformatics.org> Brad Chapman wrote: > > 1. Programs communicating by CORBA are separate programs, and so the > license won't matter. It doesn't seem like this will change soon. So > people can write replacements for any of the Piper parts if they want. Oh > well, that was a design choice, right? I still think it is a good one to > allow people to rewrite sections in a language independent manner. So we > shouldn't worry about this. I suppose if there is a bit of brokering taking place in each of the "layers" of Piper (e.g., Grandparent is part of DL), then it would not be possible to completely replace all brokering components (if we use the GPL). > 2. Then I guess all we have to worry about is free versus non-free > plugins. If we want people to be able to write non-free plugins that link > with Piper, we should go for the LGPL, if not the GPL is our choice. Give me an example of a "plugin" for Piper. Will a plugin work via CORBA? > 3. I think we should have one license for every part (as long as everyone > can agree). This will make things a whole lot clearer about this issue. One license for all of Piper? I agree with that. > So, based on all of this, I give my vote to an LGPL license. If someone > at some company wants to use Piper, and then wants to write a proprietary > plugin to deal with proprietary data, I don't see a big problem with > that. Having people in companies be able to use Piper and extend it is a > big plus to us, since then we will have more testers and users. Many of > these people may work in corporate settings and write in-house only code, > but may also contribute back to Piper. I don't want to lose these users. I think it will depend on whether plugins use CORBA or not. If they all do, then we don't really need the LGPL. We can use the GPL. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From chapmanb at arches.uga.edu Mon Sep 11 19:07:53 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] BL PL design documentation In-Reply-To: <39BD6190.26F9433F@bioinformatics.org> Message-ID: <200009112308.TAA64854@archa13.cc.uga.edu> Jeff wrote: [mother-child stuff] > I don't know. It just sounds corny to me. If you use "grandparent", > "parent", "child" (no discrimination against grandpa's and fathers > ;-)), it's fine with me. I don't know Jeff, I think you should let this drop. It doesn't seem like a big deal and we have already started using grandma, mother and baby for things. We are really over-analyzing the whole thing. I think you should just go with the majority on this one :-). > BWT, did I get this right? > > Part of the DL code base. Spawns or communicates with > multiple Parents. I don't think this is correct. Jarl's document seems to make an explicit point that there is only one BL (Parent) per Piper program running. So the DL will only communicate with one BL. The BL itself will be responsible for splitting processes up using the whole Mother to Baby thing. At least, this is my understanding. Brad From chapmanb at arches.uga.edu Mon Sep 11 19:10:39 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] GPL/LGPL issues In-Reply-To: <39BD63A0.A65D7552@bioinformatics.org> Message-ID: <200009112310.TAA74506@archa13.cc.uga.edu> Jeff wrote: > Give me an example of a "plugin" for Piper. Will a plugin work via > CORBA? I think a plugin is any "node" that runs under Piper. So a TextViewer is a plugin, a NetExec is a plugin, etc. I think that there should be a CORBA interface to write plugins in, but there is also the current linking system (writing the plugins in C++), so I think there will be both CORBA and non-CORBA plugins, eventually. Brad From jeff at bioinformatics.org Mon Sep 11 19:20:16 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] GPL/LGPL issues References: <200009112144.RAA110552@archa13.cc.uga.edu> <39BD5CF1.8F6D70A6@hermes.usherb.ca> Message-ID: <39BD68B0.C2E10F4E@bioinformatics.org> Jean-Marc Valin wrote: > > It's not as simple as that. You can link a closed-source program with an LGPL > library, but whether you can link the library (libdata-flow) to a closed source > plugin (library) is unclear. Remember that although libkdeui was LGPL, you still > couldn't link it to Qt when it was QPL (at least not within the KDE context). You can link a program to a closed-source library if you specifically permit it. The problem with KDE was that they never specifically permitted linking to Qt. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Mon Sep 11 19:27:30 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] BL PL design documentation References: <200009112308.TAA64854@archa13.cc.uga.edu> Message-ID: <39BD6A62.FE4406A4@bioinformatics.org> Brad Chapman wrote: > > I don't know Jeff, I think you should let this drop. It doesn't seem like > a big deal and we have already started using grandma, mother and baby for > things. We are really over-analyzing the whole thing. I think you should > just go with the majority on this one :-). Hey, haven't I come up with some great names so far? :-) Just don't get carried away with the theme and start calling parts of the BBL, "Diaper", "Pacifier", "Bottle", etc. ;-) > I don't think this is correct. Jarl's document seems to make an explicit > point that there is only one BL (Parent) per Piper program running. So > the DL will only communicate with one BL. The BL itself will be > responsible for splitting processes up using the whole Mother to Baby > thing. At least, this is my understanding. Oh, so one mother per grandma, and many babies per mother? Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Mon Sep 11 19:30:35 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] GPL/LGPL issues References: <200009112310.TAA74506@archa13.cc.uga.edu> Message-ID: <39BD6B1B.202EF326@bioinformatics.org> Brad Chapman wrote: > > Jeff wrote: > > Give me an example of a "plugin" for Piper. Will a plugin work via > > CORBA? > > I think a plugin is any "node" that runs under Piper. So a TextViewer is > a plugin, a NetExec is a plugin, etc. I think that there should be a > CORBA interface to write plugins in, but there is also the current > linking system (writing the plugins in C++), so I think there will be > both CORBA and non-CORBA plugins, eventually. Okay. But, as I've mentioned a number of times before, we need to write an exception for these in our license. Providing they are operating as nodes, they can use any license. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Mon Sep 11 19:40:00 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] license nominations Message-ID: <39BD6D50.F664DA96@bioinformatics.org> Here's my nomination: GPL (not LGPL) for all of Piper, with 2 modifications and clarifications: (1) A UI written to work via CORBA can be in any license (CORBA is not "linking"), and (2) plugins that operate as "nodes" can be in any license (they are not "linked" to from Piper). ("Linking" is as defined in the GPL.) My reason for choosing the GPL is that it will provide better "protection" for the infrastructure created by Piper, as argued by Jarl a couple days ago. Other nominations? Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Mon Sep 11 19:55:24 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] GPL/LGPL issues References: <200009112144.RAA110552@archa13.cc.uga.edu> <39BD5CF1.8F6D70A6@hermes.usherb.ca> <39BD68B0.C2E10F4E@bioinformatics.org> Message-ID: <39BD70EC.24751178@hermes.usherb.ca> > You can link a program to a closed-source library if you specifically permit > it. The problem with KDE was that they never specifically permitted linking > to Qt. You are right, however, adding the exception ("you can link to closed-source") *might* make the "modified" license incompatible with the GPL, meaning if we allow the exception, then we cannot link to GPL'd plugins anymore (ie. we couldn't use FFTW anymore). Of course, I'm not sure of that and that's why I asked RMS about it. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jean-marc.valin at hermes.usherb.ca Mon Sep 11 20:01:31 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] BL PL design documentation References: <200009112308.TAA64854@archa13.cc.uga.edu> Message-ID: <39BD725B.54316D07@hermes.usherb.ca> > I don't think this is correct. Jarl's document seems to make an explicit > point that there is only one BL (Parent) per Piper program running. So > the DL will only communicate with one BL. The BL itself will be > responsible for splitting processes up using the whole Mother to Baby > thing. At least, this is my understanding. There will be one grandma-BL per Piper program running, however there will be many mother-BL, one per machine, and many baby-BL per machine. Is that what you meant? Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Mon Sep 11 20:06:21 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] GPL/LGPL issues References: <200009112144.RAA110552@archa13.cc.uga.edu> <39BD5CF1.8F6D70A6@hermes.usherb.ca> <39BD68B0.C2E10F4E@bioinformatics.org> <39BD70EC.24751178@hermes.usherb.ca> Message-ID: <39BD737D.39597BB5@bioinformatics.org> Jean-Marc Valin wrote: > > You are right, however, adding the exception ("you can link to closed-source") > *might* make the "modified" license incompatible with the GPL, meaning if we > allow the exception, then we cannot link to GPL'd plugins anymore (ie. we > couldn't use FFTW anymore). Of course, I'm not sure of that and that's why I > asked RMS about it. RMS said, "What I can say about the question of non-free plug-ins is that using the LGPL would definitely permit them." It really all depends on whether Piper is a program using libraries, or is library used by programs. We can specify it either way in the license. I think RMS pretty clearly states that the LGPL will allow Piper, acting as a library to the plugins, to use any kind of plugin. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Mon Sep 11 20:09:31 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] license nominations References: <39BD6D50.F664DA96@bioinformatics.org> Message-ID: <39BD743B.C9CFAA6E@bioinformatics.org> I'll nominate a second license option: LGPL for all of Piper, specifying that Piper is a library and that all UI's and plugins/nodes are using Piper as such (a library). Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Mon Sep 11 20:17:06 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] license nominations References: <39BD6D50.F664DA96@bioinformatics.org> Message-ID: <39BD7602.C190917D@hermes.usherb.ca> > GPL (not LGPL) for all of Piper, with 2 modifications and clarifications: (1) > A UI written to work via CORBA can be in any license (CORBA is not "linking"), > and (2) plugins that operate as "nodes" can be in any license (they are not > "linked" to from Piper). ("Linking" is as defined in the GPL.) > > My reason for choosing the GPL is that it will provide better "protection" for > the infrastructure created by Piper, as argued by Jarl a couple days ago. > > Other nominations? First, the LGPL only makes sense for a library, not for a program, so licensing the UI under the GPL makes sense (I think everybody will agree on that). For the PL, I'm still undecided, but I think I'd like to have it released under the LGPL (remember, the PL is also the Overflow project). The same applies to libflowui, which is included in the DL. I have no real opinion about the BL, but since Jarl wants it GPL, it's OK with me. Now, a bit about plugins... Actually, I don't want to use the term and only refered to it so I could be understood by Piper-unaware people. The term I want to use is toolbox. A toolbox is a .so library that can contain any number of node implementations. When it starts, the PL looks for all the toolboxes in its path and loads them all (with dlopen). What I'm afraid is that if dlopen-ing a .so file is considered linking by the GPL then, regardless of the license used for the PL, using a GPL'd toolbox at the same time as a closed-source toolbox would be illegal, since the GPL'd toolbox would be linked to the closed-source toolbox. The only solution I see if dlopen-ing is considered linking is to release the PL under a dual GPL-LGPL license, so that you could legally chose between having the right to use GPL'd plugins or not. So what I'm thinking about is: UI: GPL DL: GPL (except for libflowui, which is part of Overflow and would be LGPL) BL: GPL (mother, baby, grandma) PL: Dual license -> GPL + (LGPL+linking modification) Toolboxes: any Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Mon Sep 11 20:40:11 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] license nominations References: <39BD6D50.F664DA96@bioinformatics.org> <39BD7602.C190917D@hermes.usherb.ca> Message-ID: <39BD7B6B.C04ACD12@bioinformatics.org> Jean-Marc Valin wrote: > > First, the LGPL only makes sense for a library, not for a program, so licensing > the UI under the GPL makes sense (I think everybody will agree on that). Well, that is another issue. The UIL can contain any number of UI's that work with Piper through CORBA. Perhaps you're talking about Pied and Peep. They too may have plugins (UI-dependent widgets), which I believe would be treated as libraries just like any GTK+ widget. > Now, a bit about plugins... Actually, I don't want to use the term and only > refered to it so I could be understood by Piper-unaware people. The term I want > to use is toolbox. A toolbox is a .so library that can contain any number of > node implementations. When it starts, the PL looks for all the toolboxes in its > path and loads them all (with dlopen). What I'm afraid is that if dlopen-ing a > .so file is considered linking by the GPL then, regardless of the license used > for the PL, using a GPL'd toolbox at the same time as a closed-source toolbox > would be illegal, since the GPL'd toolbox would be linked to the closed-source > toolbox. The only solution I see if dlopen-ing is considered linking is to > release the PL under a dual GPL-LGPL license, so that you could legally chose > between having the right to use GPL'd plugins or not. So, you're saying that Any licensed PL + GPL'd .so + non-GPL'd .so = Illegal linking Can you give me an example of a non-GPL'd .so that you might want to use? This is something every GPL'd program has to deal with. There are limitations, and KDE exceeded the limitations, allowing the creation of GNOME as a competitor. Of course, this is not the same issue as wrapping a pre-existing program to run as a node. I think we can consider that to NOT be linking as defined by the GPL. That's what I've been talking about regarding license modifications. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Mon Sep 11 20:50:11 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] license nominations References: <39BD6D50.F664DA96@bioinformatics.org> <39BD7602.C190917D@hermes.usherb.ca> <39BD7B6B.C04ACD12@bioinformatics.org> Message-ID: <39BD7DC3.AD1DE22@hermes.usherb.ca> > Well, that is another issue. The UIL can contain any number of UI's that work > with Piper through CORBA. Perhaps you're talking about Pied and Peep. They > too may have plugins (UI-dependent widgets), which I believe would be treated > as libraries just like any GTK+ widget. I was thinking about a simple UI program... if you have libraries going there it might change... I would settle for the license the majority decides for the UIL. > So, you're saying that > > Any licensed PL + GPL'd .so + non-GPL'd .so = Illegal linking Yes, if you plus(+) sign actually means linking in the GPL-sense. What I am unsure about is that dlopen-ing a library explicitly is considered linking by teh GPL. > Can you give me an example of a non-GPL'd .so that you might want to use? Well, you could have a node that links with the matlab library to do some stuff. Some nodes could link to other OSS libraries that are not GPL-compatible (apache license, artistic, old-style BSD, ...). Or there could be vendors that supply closed-source nodes, just like you can have closed-source drivers in Linux. I see the PL as a language, so you could compare it to gcc. The gcc license doesn't say "you cannot use gcc to link programs with closed-source libraries". > Of course, this is not the same issue as wrapping a pre-existing program to > run as a node. I think we can consider that to NOT be linking as defined by > the GPL. That's what I've been talking about regarding license modifications. Wraping an executable is definitly not linking, so it's OK with any license we use. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jarl at casema.net Wed Sep 13 04:02:04 2000 From: jarl at casema.net (Jarl van Katwijk) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] Baby-BL implementation References: <200009112108.RAA32974@archa10.cc.uga.edu> Message-ID: <39BF347C.1BCF9BD3@casema.net> > > 1. It seems like this requires that the MBL be smart enough to know that > it needs to split a network at display nodes, so that it have access > to the results that need to be displayed. So if we we had Jean-Marc's > famous A->(B->C)->D network and say C were a viewer that needed to get > results, then the MBL would have to know to split the network here and > call getOutput on C (before calling on D), so that is could have the > output for display. Is this right, or am I way off base? I think you're on track, but I dont really get this "a viever halfway a network". I always though every network has just one input on the start and one viewer at the end of a nodes network? > > > 2. I'm not sure how the Overflow *Probe will work under this system. Am I > missing something basic? > The "*Probe" system? Please explain. bye, jarl From jarl at casema.net Wed Sep 13 04:36:44 2000 From: jarl at casema.net (Jarl van Katwijk) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] BL PL design documentation References: <200009112128.RAA152736@archa11.cc.uga.edu> Message-ID: <39BF3C9C.B2679966@casema.net> > You only mention briefly the idea of the centralized name-server that > Piper has to connect with to know about other Piper instances on the net. > >From your listing of tasks, it appears as if you would like the BL to be > the place where all of this communication occurs, and hence the place > where the name-server stuff would be implemented. This central coordination database will a separate application outside the scope of the 4 Layers of Piper (opposing the 666 Layers of Hell :) ). It will register all BL together with their 'allowed users list' so that a BL can locate other BL's where it can distribute nodes to. > > > You only talk about KQML remote communication, so I'm not sure exactly > how this implementation will work. What are your thoughts on this? In order to have BL's communicate across a distributed network, KQML is the API that is specialized for this job. I cant go into details for now simply because I'm not really familiar with the KQML api yet. But I've been reading about this and everybody that has something intelligent to say about handling communications within a distributed framework say kqml is the way to go. Ofcourse it will be KQML over Corba. I'll try to integrate settled KQML technology into the BL like http://www.forwiss.uni-erlangen.de/~msnutt/kapi/kapi-2.7d.tgz. Also see: http://www.cs.umbc.edu/kqml/kqmlspec/spec.html > > I would like to argue that we give the DL the responsibility of finding > other available Piper instances, and then returning this info to the BL > (we could design an addition to the dl2bl.idl interface to handle the BL > requesting the info from the DL). I think this is good for a couple of > reasons: > > 1. This way we can write and prototype the whole thing in python, which, > IMHO, is very nice for designing CORBA stuff in. I'm quite worried about > fighting with C++ memory issues, especially for a long running > name-server. Right. > > 2. I'm a little worried about the BL becoming huge and unweildy if it > gets assigned a ton of tasks (it already has communicating with the DL > and BBL, doing node splitting, GA stuff, using KQML to communicate with > remote BLs). Getting some work out of my hand? Sure! :) > > 3. I think this is a nice separation between "high-level" remote > communication (ie. finding out what other Piper programs are out there) > versus "low-level" remote communication (ie. querying for getOutput() > requests). ok, the DL will handle Piper instance localisation.. maybe you can also think about this cantral database that will supply this info IYKWIM? > > Nah, you can give my +1 to keeping the "family names." Not only is it > more descriptive of what is going on, it is also a heck of a lot more > interesting, even if it may be a bit corny :-). Man, if we can't be a > little corny, what else do we have left? > We'll only have 2 pices of code that bare the name BL: THE BL and a PL wrapper that people call the BabyBL. This name is ok to me, but much clearer would be to call it "The PL plugin for the BL". I think you can see why. It's just a diff. way of looking at the situation.. bye, jarl From jarl at casema.net Wed Sep 13 04:39:51 2000 From: jarl at casema.net (Jarl van Katwijk) Date: Fri Feb 10 19:20:31 2006 Subject: [Pipet Devel] Baby-BL implementation References: <200009112108.RAA32974@archa10.cc.uga.edu> <39BD5AC4.3613D019@hermes.usherb.ca> Message-ID: <39BF3D56.FB4DD1CD@casema.net> > > > 2. I'm not sure how the Overflow *Probe will work under this system. Am I > > missing something basic? > > You're right, they won't work! My "implementation" was only meant to include the > basic functionalities, but for the probes/viewers, we'll need to add a > sendMessage() and a distributeMessage() to the CORBA stuff in order to have > communication between the DL and PL. BTW, there is no reason to do anything > particular about the splitting for the probes. I'll takes notes of all updates to the BL-PL design... when the mail-flow dies dry I'll update the ducoment and post it, bye, jarl From jarl at casema.net Wed Sep 13 04:42:06 2000 From: jarl at casema.net (Jarl van Katwijk) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] BL PL design documentation References: <39BC851A.45A9BE44@casema.net> <39BC171A.FE6CE4D2@bioinformatics.org> <39BC63A5.FEC42336@hermes.usherb.ca> <39BD5B3B.D39822B@bioinformatics.org> Message-ID: <39BF3DDD.E9C6D9EC@casema.net> > Let me know if I've got this right: > > G1B (a.k.a. "Granny"): > > Part of the DL code base. Spawns or communicates with > multiple G2B's NO grannies, no 3th BL, arrrghh :) This issue of 'a distributor' wont be handled by a separate 'layer', it will be part of the (main) BL. > > > G2B (a.k.a. "Mommy"): > > The BL code base written by Jarl. Spawns or communicates with > multiple G3B's > > G3B (a.k.a. "Baby"): > > Part of the PL code base. > Other two are OK. bye, jarl From jarl at casema.net Wed Sep 13 05:51:27 2000 From: jarl at casema.net (Jarl van Katwijk) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] BL PL design documentation References: <200009112308.TAA64854@archa13.cc.uga.edu> Message-ID: <39BF4E1F.45E974CA@casema.net> > > Part of the DL code base. Spawns or communicates with > > multiple Parents. > > I don't think this is correct. Jarl's document seems to make an explicit > point that there is only one BL (Parent) per Piper program running. So > the DL will only communicate with one BL. The BL itself will be > responsible for splitting processes up using the whole Mother to Baby > thing. At least, this is my understanding. Yes, this is true. bye, jarl From jarl at casema.net Wed Sep 13 05:55:14 2000 From: jarl at casema.net (Jarl van Katwijk) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] BL PL design documentation References: <200009112308.TAA64854@archa13.cc.uga.edu> <39BD725B.54316D07@hermes.usherb.ca> Message-ID: <39BF4F01.7FB1D374@casema.net> Jean-Marc Valin wrote: > > I don't think this is correct. Jarl's document seems to make an explicit > > point that there is only one BL (Parent) per Piper program running. So > > the DL will only communicate with one BL. The BL itself will be > > responsible for splitting processes up using the whole Mother to Baby > > thing. At least, this is my understanding. > > There will be one grandma-BL per Piper program running, however there will be > many mother-BL, one per machine, and many baby-BL per machine. Is that what you > meant? > Arf, JM, maybe I'm still not clear about this. The functionality of scheduling you want the GrnatMother BL to do, will actually be done by what you call the Mother BL. So there's no point having the GrantMother-BL neither the Mother-BL names, that's why I call both just plain BL. I mean, there's no point naming features, or functionality. Am I clear now, and, if you disagree, maybe we should have one more irc session about this ;) bye, jarl From jean-marc.valin at hermes.usherb.ca Tue Sep 12 12:05:12 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] Baby-BL implementation References: <200009112108.RAA32974@archa10.cc.uga.edu> <39BF347C.1BCF9BD3@casema.net> Message-ID: <39BE5438.E641B3C9@hermes.usherb.ca> > > 1. It seems like this requires that the MBL be smart enough to know that > > it needs to split a network at display nodes, so that it have access > > to the results that need to be displayed. So if we we had Jean-Marc's > > famous A->(B->C)->D network and say C were a viewer that needed to get > > results, then the MBL would have to know to split the network here and > > call getOutput on C (before calling on D), so that is could have the > > output for display. Is this right, or am I way off base? > > I think you're on track, but I dont really get this "a viever halfway a > network". I always > though every network has just one input on the start and one viewer at the > end of a > nodes network? viewers are a totally different thing (you haven't seen one yet). Viewers are nodes that take their input and send that to the GUI to display it... then they return the same object as their output. They're bypass nodes with the side effect of displaying the value in the GUI (mostly for debugging). So viewers have nothing to do with the "end nodes". > > 2. I'm not sure how the Overflow *Probe will work under this system. Am I > > missing something basic? > > > > The "*Probe" system? Please explain. Probes are a bit like viewers, debugging nodes. You insert them in the middle of your network and they allow you to trace the flow. They make next/continue/stop buttons appear on the GUI. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jean-marc.valin at hermes.usherb.ca Tue Sep 12 12:16:35 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] BL PL design documentation References: <200009112308.TAA64854@archa13.cc.uga.edu> <39BD725B.54316D07@hermes.usherb.ca> <39BF4F01.7FB1D374@casema.net> Message-ID: <39BE56E3.101545F@hermes.usherb.ca> Jarl van Katwijk a ?crit : > > Jean-Marc Valin wrote: > > > > I don't think this is correct. Jarl's document seems to make an explicit > > > point that there is only one BL (Parent) per Piper program running. So > > > the DL will only communicate with one BL. The BL itself will be > > > responsible for splitting processes up using the whole Mother to Baby > > > thing. At least, this is my understanding. > > > > There will be one grandma-BL per Piper program running, however there will be > > many mother-BL, one per machine, and many baby-BL per machine. Is that what you > > meant? > > > > Arf, JM, maybe I'm still not clear about this. The functionality of scheduling you > want the > GrnatMother BL to do, will actually be done by what you call the Mother BL. So > there's > no point having the GrantMother-BL neither the Mother-BL names, that's why I call > both just plain BL. I mean, there's no point naming features, or functionality. > > Am I clear now, and, if you disagree, maybe we should have one more irc session > about this ;) Well, I would like to think that "all the BL's are born equal". To me, the BL on the local machine shouldn't have more to do than any other BL. If you don't like the name grandma-BL, we could call it "DL2BL interface". It would be a library used by the DL. I'd like the scheduling to be done "in DL space". That's what I meant by grandma-BL: one (small) part of the BL code that would be unique for a piper program (while you have many parent-BL, one per machine). In my mind, we would not even need a (parent-)BL on the local machine if we don't want to do any processing there. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jean-marc.valin at hermes.usherb.ca Tue Sep 12 13:17:28 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] BL PL design documentation References: <200009112308.TAA64854@archa13.cc.uga.edu> <39BD725B.54316D07@hermes.usherb.ca> <39BF4F01.7FB1D374@casema.net> <39BE56E3.101545F@hermes.usherb.ca> Message-ID: <39BE6528.ECD8AB51@hermes.usherb.ca> > Well, I would like to think that "all the BL's are born equal". To me, the BL on > the local machine shouldn't have more to do than any other BL. If you don't like > the name grandma-BL, we could call it "DL2BL interface". It would be a library > used by the DL. I'd like the scheduling to be done "in DL space". That's what I > meant by grandma-BL: one (small) part of the BL code that would be unique for a > piper program (while you have many parent-BL, one per machine). In my mind, we > would not even need a (parent-)BL on the local machine if we don't want to do > any processing there. I would like to add that if we put the schetuling/splitting of the XML networks in the DL, we also save some complexity in the BL: it will not have to know what the XML means at all (hence, no need to link to libflowui and libxml). I like to think of the BL as a middle pary that handles all the communications (and authentication), yet without having to understand anything about the content of the said communications. This applies for the XML networks, the PL-DL communications, ... I think this might address Brad's concern that the BL would grow as to include everything including the kitchen sink. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Tue Sep 12 13:22:00 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] BL PL design documentation References: <200009112308.TAA64854@archa13.cc.uga.edu> <39BD725B.54316D07@hermes.usherb.ca> <39BF4F01.7FB1D374@casema.net> <39BE56E3.101545F@hermes.usherb.ca> <39BE6528.ECD8AB51@hermes.usherb.ca> Message-ID: <39BE6638.CC7AD707@bioinformatics.org> Jean-Marc Valin wrote: > > I would like to add that if we put the schetuling/splitting of the XML networks > in the DL, we also save some complexity in the BL: it will not have to know what > the XML means at all (hence, no need to link to libflowui and libxml). I like to > think of the BL as a middle pary that handles all the communications (and > authentication), yet without having to understand anything about the content of > the said communications. This applies for the XML networks, the PL-DL > communications, ... I think this might address Brad's concern that the BL would > grow as to include everything including the kitchen sink. I agree and recall that the original plan was to not have any XML processing in the BL. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Tue Sep 12 19:15:22 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] CNET.com - Will P2P companies thrive or die? Message-ID: <39BEB90A.F1C9209D@bioinformatics.org> Some might find this interesting: ``Throughout the spring and summer, entrepreneurs and media pundits boasted that the technology--which allows people to search for and retrieve files from individual computers around the world--would transform the Internet and become a lucrative investment niche. A June article in Fortune magazine dubbed file sharing `the hot idea of the year,' alerting investors to a trend that would `revolutionize infotech and reinvigorate the PC industry.''' http://news.cnet.com/news/0-1005-200-2758911.html Jeff From jeff at bioinformatics.org Wed Sep 13 18:08:13 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] piper article Message-ID: <39BFFACD.899CCE5E@bioinformatics.org> Cameron told me that the article on Piper will appear in SunWorld Online http://www.sunworld.com/ on Friday or Monday. I'll send the link when I see it. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From pvh at egenetics.com Thu Sep 14 04:55:47 2000 From: pvh at egenetics.com (Peter van Heusden) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] First release of the Snapper Development System Message-ID: There is an interesting article on the GNOME news site about this Snapper thing. http://news.gnome.org/gnome-news/968873979/ Here's the first paragraph: "The Snapper Development System enables a Linux GTK+/Gnome application to be "beamed" to a user desktop or web appliance running Linux. Snapper works by embedding the application in an XML file, which can reside in a relational database or on a file or web server, and since the application is GTK+/Gnome based the user will enjoy the "look and feel", and speed of a native Linux program. By using this foundation the possibilities are endless." Sounds vaguely like BlueBox, without the cross platform stuff, or ALBeanBox with more precision in terms of GUI. Peter _______________/\/eGenetics.com\/\_______________ Peter van Heusden pvh@egenetics.com Electric Genetics From nile at dloo.com Thu Sep 14 14:21:20 2000 From: nile at dloo.com (Nile Geisinger) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] First release of the Snapper Development System References: Message-ID: <39C1171F.F853BFAE@dloo.com> >From what I can see, Snapper and BlueBox are very similar in that they send over XML files to represent the program's interface. The main difference is that Snapper is client side while BlueBox is server side. 1) Snapper sends over the PERL and Python files to run the program 2) BlueBox connects widgets on the client and code on the server Snapper, in this way, serves the entire program to client. I assume the algorithm is something like. 1) Connect to Server. 2) Download Perl, Python, and XML files 3) Create interface from XML files with libglade 4) Run the program The best way to understand this is to think of the difference between a CGI and Javascript on an HTML page. The first runs on the server and serves an interface(like BlueBox), the second runs on the client and sends both the interface and the program (like Snapper). Both models complement each other nicely. I think this will have a very high return for GNOME people if people use it. The article says it will be out on Sept 15th, I'm not sure this will be that helpful for Piper since its point is to have several programs running on multiple servers (i.e., processing is server side, not client), but it's worth investigating. Nile Peter van Heusden wrote: > There is an interesting article on the GNOME news site about this Snapper > thing. http://news.gnome.org/gnome-news/968873979/ > > Here's the first paragraph: > > "The Snapper Development System enables a Linux GTK+/Gnome application to > be "beamed" to a user desktop or web appliance running Linux. Snapper > works by embedding the application in an XML file, which can reside in a > relational database or on a file or web server, and since the application > is GTK+/Gnome based the user will enjoy the "look and feel", and speed of > a native Linux program. By using this foundation the possibilities are > endless." > > Sounds vaguely like BlueBox, without the cross platform stuff, or > ALBeanBox with more precision in terms of GUI. > > Peter > _______________/\/eGenetics.com\/\_______________ > Peter van Heusden pvh@egenetics.com > Electric Genetics > > _______________________________________________ > pipet-devel maillist - pipet-devel@bioinformatics.org > http://bioinformatics.org/mailman/listinfo/pipet-devel From jeff at bioinformatics.org Tue Sep 19 21:32:22 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] Piper article at SunWorld Message-ID: <39C813A6.2B867035@bioinformatics.org> Here is the article that Cameron Laird wrote about Piper: http://www.sunworld.com/sunworldonline/swol-09-2000/f_swol-0915-regex.html For some reason, it is part of an article on Qt scripting. We're in the limelight now. The website is getting lots of hits. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From daniel.bergdahl at epk.ericsson.se Wed Sep 20 02:01:43 2000 From: daniel.bergdahl at epk.ericsson.se (Daniel Bergdahl) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] (no subject) Message-ID: <39C852C7.FFACBE50@epk.ericsson.se> -- ---------------------------------------------------------------- Daniel Bergdahl Software Designer E-mail: daniel.bergdahl@epk.ericsson.se Phone : +46 (0) 457 775 68 (direct) : +46 (0) 457 775 00 (office) Mobile: +46 (0) 703 305 7568 Fax : +46 (0) 270 35 ---------------------------------------------------------------- From jeff at bioinformatics.org Wed Sep 20 02:27:08 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] (no subject) References: <39C852C7.FFACBE50@epk.ericsson.se> Message-ID: <39C858BC.4EA8358D@bioinformatics.org> Very profound ;-) Daniel Bergdahl wrote: > > -- > ---------------------------------------------------------------- > Daniel Bergdahl > Software Designer > > E-mail: daniel.bergdahl@epk.ericsson.se > > Phone : +46 (0) 457 775 68 (direct) > : +46 (0) 457 775 00 (office) > Mobile: +46 (0) 703 305 7568 > Fax : +46 (0) 270 35 > ---------------------------------------------------------------- -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Wed Sep 20 20:25:37 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] Re: Qt on Windows (was Open your eyes: TCL, TIX and wxWidnwos) References: <4792DCCC5E5FB1B4.C843FC4D4F22AFE8.0B25CCE0ADE26752@lp.airnews.net> <8qaknm$7kp$1@m1.cs.man.ac.uk> Message-ID: <39C95581.D72F280E@bioinformatics.org> "Donal K. Fellows" wrote: > > Hmm. What you write about Piper seems interesting. Are there any > similarities with the (utterly amazing IMHO) AVS? As a way to connect > up complex software modules to form a visualisation application, it > was pretty amazing to me, and had a number of influences on some of my > subsequent work... Yes, there are similarities between Piper, AVS and OpenDX: http://www.opendx.org/ However, Piper is more flexible by design. For example, Piper can be controlled by different user interfaces (UI's), even simultaneously. UI's control Piper through CORBA and the MVC (Model, View, Controller) paradigm. The Model (the network of nodes) resides with Piper, while the View and the Controller are part of a separate UI. In operation, the user sees the View, attempts to change the View by means of a Controller. (Note that this part takes place in the UI.) The UI then sends a message to Piper, and this causes the Model to be changed. The change in the Model is then applied back to the View, and we start all over again. But, because the UI's are separate from Piper, there can be more than one UI. They wait for the Piper to command a change in the View, so commands "broadcasted" by the Piper can cause a change in the View of multiple UI's. This effectively means ONE UI CAN CONTROL ALL OF THE OTHERS. That's just one unique feature about Piper. On the UI side again, the arrangement of nodes, where each node has its own UI component (e.g., a button) can be used to determine the layout of a COMBINED UI. Generally, sibling relationships mean the components will be adjacent to one another. And, parent-child relationships mean the components will be nested. Most RAD GUI builders show you a hierarchy of widgets drawn as a tree. Since nodes in Piper are already in a tree, we take the same approach. You can see that there some great ideas in Piper that don't appear in AVS and OpenDX, and there are more beyond the UI (the UI happens to be my end of the project). AVS and OpenDX are great for data visualization, as that is what they are designed for. Piper is like them in many ways, but the GENERAL PURPOSE design of Piper means it has to be more flexible. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Wed Sep 20 20:49:01 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] Re: Qt on Windows (was Open your eyes: TCL, TIX and wxWidnwos) References: <4792DCCC5E5FB1B4.C843FC4D4F22AFE8.0B25CCE0ADE26752@lp.airnews.net> <8qaknm$7kp$1@m1.cs.man.ac.uk> <39C95581.D72F280E@bioinformatics.org> Message-ID: <39C95AFD.EEF83F3F@bioinformatics.org> "J.W. Bizzaro" wrote: > > On the UI side again, the > arrangement of nodes, where each node has its own UI component (e.g., a > button) can be used to determine the layout of a COMBINED UI. Generally, > sibling relationships mean the components will be adjacent to one another. > And, parent-child relationships mean the components will be nested. Most RAD > GUI builders show you a hierarchy of widgets drawn as a tree. Since nodes in > Piper are already in a tree, we take the same approach. Hmmm. It does seem that AVS has this feature. I'm not sure if you can create a peer-to-peer network with AVS though. There are many similarities, but if you took apart both systems, you'd find many differences. Another is price. Piper is GPL while AVS is very expensive and proprietary. I think Piper will be simpler to use too :-) Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Wed Sep 20 20:52:00 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] AVS and OpenDX Message-ID: <39C95BB0.6DE6E783@bioinformatics.org> Could someone in the know give us a quick comparison between Piper, AVS and OpenDX? Harry? I haven't used either AVS or OpenDX, so I'm talking out of my caboose. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Thu Sep 21 00:19:26 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] gnotices announcement Message-ID: <39C98C4E.E24D926E@bioinformatics.org> I submitted an announcement about Piper to the Gnome Gnotices (news) page: http://news.gnome.org/gnome-news/index_html? I wrote about the SunWorld article and said that we are looking for developers to help out. The announcement should appear on the page in the morning. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Fri Sep 22 23:06:59 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] Piper article in Tecnologia magazine Message-ID: <39CC1E53.34920CD3@bioinformatics.org> There is a reference to Piper in the Portuguese magazine Tecnologia. Here is a Babelfish translation of the paragraph where Piper is mentioned: ``Infinite possibilities? On the other hand distributed computation has other sources without lucrative ends, to the example of Seti@home. Science will be able to benefit enormously in the form of interactive computational work where technician informaticists and scientists can work together using networks of program `clients' appropriate to speed up studies and scientific discoveries that demand a highly cooperative effort. It is the case of the Piper network. And the possibilities of this `method' in the branches of science and technology are almost infinite...'' http://www.digito.pt/tecnologia/artigos/tecart73.html Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jarl at casema.net Sat Sep 23 16:15:32 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] DOA'00 Message-ID: <39CD0F64.FF14B4A8@casema.net> Hi all, I just returned from Antwerp, Belgium, where I attended the DOA'00 converance. I think you people would like to know the Piper project design is among the most high tech around. In both scientific and enterprise worlds there're much big problems and great solutions out of which Piper has a very high fitness. In other words: We are on the right track :) Let me address some of the mayor issues that were discussed and are of interest to the Piper project: = Integration - The need of COST intergration (Commercial of the shelf technology). Ofcourse this term can be replaced by OOST (Open source of the shelf tech.). Is means that it's impossible to create everything from scratch every time, so the availeble software should somehow be easily integrated into a system. - Middleware standards. Personally I dont really care for this and dont see it of much of a problem. Just let the middleware (cq Piper) be transparant to the applications and the problem dont exsists anymore. Maybe I'm being lazy on this one.. = Representation - Data representation by models. This is done by models like UML and MOF and seems to be a solved problem. Only I dont want this to be touched by Piper, I think it's out of the scope. - Functional representation by patterns. This one is very interesting because it seems noone has done a very good job yet. I'm thinking about the XML node description of Piper ofcourse. - Service Classification. Piper has two places where this is important: on the BL->BL connection and the PL->Application level. It means that both parties need to determine what protocols\scematics\etc must be used in order to operate together. I think this is a much bigger problems as we though before. But somebody has held a very good presentation about this, so we're not alone :) = Environment - Resource management. Our proposal of the BL->PL design has a alinea dedicated to scheduling, or XML splitting. This spoke about three variants: User defined, Central controll for a user addres space and based upon local reality. This design seems to be very good. - Temporary environments. Applications executed by the PL will be part of a temporary system; the environment wont excist anymore once the network has been executed. This again is a much bigger problem as we (or just me?) exspected. At first though it looks the DL will need to do some serieus node versioning together with node history- and configuration tracking. - Location problems (naming, lookup). Solutions for a world wide naming & localisation system are numberous, we just need to pick one or combine to get our own - Environment isolation. This problem only arises inbetween the lines, the 'normal' solution to having seperated node spaces is fixed by physically seperating the hardware. The Piper solution of User Address Spaces is way cleaner. - Replication and relocation. Quite complex stuff, but somehow very attractive to reseach, so there're a lot of solutions. There also was a presentation (partially) about relocalisation by sourcecode & M4 macros. - Realtime Corba specifications. OH YES, they will be in Corba 3.0! (I met the persone who wrote the Corba 2.0 specs) bye, jarl From jeff at bioinformatics.org Sun Sep 24 21:41:32 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] license issue revisited Message-ID: <39CEAD4C.D86A01F5@bioinformatics.org> The last we spoke about the license, Jean-Marc brought up a serious problem. Basically, Overflow (the processing layer) could not legally link with GPL and non-GPL libraries, BOTH AT THE SAME TIME. This stopped the discussion dead in its tracks, as there seemed to be no way around it. It also seemed to smack of the licensing problems that KDE had: We could make an exception for non-GPL libraries, but the owners of the GPL libraries would each have to make the same exception, and that doesn't seem likely. But, Jean-Marc, isn't this library linking done by the user at compile time? Providing we don't distribute Piper with links to both non-GPL and GPL libraries, it would then be left up to the user. And, an important thing to consider is that a user can't really violate the GPL unless he/she re-distributes. As far as I know, a user can link GPL with a license from Mars, and it won't matter, if it is for personal use. Comments? Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Sun Sep 24 22:52:34 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] license issue revisited References: <39CEAD4C.D86A01F5@bioinformatics.org> Message-ID: <39CEBDF2.6A36DD1A@hermes.usherb.ca> > The last we spoke about the license, Jean-Marc brought up a serious problem. > Basically, Overflow (the processing layer) could not legally link with GPL and > non-GPL libraries, BOTH AT THE SAME TIME. This stopped the discussion dead in > its tracks, as there seemed to be no way around it. It also seemed to smack > of the licensing problems that KDE had: We could make an exception for non-GPL > libraries, but the owners of the GPL libraries would each have to make the > same exception, and that doesn't seem likely. > > But, Jean-Marc, isn't this library linking done by the user at compile time? > Providing we don't distribute Piper with links to both non-GPL and GPL > libraries, it would then be left up to the user. And, an important thing to > consider is that a user can't really violate the GPL unless he/she > re-distributes. As far as I know, a user can link GPL with a license from > Mars, and it won't matter, if it is for personal use. I was actually discussing that with Richard Stallman yesterday... Here's what he had to say: "In a technical sense, the user does perform the act of combining them. But in truth they are inherently connected regardless of what any specific user does. They are designed to work as one program, and this is true before any specific user obtains his copy and runs it. So if you want to permit non-free plugins for Piper, you should say so explicitly in the license. For instance, you could use the GPL and state an exception permitting non-free plug-ins, if you want to." I agree with him: we should play safe by explicitly specifying the exception, but the fact that the user does the linking, makes (I think) linking a non-free node with a GPL node legal. Since the linking is only done on demand at run-time, there's no redistribution problem. Here's about the CORBA layers: Jean-Marc: It is still unclear (undecided) how closely the layers would be linked. However, I think a safe way to deal with the issue is to explicitly allow the CORBA linking for layers which we want to make that possible (even if it's already allowed by the GPL) and keep the GPL for layers where we don't want that, unless the GPL permits it (we don't want to add restrictions to the GPL, since it could make the license GPL-incompatible). Richard Stallman: Yes, that makes sense. You could say, for instance, that CORBA communication through the interfaces you have designed, is in your view interaction between two separate programs and does not make the communicating modules into a single program. Do we all agree? Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Sun Sep 24 23:15:41 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] license issue revisited References: <39CEAD4C.D86A01F5@bioinformatics.org> <39CEBDF2.6A36DD1A@hermes.usherb.ca> Message-ID: <39CEC35D.A33489F6@bioinformatics.org> Jean-Marc Valin wrote: > > I was actually discussing that with Richard Stallman yesterday... Here's what he > had to say: > > "In a technical sense, the user does perform the act of combining them. > But in truth they are inherently connected regardless of what any > specific user does. They are designed to work as one program, and > this is true before any specific user obtains his copy and runs it. There is a problem, as he says, if Piper requires these libraries to run. I don't believe that any of the libraries that Jean-Marc is concerned about will be required. But, they should not be shipped/distributed with Piper either. Then, it is the user's concern. > I agree with him: we should play safe by explicitly specifying the exception, > but the fact that the user does the linking, makes (I think) linking a non-free > node with a GPL node legal. Since the linking is only done on demand at > run-time, there's no redistribution problem. Yes, I agree. We should include that exception. It doesn't mean, however, that GPL and non-GPL libraries can legally be linked and then distributed together. I think that we should also provide such a warning to users in the license. > Richard Stallman: > Yes, that makes sense. You could say, for instance, that CORBA > communication through the interfaces you have designed, is in your > view interaction between two separate programs and does not make the > communicating modules into a single program. Yes, it makes sense to provide that exception, just for clarity's sake, although it doesn't seem the GPL will make CORBA linking illegal anytime soon. > Do we all agree? I agree. Now, the question is GPL or LGPL ;-) I vote for GPL with the afore mentioned exceptions. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Mon Sep 25 01:55:53 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] license issue revisited References: <39CEAD4C.D86A01F5@bioinformatics.org> <39CEBDF2.6A36DD1A@hermes.usherb.ca> <39CEC35D.A33489F6@bioinformatics.org> Message-ID: <39CEE8E9.9DCDAB31@hermes.usherb.ca> > There is a problem, as he says, if Piper requires these libraries to run. I > don't believe that any of the libraries that Jean-Marc is concerned about will > be required. But, they should not be shipped/distributed with Piper either. > Then, it is the user's concern. Piper will not require any of these plugins... and I don't think we should have restrictions about what we can ship together. After all, RedHat used to ship StarOffice (non-Free) with its Linux distribution. > > > I agree with him: we should play safe by explicitly specifying the exception, > > but the fact that the user does the linking, makes (I think) linking a non-free > > node with a GPL node legal. Since the linking is only done on demand at > > run-time, there's no redistribution problem. > > Yes, I agree. We should include that exception. It doesn't mean, however, > that GPL and non-GPL libraries can legally be linked and then distributed > together. I think that we should also provide such a warning to users in the > license. We're OK, since it's always the user that performs the linking... and the linking is "undone" when Piper exits. So you can never distribute a "derived" (linked) work. You can then use any plugin you like... > I agree. Now, the question is GPL or LGPL ;-) I vote for GPL with the afore > mentioned exceptions. I vote for the LGPL for the PL though, because a non-free plugin will need to use classes from the PL (this is done at compile time). Otherwise, the GPL i OK with me. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Mon Sep 25 20:07:07 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:32 2006 Subject: [Pipet Devel] [Fwd: Piper] Message-ID: <39CFE8AB.27D9B6B3@bioinformatics.org> -------------- next part -------------- An embedded message was scrubbed... From: Slavisa Jovanovic Subject: Piper Date: Mon, 25 Sep 2000 17:10:32 -0500 (CDT) Size: 1600 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000926/9df55d9e/attachment.mht From josh at narf.com Tue Sep 26 14:47:16 2000 From: josh at narf.com (Josh Prismon) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] Quest for Understanding. Message-ID: <021401c027ea$318dc8e0$29156cc0@AVIENDHA> I just read the gnome announcement. I am still confused by a couple of things, so let me make some hypotheticals. I agree w/ the fundamental tenet that the OS world must respond to .NET, or find itself obsoleted by it. The current plan is very back end centric (because of the problem it is designed to solve). I just spent a while looking at the documentation for the project. Here are my comments: 1) The idea of distributed componets and routing are all very well and good (and needed) but it still does not address one of the biggest problems with the OS world. Inconsistant Metaphor. The user interface is not very well defined at all (Bluebox?) 2) If one is going to write a user interface that runs off of this, why re-invent the wheel w/ bluebox? Mozilla's XUL is designed for problems like this, and Mozilla can easily be extended because of the quality of it's object model and plug in structures. Plus it would also give us something that I really did not see harped on. COMPATABILITY WITH WINDOWS. Ignore 90% of the Computer market at your own peril. 3) Backend is all well and good. Can we hook up a Object Database (Zope's Zobj for example?). I would really like a way to connect to a server and ask: Give me all transactions that are over $500. Plus we could add objects with publicly defined methods and properties to the database ala zope. This would be amazingly powerfull, and would enable something like what I want to do below. Wouldn't it be neat if we could do the following: 1) A user needs a new PC on his desk. He opens up some sort of a browser. Clicks on "World" Then Comapny, then HR, then requests. There is a little application there to purchase a new machine. That purchese order connects to a server on his machine or a bigger server which has a proccess that says "put a request for a new machine in his bosses inbox". The boss opens up the same browser, and hits his inbox. There is a little application there called "Request for a new PC by XXXXX". He clicks Aprove on that. His machine has a proccess called "approve" which then takes that information and sends it to a Purchasing server on the other side of the world. The Purchising server automagically purchases the server, debits the amount out of the departments budget, and sends confirmation letters.... If we can do that... we can rule the world. If this is possible, count me in on this project. I would love to focus on client interface and windows compatibility. From jeff at bioinformatics.org Tue Sep 26 21:52:14 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] Re: Quest for Understanding. References: <021401c027ea$318dc8e0$29156cc0@AVIENDHA> Message-ID: <39D152CE.DD43CC3@bioinformatics.org> Hi Josh. Thanks for the quest. I'm probably the most appropriate person to answer your questions, since I am the sub-project leader for Piper's user interfaces. But, I'd like Nile Geisinger (BlueBox developer) to comment as well. Josh Prismon wrote: > > 1) The idea of distributed componets and routing are all very well and good > (and needed) but it still does not address one of the biggest problems with > the OS world. Inconsistant Metaphor. The user interface is not very well > defined at all (Bluebox?) This question may be best for Nile, but I'll add my $0.02. I believe there are certain user interface components and events that are common to the major UI toolkits on various platforms. If a UI for Piper is to be cross-platform, it is that minimal set of components and events that will be defined and used. I believe this applies to BlueBox. But, I'm not sure if that's the concern you're trying to express. When you say that the user interface is not well defined, do you object to there being custom widgets that work on only one platform? Are you arguing for a standard? > 2) If one is going to write a user interface that runs off of this, why > re-invent the wheel w/ bluebox? Mozilla's XUL is designed for problems like > this, and Mozilla can easily be extended because of the quality of it's > object model and plug in structures. Plus it would also give us something > that I really did not see harped on. COMPATABILITY WITH WINDOWS. Ignore 90% > of the Computer market at your own peril. Right, I agree with you in general. The problem, however, is not that we are trying to ignore Windows but that we don't have anyone to develop for it :-) My personal project is the Pied/Piper UI, which is GNOME/GTK-based and allows the run-time configuration of networks by the user. It could be written to work on Windows as well. BlueBox actually does plan on supporting Windows, but without the network configuration abilities of Pied. However, I don't believe they have anyone working on that at the moment. Then there's Mozilla. It wasn't my first choice for a UI, or Nile's, because of its size. But, we were hoping for WWW integration with Piper, and a Mozilla client may be the way to do it. > 3) Backend is all well and good. Can we hook up a Object Database (Zope's > Zobj for example?). I would really like a way to connect to a server and > ask: [...] Oh yeah, database connectivity is ABSOLUTELY a feature of Piper, and it's one of the first things Brad Chapman has worked on. Check these links out for ideas on how the Pied/Piper UI might be used for database querying/mining: Clementine: http://www.spss.com/clementine/newshow/sld001.htm Cerebellum: http://www.cerebellumsoft.com/product/cereproduct.html > If this is possible, count me in on this project. I would love to focus on > client interface and windows compatibility. It sure is possible :-) It seems you have a pretty good understanding of what we want to accomplish, and you're welcome to working on anything you want: Windows, Mozilla, databases, etc. There's LOTS to do, and that's an understatement :-) But, we're committed. I've been with the Loci end of the project since 1998 and have written 1,362+ e-mails (I just did a search, and yes that is amazing) about the system. Let me know if and how you want to proceed, and I can set you up. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Tue Sep 26 22:41:52 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] piper-20000926 snapshot Message-ID: <39D15E70.58C80014@bioinformatics.org> I just made a snapshot of Piper in CVS: ftp://bioinformatics.org/pub/piper/piper-20000926.tar.gz It may not be the greatest, but it's the latest. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Wed Sep 27 17:38:55 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] Slashdot: Pipes In GUI's Message-ID: <39D268EF.C7BDECBA@bioinformatics.org> I don't know how I missed this. It's over a month old: ``Caine asks: "While having some dead time at work, I spent some time thinking about how to write a good GUI. I started thinking about if you could implement pipes in a GUI. My thought was something like this: You could connect two programs and/or widgets by a GUI pipe. This could be visualized if the user wanted to, as a thin, pulsating red line or something. For example, you could pipe your browsers' status/error window to a log colorer. You could also pipe other things than text, such as pictures; if you're watching a streaming movie you could pipe it into the graphic programs' filters for example. Basically they would do the same as a normal "|" but with all kinds of data and in a GUI. Has this ever been done or written about before? Any implementations?" Interesting idea, but before this would be possible, we'd have to introduce the concept of discrete input-channels and output-channels into our GUIs, and that's not as easy as it sounds.'' http://slashdot.org/askslashdot/00/08/11/2052243.shtml There are some good comments to read, which will be your homework assignment :-) I did just contact Caine, and someone else posted a comment with a link to Piper's website, which is how I found the article. Jeff From jeff at bioinformatics.org Thu Sep 28 19:02:05 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome Message-ID: <39D3CDED.1F4D726D@bioinformatics.org> We've added 17 people to the list since the Gnotices announcement of Piper! Welcome to the list, if this includes you. I hope that not everyone is a lurker ;-) If anyone wants to help with something but maybe doesn't know what to do, then don't be shy, speak up, tell us your interests, and we can find something appropriate. Maybe out of 17 people we've got one or two who can help out. That's why we made the announcement :-) Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Thu Sep 28 19:23:52 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] mention of Piper on Linux Weekly News Message-ID: <39D3D308.A5400BAB@bioinformatics.org> ``Not paying the piper. Piper is "a system for managing multi-protocol connections between Internet-distributed objects." It's based on a number of GNOME components (Loci, GMS, and Overflow), and is seen as an open source answer to Microsoft's ".NET". The project is in its early stages, but has gotten far enough to have a screenshot up. ``They are, of course, looking for people who want to help. For more information, see the Piper web page and this GNOME News writeup.'' http://lwn.net/2000/0928/devel.php3 Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From mogul-piper at gelatinous.com Thu Sep 28 20:01:21 2000 From: mogul-piper at gelatinous.com (Bret Mogilefsky) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] mention of Piper on Linux Weekly News In-Reply-To: <39D3D308.A5400BAB@bioinformatics.org>; from jeff@bioinformatics.org on Thu, Sep 28, 2000 at 11:23:52PM +0000 References: <39D3D308.A5400BAB@bioinformatics.org> Message-ID: <20000928170121.A99553@gelatinous.com> On Thu, Sep 28, 2000 at 11:23:52PM +0000, J.W. Bizzaro wrote: > ``Not paying the piper. Piper is "a system for managing multi-protocol > connections between Internet-distributed objects." It's based on a number of > GNOME components (Loci, GMS, and Overflow), and is seen as an open source > answer to Microsoft's ".NET". The project is in its early stages, but has > gotten far enough to have a screenshot up. > > ``They are, of course, looking for people who want to help. For more > information, see the Piper web page and this GNOME News writeup.'' > > http://lwn.net/2000/0928/devel.php3 How do you think we all got here? =) I'd planned to lurk, but seeing that there are so few developers currently perhaps I'll say more. This project caught my interest mainly because I keep waiting to see what the next evolution of the ultra-powerful but ultra-unfriendly prompt-pipe combo. This project and the Enlightenment File Manager are the first positive steps away from a pure prompt that I've yet seen. I don't have a lot of Gnome/Bonobo development experience, but I'm strong in many other areas (game development background) which may or may not be useful as the project develops. Hope it all goes well. =) Bret -- "Why, that's the second biggest monkey head I've ever seen!" --Guybrush "LeChuck's dead. I blew him into a million gooey pieces." --Guybrush Bret Mogilefsky ** mogul-piper@gelatinous.com ** Programmer, SCEA R&D From jeff at bioinformatics.org Thu Sep 28 20:21:00 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] mention of Piper on Linux Weekly News References: <39D3D308.A5400BAB@bioinformatics.org> <20000928170121.A99553@gelatinous.com> Message-ID: <39D3E06C.C93BAFBB@bioinformatics.org> Bret Mogilefsky wrote: > > I'd planned to lurk, but seeing that there are so few developers > currently perhaps I'll say more. This project caught my interest > mainly because I keep waiting to see what the next evolution of > the ultra-powerful but ultra-unfriendly prompt-pipe combo. This > project and the Enlightenment File Manager are the first positive > steps away from a pure prompt that I've yet seen. I don't have a > lot of Gnome/Bonobo development experience, but I'm strong in many > other areas (game development background) which may or may not be > useful as the project develops. Hi Bret! We're using Python and C++. The way it breaks apart is the "Build-Time Subsystem" (the part that lets the user interactively connect/disconnect components) is in Python. This includes the Pied UI, which is what all the screenshots show. The "Run-Time Subsystem" (the part that executes components and controls data flow) is in C++. If you want to help, we can use it in any area. You're probably a C/C++ developer, since you program games. So, maybe the Run-Time Subsystem would be a good match for your skills. But, if you like to work with graphics, Pied or BlueBox (uses several languages) may be best. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From ecn at fault-tolerant.org Thu Sep 28 22:06:30 2000 From: ecn at fault-tolerant.org (ecn@fault-tolerant.org) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome In-Reply-To: <39D3CDED.1F4D726D@bioinformatics.org> References: <39D3CDED.1F4D726D@bioinformatics.org> Message-ID: <14803.63782.86151.562737@dsl-64-34-115-49.telocity.com> > I hope that not everyone is a lurker ;-) Ok, I've been lurking for a while. Might as well throw out some of my wacky ideas. About 10 years ago I was involved in building some basic business systems using object-oriented programming techniques. I was disappointed by various architectures I built at the time. Yet, many of these were approaches were encouraged by experts at the time. Specifically, Model-View-Controller was pushed as some sort magical fairy dust which would simplify user interface design. I've never found it helpful for separating concerns in ways that anticipated future changes. Another bit of magic bandied about was Business Objects, which would somehow let you build Business Systems by integrating pre-existing components. I never found two people in the same company that could agree on a hard definition of "Person" or "PurchaseOrder". In fact, maintenance changes to systems were the result of changing definitions and roles of existing business entities. After I stopped building these systems, I sat down and did some heavy thinking. I tried to think of the day-to-day changes that I actually saw, vs the kinds of changes my software could absorb. For example, one of the most common changes was to add a field to the database and the screens for new business rules. But with MVC or Business Objects these sorts of changes were not trivial. After much thinking (and with some help from some friends) I began to envision business systems built around data flow. Data would flow from databases and into forms and back to databases, or printed reports, or as email, or instant messages, or whatever. Flexible representations of data (XML) and configurable user interfaces (toolkits, scripted GUIs, browsers) would respond better to the most common sorts of changes. Furthermore, business rules were easier to see and manipulate as dataflow, rather than "permissions" or "state-change diagrams". > If anyone wants to help with something but maybe doesn't know what > to do, then don't be shy, speak up, tell us your interests, and we > can find something appropriate. So I'm interested in business-systems-as-dataflow. Anyone else? -Eric From jeff at bioinformatics.org Thu Sep 28 22:51:51 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <14803.63782.86151.562737@dsl-64-34-115-49.telocity.com> Message-ID: <39D403C7.C979628F@bioinformatics.org> ecn@fault-tolerant.org wrote: > > So I'm interested in business-systems-as-dataflow. Anyone else? Well, it appears as though Josh Prismon is. A couple days ago he wrote that message titled "Quest for Understanding": Josh Prismon wrote: > > The boss opens up the same browser, and hits his inbox. There is a little > application there called "Request for a new PC by XXXXX". He clicks Aprove > on that. His machine has a proccess called "approve" which then takes that > information and sends it to a Purchasing server on the other side of the > world. > > The Purchising server automagically purchases the server, debits the amount > out of the departments budget, and sends confirmation letters.... It seems we have a couple people interested in the same thing here. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Thu Sep 28 23:46:51 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <14803.63782.86151.562737@dsl-64-34-115-49.telocity.com> <39D403C7.C979628F@bioinformatics.org> Message-ID: <39D410AB.BE5DD061@hermes.usherb.ca> > > So I'm interested in business-systems-as-dataflow. Anyone else? > > Well, it appears as though Josh Prismon is. A couple days ago he wrote that > message titled "Quest for Understanding": > > It seems we have a couple people interested in the same thing here. I am not, but that's OK! I just thought it's a good time to let everyone know of another, more scientific application for Piper. I'm working on developing Piper building blocks ranging from speech/signal processing to neural networks and real-time audio effect processing. This is the part of Piper that comes from the Overflow (http://freespeech.sourceforge.net/overflow.html) project. Although Piper integration is not fully completed, all these tools are already quite usable (I've been exclusivly using them for my master project for ~1 year). So to all the ones interested in more scientific applications... you are not alone! ...not to forget Brad and Jeff (and others I forget) working on bioinformatics applications. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Fri Sep 29 10:49:53 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] about focus Message-ID: <39D4AC11.A65A9BE4@bioinformatics.org> I just wanted to emphasize something at this point. We have a number of people coming together with different interests, and I think that is great. But, because Piper is not focused on a particular application, that doesn't mean Piper is unsuitable for it. I don't want people to think that, because it is not true. Take GNOME as an example. GNOME is a general-purpose system. But, because it is being used to write games by some people, that doesn't mean GNOME can't be used to write other things, such as productivity applications. Likewise, because Piper might be use to build non-scientific systems, that doesn't mean Piper can't be used to build scientific systems. It has been my conviction that a general-purpose system will attract developers and users with all sorts of interests, and that that will lead to the development of things that can be used by developers with special interests. For example, signal processing tools may be useful to bioinformaticists, and visa versa. And, bioinformatics tools may be useful to business people, and visa versa. That's not even to mention that more developers and more users will mean a more developed infrastructure and more eyes to catch bugs. The more the merrier :-) Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Fri Sep 29 10:54:03 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] Hmmm Message-ID: <39D4AD0B.8CB58F83@bioinformatics.org> Hey, the GNOME people changed the title of my post from "GNU/GNOME Alternative to Microsoft's .NET" to "Redistribution of software". Kind of makes you wonder. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From claws at hem.passagen.se Fri Sep 29 14:54:15 2000 From: claws at hem.passagen.se (Marcus Rosell) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> Message-ID: <39D4E556.A2DF8048@hem2.passagen.se> "J.W. Bizzaro" wrote: > We've added 17 people to the list since the Gnotices announcement of Piper! > Welcome to the list, if this includes you. > > I hope that not everyone is a lurker ;-) If anyone wants to help with > something but maybe doesn't know what to do, then don't be shy, speak up, tell > us your interests, and we can find something appropriate. Maybe out of 17 > people we've got one or two who can help out. That's why we made the > announcement :-) Well I don?t intend to be just a lurker. I want to help :-) I?ve just been running Linux for 2 months, but I have programmed allot for other platforms (mostly amiga) and microprocessors. Now days I mostly program in C/C++, but there were day that i thought that Assembler was the only right way to hack. This is my first open source project so I don?t know how much time it is involved , but I intend to help the much I can. /Marcus Rosell From jeff at bioinformatics.org Fri Sep 29 17:38:36 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <39D4E556.A2DF8048@hem2.passagen.se> Message-ID: <39D50BDC.2D89DD57@bioinformatics.org> Marcus Rosell wrote: > > Well I don?t intend to be just a lurker. I want to help :-) > I?ve just been running Linux for 2 months, but I have programmed allot for other > platforms (mostly amiga) and microprocessors. Now days I mostly program in C/C++, > but there were day that i thought that Assembler was the only right way to hack. Great. Well, for people interested in C/C++ programming and the infrastructure of Piper, I'd recommend helping with either the Brokering Layer (lead by Jarl Van Katwijk) or the Processing Layer (lead by Jean-Marc Valin). For people interested in C/C++ and GUI programming, I believe there are some things to do in the BlueBox UI (lead by Nile Geisinger of dLoo). In any case, anyone interested in helping should first get Piper to run on their home systems. There's a recent snapshot of the CVS module in anon FTP. Follow the instructions for "building from CVS". And, PLEASE contact the list if you are having trouble. I hear that omniORB has trouble with FreeBSD. The second step would be to join The Open Lab, which is the project management system we're using: http://theopenlab.org >From there, you will get a shell account, CVS access, etc. But, ask me to "add" you to the project when ready. The third step would be to look at the code for the layers you're considering. If you find something that suits your interests, contact the respective leader via this list, and they'll tell you what needs working on. > This is my first open source project so I don?t know how much time it is involved > , but I intend to help the much I can. Oh, there's no time commitment at all. Do as much or as little as you want. We're just glad to have the help. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jeff at bioinformatics.org Fri Sep 29 17:53:27 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] C -> C++ conversion of the BL Message-ID: <39D50F57.BDF16E21@bioinformatics.org> I know I'm going to hear some groans about this, but Jarl and Brad have been working to convert the code base of GMS (now the Brokering Layer) from C to C++. Why the Hell do we want to do that? We have 3 related reasons: o The PL (Overflow) and the BL need to be built together, and since the PL is in C++ (and probably can't be in C), we thought the compilations would go more smovely for the user if both layers were in C++. (Recall that the BL and PL are GMS and Overflow, two different projects that are now being merged.) o One language for the Run-Time Subsystem would mean less dependencies. For example, STL vs. STL+GLIB. o Two languages (Python and C++) for Piper would be less complicated for development than 3 languages. Anyway, I guess Jarl is half-way or so through with the conversion, so there's no use in debating the qualities of C++. What I am hoping is that someone here with C and C++ experience could help expedite the conversion, which would speed the merger of GMS and Overflow as well as the release of a networkable Piper. Any volunteers? Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From Timothy.Keitt at SUNYSB.Edu Fri Sep 29 17:50:08 2000 From: Timothy.Keitt at SUNYSB.Edu (Timothy H. Keitt) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <39D4E556.A2DF8048@hem2.passagen.se> <39D50BDC.2D89DD57@bioinformatics.org> Message-ID: <39D50E90.B0FAF33C@SUNYSB.Edu> I am a lurker. :-) Really what I want to do is evaluate piper to see if it fits my needs (environmental systems modeling, spatial ecology). It seems that the first order of business is packaging. If users can download and try the software using rpm/dpkg, then they will get interested contributing. (My preference would be debs as I'm running debian. Most of the support libraries are already available in woody.) Cheers, Tim "J.W. Bizzaro" wrote: > Marcus Rosell wrote: > > > > Well I don?t intend to be just a lurker. I want to help :-) > > I?ve just been running Linux for 2 months, but I have programmed allot for other > > platforms (mostly amiga) and microprocessors. Now days I mostly program in C/C++, > > but there were day that i thought that Assembler was the only right way to hack. > > Great. Well, for people interested in C/C++ programming and the > infrastructure of Piper, I'd recommend helping with either the Brokering Layer > (lead by Jarl Van Katwijk) or the Processing Layer (lead by Jean-Marc Valin). > > For people interested in C/C++ and GUI programming, I believe there are some > things to do in the BlueBox UI (lead by Nile Geisinger of dLoo). > > In any case, anyone interested in helping should first get Piper to run on > their home systems. There's a recent snapshot of the CVS module in anon FTP. > Follow the instructions for "building from CVS". And, PLEASE contact the list > if you are having trouble. I hear that omniORB has trouble with FreeBSD. > > The second step would be to join The Open Lab, which is the project management > system we're using: > > http://theopenlab.org > > >From there, you will get a shell account, CVS access, etc. But, ask me to > "add" you to the project when ready. > > The third step would be to look at the code for the layers you're > considering. If you find something that suits your interests, contact the > respective leader via this list, and they'll tell you what needs working on. > > > This is my first open source project so I don?t know how much time it is involved > > , but I intend to help the much I can. > > Oh, there's no time commitment at all. Do as much or as little as you want. > We're just glad to have the help. > > Cheers. > Jeff > -- > J.W. Bizzaro jeff@bioinformatics.org > Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff > "Injustice anywhere is a threat to justice everywhere." > -- Martin Luther King, Jr. > -- > > _______________________________________________ > pipet-devel maillist - pipet-devel@bioinformatics.org > http://bioinformatics.org/mailman/listinfo/pipet-devel -- Timothy H. Keitt Department of Ecology and Evolution State University of New York at Stony Brook Phone: 631-632-1101, FAX: 631-632-7626 http://www.nceas.ucsb.edu/~keitt/ From jeff at bioinformatics.org Fri Sep 29 18:12:16 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <39D4E556.A2DF8048@hem2.passagen.se> <39D50BDC.2D89DD57@bioinformatics.org> <39D50E90.B0FAF33C@SUNYSB.Edu> Message-ID: <39D513C0.9B0EA8B0@bioinformatics.org> "Timothy H. Keitt" wrote: > > I am a lurker. :-) Really what I want to do is evaluate piper to see if it fits my > needs (environmental systems modeling, spatial ecology). It seems that the first > order of business is packaging. If users can download and try the software using > rpm/dpkg, then they will get interested contributing. (My preference would be debs as > I'm running debian. Most of the support libraries are already available in woody.) Hey Tim, Since I'm promoting the system to computer users of intermediate experience, packaging is a big issue to me. But, like everything else, we can't do it if no one here can help. I know that Brad, Jarl and Jean-Marc have all said that they don't care for packages, that they'd rather build from the source, so we can't expect them to do it. I might be able to help with RPM's, but I'll have to learn first :-P If you want to help build .deb packages, that'd be great. It would help bring Piper to a larger audience (although Debian users may know how to compile code, Corel users use dpkg too). Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Fri Sep 29 18:21:09 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <39D4E556.A2DF8048@hem2.passagen.se> <39D50BDC.2D89DD57@bioinformatics.org> <39D50E90.B0FAF33C@SUNYSB.Edu> <39D513C0.9B0EA8B0@bioinformatics.org> Message-ID: <39D515D5.6CDAA3F3@hermes.usherb.ca> > Since I'm promoting the system to computer users of intermediate experience, > packaging is a big issue to me. But, like everything else, we can't do it if > no one here can help. I know that Brad, Jarl and Jean-Marc have all said that > they don't care for packages, that they'd rather build from the source, so we > can't expect them to do it. I might be able to help with RPM's, but I'll have > to learn first :-P Did I ever say that? Actually I'm doing my best to ease the packaging operation of the PL stuff... I'm just going to leave the actual packaging to someone else ;-) Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jeff at bioinformatics.org Fri Sep 29 18:29:55 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <39D4E556.A2DF8048@hem2.passagen.se> <39D50BDC.2D89DD57@bioinformatics.org> <39D50E90.B0FAF33C@SUNYSB.Edu> <39D513C0.9B0EA8B0@bioinformatics.org> <39D515D5.6CDAA3F3@hermes.usherb.ca> Message-ID: <39D517E3.ED7FE13F@bioinformatics.org> Jean-Marc Valin wrote: > > Did I ever say that? Actually I'm doing my best to ease the packaging operation > of the PL stuff... I'm just going to leave the actual packaging to someone else > ;-) Perhaps it was just Jarl and Brad, but I don't recall anyone else being excited when I brought the topic up :-) Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From Timothy.Keitt at SUNYSB.Edu Fri Sep 29 18:39:10 2000 From: Timothy.Keitt at SUNYSB.Edu (Timothy H. Keitt) Date: Fri Feb 10 19:20:33 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <39D4E556.A2DF8048@hem2.passagen.se> <39D50BDC.2D89DD57@bioinformatics.org> <39D50E90.B0FAF33C@SUNYSB.Edu> <39D513C0.9B0EA8B0@bioinformatics.org> Message-ID: <39D51A0E.598E0098@SUNYSB.Edu> "J.W. Bizzaro" wrote: > "Timothy H. Keitt" wrote: > > > > I am a lurker. :-) Really what I want to do is evaluate piper to see if it fits my > > needs (environmental systems modeling, spatial ecology). It seems that the first > > order of business is packaging. If users can download and try the software using > > rpm/dpkg, then they will get interested contributing. (My preference would be debs as > > I'm running debian. Most of the support libraries are already available in woody.) > > Hey Tim, > > Since I'm promoting the system to computer users of intermediate experience, > packaging is a big issue to me. But, like everything else, we can't do it if > no one here can help. I know that Brad, Jarl and Jean-Marc have all said that > they don't care for packages, that they'd rather build from the source, so we Its much cleaner to build from a source package and then install the package rather than build from a tar ball, unless of course you're one of those unfortunate individuals who doesn't have a packing system! ;-) > > can't expect them to do it. I might be able to help with RPM's, but I'll have > to learn first :-P > > If you want to help build .deb packages, that'd be great. It would help bring > Piper to a larger audience (although Debian users may know how to compile > code, Corel users use dpkg too). > I've built rpms, but not debs. Anyway, I can't really offer the time although the project sounds quite interesting. Cheers, Tim -- Timothy H. Keitt Department of Ecology and Evolution State University of New York at Stony Brook Phone: 631-632-1101, FAX: 631-632-7626 http://www.nceas.ucsb.edu/~keitt/ From jeff at bioinformatics.org Fri Sep 29 19:10:49 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:34 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <39D4E556.A2DF8048@hem2.passagen.se> <39D50BDC.2D89DD57@bioinformatics.org> <39D50E90.B0FAF33C@SUNYSB.Edu> <39D513C0.9B0EA8B0@bioinformatics.org> <39D515D5.6CDAA3F3@hermes.usherb.ca> <39D517E3.ED7FE13F@bioinformatics.org> Message-ID: <39D52179.F577E341@bioinformatics.org> "J.W. Bizzaro" wrote: > > > Did I ever say that? Actually I'm doing my best to ease the packaging operation > > of the PL stuff... I'm just going to leave the actual packaging to someone else > > ;-) > > Perhaps it was just Jarl and Brad, but I don't recall anyone else being > excited when I brought the topic up :-) It was during an IRC Chat with Jarl and Brad, and I guess you weren't there. I'm sorry I accused you of being anti-package, Jean-Marc ;-) Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Injustice anywhere is a threat to justice everywhere." -- Martin Luther King, Jr. -- From jean-marc.valin at hermes.usherb.ca Fri Sep 29 19:16:58 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:34 2006 Subject: [Pipet Devel] welcome References: <39D3CDED.1F4D726D@bioinformatics.org> <39D4E556.A2DF8048@hem2.passagen.se> <39D50BDC.2D89DD57@bioinformatics.org> <39D50E90.B0FAF33C@SUNYSB.Edu> <39D513C0.9B0EA8B0@bioinformatics.org> <39D515D5.6CDAA3F3@hermes.usherb.ca> <39D517E3.ED7FE13F@bioinformatics.org> <39D52179.F577E341@bioinformatics.org> Message-ID: <39D522EA.39D5430C@hermes.usherb.ca> > It was during an IRC Chat with Jarl and Brad, and I guess you weren't there. > I'm sorry I accused you of being anti-package, Jean-Marc ;-) I'm totally *for* packages... as long as I'm not doing the packaging :-) Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jarl at casema.net Sat Sep 30 16:22:59 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:34 2006 Subject: [Pipet Devel] C -> C++ conversion of the BL References: <39D50F57.BDF16E21@bioinformatics.org> Message-ID: <39D64BA3.2BE40B50@casema.net> > Hi All, About the conversion of the BL to C++: I didn't want to announce the news about the current status because it's just one coding session to early, but let me say I dont need help on this one. More on this a little later. bye, jarl From andy.elvey at paradise.net.nz Sat Sep 30 20:50:19 2000 From: andy.elvey at paradise.net.nz (Andy Elvey) Date: Fri Feb 10 19:20:34 2006 Subject: [Pipet Devel] Re: pipet-devel digest, Vol 1 #175 - 11 msgs References: <200009301700.NAA18466@www.bioinformatics.org> Message-ID: <000901c02b41$935ba7e0$309260cb@andyelve> I'm also a lurker, but I'd like to mention that I strongly support the conversion of B.L. from C to C++ (KDE2 being a good example of a very well-done C++ app ... ) If Piper can eventually end up as good as that , then it will really be a "killer app" :-)