From bizzaro at geoserve.net Sat Jul 1 06:49:55 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:15 2006 Subject: [Pipet Devel] NewsAlert - Story Message-ID: <395DCCD3.C6A5B240@geoserve.net> ``The Open Speech Web - a network of interconnected speech services - combines technologies and products that will allow consumers to `speech-surf' or to connect to any speech-enabled application, simply by using spoken commands and the nearest phone.'' http://www.newsalert.com/bin/story?StoryId=CovWBqbKbytaWotu Jeff From chapmanb at arches.uga.edu Mon Jul 3 14:05:19 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:16 2006 Subject: [Pipet Devel] User interface stuff Message-ID: <200007031805.OAA88068@archa11.cc.uga.edu> Hey Jeff (and anyone else interested in Pied stuff): I just finished up with all of the MVC stuff that I'm going to do for the moment, so now you are free and clear in the user interface. Have fun! I started working on adding and removal of inputs and outputs, so that I could test and get working the "mapping unconnected inputs and outputs to the composite locus" stuff. This works "sort of" but has a number of problems: 1. The world x and y problems we were having before are rearing their ugly head again, and the new inputs get inserted in incorrect places, although they seem to come back to the right place once they are moved (weird). 2. I'm not sure how to deal with what happens when an 'external composite locus input mapping to an internal input' gets connected. (man, that's confusing). What should happen to the internal input? What happens in the internal locus having that input gets connected? What happens if the internal locus with the input gets deleted? Let me know if the way things are working is how you envisioned it, or if stuff needs to be changed. Brad From chapmanb at arches.uga.edu Mon Jul 3 14:07:28 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:16 2006 Subject: [Pipet Devel] Your ISMB 2000 poster Message-ID: <200007031807.OAA137038@archa10.cc.uga.edu> Jeff wrote: > Piper (http://bioinformatics.org/piper) is an interactive system for creating > and managing links between Internet-distributed components such as those used > for bioinformatics analyses. Components can reside remotely, on higher > performance and capacity computers, while only representations reside > locally. Links can depict protocol-independent data flow, procedural steps, > and relationships. > > 48 words. How's that? Very nice summary, and with two words to spare :-). Maybe we could make it exactly 50 by adding 'Piper rulz!' after the last sentence, whadda you think? Brad From chapmanb at arches.uga.edu Tue Jul 4 13:07:49 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:16 2006 Subject: [Pipet Devel] Overflow/gms integration into Piper Message-ID: <200007041707.NAA151716@archa12.cc.uga.edu> Hello all; I've just checked in a bunch of changes that allow Piper to be able to both save and load workflow-diagrams in a format compatible with Overflow (well, the saving part has been around for a while). This means that, at least for simple cases, we are able to make a work-flow-diagram in Overflow, save it, and load it up in Piper (and vice versa). This is pretty snazy, and I'd like to start working to expand it further so that all Overflow saved files can be opened in Piper. To do this, I'd like to have more Overflow 'node' types to play with, so that I can do crazier things and don't get too fixated on just making a couple of cases. Jean-Marc has written a snazzy perl script which can generate xml files for 'nodes' based on the information in the C++ header/implementation file. What I'd like to try and do is use this to generate the xml files for all the nodes at compile time, so then we don't need to distribute all of these millions of XML files with Piper, and also so that we can change XML formats if we need to without having to change a billion XML files. So in order to do this, we would need to have the Overflow C++ files in the piper module. So, I was wondering about how people felt about putting Overflow and also GMS into Piper as bl and pl directories, respectively. I've talked with Jarl about this a bit and I think he might almost be ready to do this. From Overflow, I would just like to put in the 'data-flow' directory for now, since this will give us the "basic" nodes for doing things. Putting things into Piper will also bring us closer to integration, which I also like :-). So, what does everything think about this? Objections? Thoughts? Comments? Are there better ways to move forward? Thanks for listening! Brad From bizzaro at geoserve.net Tue Jul 4 14:15:56 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:16 2006 Subject: [Pipet Devel] Overflow/gms integration into Piper References: <200007041707.NAA151716@archa12.cc.uga.edu> Message-ID: <002701bfe5e3$e6fadb80$6b5dba8c@bizzaro> Greetings from the Berkshire mountains. The reason I've been quiet lately, is because I'm on vacation (holiday, for all you Brittish chaps) with my family. I actually brought my computer with me (I am truly die-hard) so that I could have some real fun with all of my projects. The only problem is, my ISP just changed over to some non-standard dial-in system that only works with Windblows (probably made by Mickysoft), and so I can't use my box to get on the net. I'm stuck using my brother's Libretto with Win-D'Ohs and teeeny leeetle keys. The short of it is, I can check my e-mail, but I can't use CVS. Brad, what you're planning on doing sounds great to me. Keep us up-to-date. Also, thanks for all the hard work :-) Cheers. Jeff From jarl at casema.net Wed Jul 5 00:39:33 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:16 2006 Subject: [Pipet Devel] Overflow/gms integration into Piper References: <200007041707.NAA151716@archa12.cc.uga.edu> Message-ID: <3962BC05.95798E35@casema.net> > So in order to do this, we would need to have the Overflow C++ > files in the piper module. So, I was wondering about how people felt > about putting Overflow and also GMS into Piper as bl and pl > directories, respectively. I've talked with Jarl about this a bit and > I think he might almost be ready to do this. From Overflow, I would > just like to put in the 'data-flow' directory for now, since this will > give us the "basic" nodes for doing things. Putting things into Piper > will also bring us closer to integration, which I also like :-). > So, what does everything think about this? Objections? Thoughts? > Comments? Are there better ways to move forward? > Thanks for listening! I have no problem with intergrating GMS into the piper cvs, but 1st a few TODO's need to be finished: 1- debugging of previous work 2- removal of gnome dependancies in the dataflow processor ( 3- some 'primitive' linking with overflow 1 & 2 are mostly done, and when they are finished I'll start working on 3. After this I'll take action to have GMS put into cvs. bye, jarl From bizzaro at geoserve.net Wed Jul 5 01:03:36 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:16 2006 Subject: [Pipet Devel] links and stuff Message-ID: <000901bfe63e$650bdba0$455dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: MIT Oxygen Project.url Type: application/octet-stream Size: 132 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000705/01192b06/MITOxygenProject.obj From bizzaro at geoserve.net Thu Jul 6 12:45:41 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:17 2006 Subject: [Pipet Devel] The Gestalt System Project Message-ID: <000901bfe769$a37ca320$675dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: The Gestalt System Project.url Type: application/octet-stream Size: 148 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000706/a45f0952/TheGestaltSystemProject.obj From bizzaro at geoserve.net Thu Jul 6 15:42:59 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:17 2006 Subject: [Pipet Devel] Fw: [GS-discuss] Piper Message-ID: <00ea01bfe782$656efb00$6b5dba8c@bizzaro> I hope this is the start of a discussion thread with the Gestalt developers. Jean-Marc or anyone else, could you reply to Richard (and the gestalt list) with an answer to his question below? Jeff ----- Original Message ----- From: Richard Wherry To: 'J.W. Bizzaro' ; gestalt-system-discuss@lists.sourceforge.net Sent: Thursday, July 06, 2000 2:56 PM Subject: RE: [GS-discuss] Piper Jeff, Looks very interesting. It could be used as a framework for the Gestalt components which could be individual data manipulation programs (for example). I'd be interested in hearing the opinion of someone who has a more concrete idea of the goals of Gestalt System than myself. If I had a dataset consisting of continuous variables and categorical variables sitting on a machine somewhere and wanted to compute averages for all continuous variables and frequencies for all categorical variables, how would this be accomplished? It brings to mind Clementine from SPSS - at least in the "Work Flow Diagram" sense. Regards, Richard -----Original Message----- From: J.W. Bizzaro [mailto:bizzaro@geoserve.net] Sent: Thursday, July 06, 2000 12:53 PM To: gestalt-system-discuss@lists.sourceforge.net Subject: [GS-discuss] Piper Greetings. I think Gestalt and Piper have some similar goals: http://theopenlab.org/piper Piper is a recent merger or collaboration between 4 projects: Loci, Overflow, GMS, and BlueBox. I'm not certain where there may be some overlap. Perhaps you could take a look at Piper and let me know what you think. Cheers. Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000706/530173e2/attachment.html From chapmanb at arches.uga.edu Fri Jul 7 13:01:40 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:17 2006 Subject: [Pipet Devel] Fw: [GS-discuss] Piper Message-ID: <200007071701.NAA71762@archa11.cc.uga.edu> On the Piper list Jeff wrote: > I hope this is the start of a discussion thread with the Gestalt developers. > > Jean-Marc or anyone else, could you reply to Richard (and the gestalt list) > with an answer to his question below? Well, I can be the "anyone else" and take a stab at it :-) Richard wrote, on the Gestalt list: > Looks very interesting. It could be used as a framework for the > Gestalt components which could be individual data manipulation programs > (for example). I think that makes sense for integrating the ideas (although I don't completely understand the Gestalt system yet). Piper is meant to be a system for connecting together different "components" which actually do the functional work, so it is kind of a framework, as you suggest. > If I had a dataset consisting of continuous variables and categorical > variables sitting on a machine somewhere and wanted to compute averages for > all continuous variables and frequencies for all categorical variables, > how would this be accomplished? All of the actual computational work in this type of example would be done by individual programs, and not by Piper itself. The purpose of Piper would be to contact the remote machine with the data (which is presumedly also running Piper) to obtain the dataset. The actual computation could then be done at the remote machine (if a program was available to do it), or at the local machine. Either way, the results would be displayed at the local machine (obviously, otherwise you would never get to see them :-). The idea behind Piper is to create a flexible way to interact with remote and local data and programs and connect them together to make something happen. Currently, coding is directed at integrating Loci, GMS and Overflow (this is/will become Piper) and just dealing with local integrating amongst "components." This is coming along well so far, and once this gets going, remote communication will be the next thing to come (since that is really what I'm interested in :-). Does this make any sense? If not, let me know and I can try to make more sense... Brad From bizzaro at geoserve.net Sun Jul 9 08:13:52 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:17 2006 Subject: [Pipet Devel] [fm] modularity Message-ID: <001301bfe99f$29ba2480$3d5dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: [fm] modularity.url Type: application/octet-stream Size: 174 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000709/85a22e86/fmmodularity.obj From bizzaro at geoserve.net Sun Jul 9 08:47:49 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:17 2006 Subject: [Pipet Devel] Idle PCs to help research projects Message-ID: <000901bfe9a3$e921a2e0$3d5dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: Linux Today - VNU Net Idle PCs to help research projects.url Type: application/octet-stream Size: 706 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000709/d85b2e78/LinuxToday-VNUNetIdlePCstohelpresearchprojects.obj From bizzaro at geoserve.net Sun Jul 9 08:55:38 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:17 2006 Subject: [Pipet Devel] Low-Bandwidth Communication Tools for Science Message-ID: <001301bfe9a4$fe84fd20$3d5dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: LJ 75 Low-Bandwidth Communication Tools for Science.url Type: application/octet-stream Size: 184 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000709/03c7d035/LJ75Low-BandwidthCommunicationToolsforScience.obj From bizzaro at geoserve.net Sun Jul 9 11:15:06 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:17 2006 Subject: [Pipet Devel] this is scary Message-ID: <000901bfe9b8$826d03e0$3d5dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: The BizTalk Framework.url Type: application/octet-stream Size: 160 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000709/4607577e/TheBizTalkFramework.obj From bizzaro at geoserve.net Sun Jul 9 13:27:52 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:17 2006 Subject: [Pipet Devel] osOpinion: .NET Are we insane? Message-ID: <000901bfe9cb$08adbd20$7f5dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: osOpinion .NET Are we insane.url Type: application/octet-stream Size: 202 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000709/9b1ce68d/osOpinion.NETAreweinsane.obj From bizzaro at geoserve.net Sun Jul 9 16:23:36 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] Re: [GS-discuss] Piper References: Message-ID: <008301bfe9e5$50a16400$be5dba8c@bizzaro> From: Richard Wherry Looks very interesting. It could be used as a framework for the Gestalt components which could be individual data manipulation programs (for example). As you and Brad have suggested, I think it's a strong possibility. I'd be interested in hearing the opinion of someone who has a more concrete idea of the goals of Gestalt System than myself. I look forward to hearing from Tim and Karsten about how useful Piper might be in achieving the goals of Gestalt. (I appreciate your reply too, Richard :-)) If I had a dataset consisting of continuous variables and categorical variables sitting on a machine somewhere and wanted to compute averages for all continuous variables and frequencies for all categorical variables, how would this be accomplished? Brad replied to this part of your message, but I'll add that you may want to take a closer look at Overflow, one of the collaborating projects: http://freespeech.sourceforge.net/FreeSpeech/html/Overflow/home.html Overflow is a very fast, low-level, ``data flow'' system that operates on a local host. It can be said that the goal of Piper is to distribute instances of Overflow around the Internet. Overflow forms the basis of Piper. All Piper nodes are either Overflow nodes or they are programs that are wrapped by Overflow. It may be that it can be extended to manipulate data the way the Gestalt wants to, if it can not already. It brings to mind Clementine from SPSS - at least in the "Work Flow Diagram" sense. You're very observant. Clementine was actually the inspiration for Piper's WFD (formerly part of The Loci Project and now part of the Pied/Piper UI). You may also be interested in Cerebellum: http://www.cerebellumsoft.com/product/designer.html But, so far, the only person to take on the responsibility of database integration (doing mining, manipulation, etc.) with Piper is Brad, and he is already over-extended and over-worked. As I mentioned in my first message, Piper is a collaborative between 4 relatively independent projects: BlueBox, Loci, GMS, and Overflow. All 4 projects are complementary, and we respect the code base of each (at least Brad keeps reminding me of that ;-) -- inside joke). Our reason for collaborating is, as appears in the Gestalt Manifesto, that this project is too large for one or two developers to take on. Perhaps, if it fits in with Gestalt's goals, Gestalt can be the 5th collaborator and take on the task of database access, as well as some aspects of data manipulation and transfer...? Let me know what you think. Cheers. Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000709/d903c820/attachment.html From bizzaro at geoserve.net Sun Jul 9 17:22:03 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] Gri A Language for Scientific Illustration Message-ID: <000901bfe9eb$c01bd8a0$a25dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: LJ 75 Gri A Language for Scientific Illustration.url Type: application/octet-stream Size: 184 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000709/bc25df9b/LJ75GriALanguageforScientificIllustration.obj From bizzaro at geoserve.net Sun Jul 9 18:24:44 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] Fw: [GS-discuss] Piper Message-ID: <00a501bfe9f4$7d5e7e60$be5dba8c@bizzaro> ----- Original Message ----- From: Tim Churches To: J.W. Bizzaro Cc: Karsten M. Self Sent: Sunday, July 09, 2000 6:53 PM Subject: Re: [GS-discuss] Piper > Jeff, > > I've been out of action with some health problems for the last few > weeks, but am just catching up with my mail - will read all the piper > papers and docs over the next few days but at first glance Piper appears > to be manna from heaven. We have been making slow but definite progress > on some elementary data analysis and reporting tools and have done some > initial detailed design of data definition and metadata components, but > have been floundering on how to hook it all up and have been depressed > about having only a non-graphical command line interface for the > forseeable future. The Piper paradigm looks ideally suited to the sort > of work we envisage for Gestalt - or at least, the sort of work I do - > human population-based epidemiology and demographics - and for many > other fields as well, no doubt. I'll respond in detail in a few days but > thanks for the post - very helpful indeed. > > Cheers, > > Tim C > From bizzaro at geoserve.net Mon Jul 10 10:48:57 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] XLink Message-ID: <000901bfea7d$ff6b58e0$b25dba8c@bizzaro> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: Linux Today - InternetNews.com XML Standards Move Forward.url Type: application/octet-stream Size: 662 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/20000710/b61a1f31/LinuxToday-InternetNews.comXMLStandardsMoveForward.obj From lahondes at pasteur.fr Mon Jul 10 13:22:55 2000 From: lahondes at pasteur.fr (Raynald de Lahondes) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] XML and Pise Message-ID: <20000710192255.A6931@pombe.vironc.pasteur.fr> Hi, Although I feel interested by Piper project, I followed a bit the Loci project, but I am new to lot of things there, so it is likely that I am not aware of some decisions already taken. However, it seems that the Piper XML is not yet completely defined. I am working in the Pasteur Institute and there is a software called Pise: http://www-alt.pasteur.fr/~letondal/Pise/ written by Catherine Letondal (letondal@pasteur.fr). In two words, this is an interface builder. It makes HTML pages and CGI scripts for unix programmes related with Biology. My point is it uses XML to describe the program to be interfaced. This program is under GPL and well documented. I think it would be great if Piper and Pise could share the same DTD... Pise DTD http://www-alt.pasteur.fr/~letondal/XML/pise.dtd I also think that Pise is a working program with some (short) history now and maybe Piper could benefit from the maturity of its DTD. Although Catherine Letondal is not willing to modify her code right now, I know that she may accept if the Piper project is willing to adopt her DTD with some (minor) modifications. What do you think of that ? -- Raynald de Lahondes Unite des Virus Oncogenes - Departement de Biotechnologie Institut Pasteur - 25, rue du Docteur Roux 75724 Paris Cedex 15 - FRANCE tel: 01.45.68.84.54 - fax: 01.40.61.30.33 - cellular: 06.15.65.85.08 email: lahondes@pasteur.fr From bizzaro at geoserve.net Mon Jul 10 14:36:59 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] XML and Pise References: <20000710192255.A6931@pombe.vironc.pasteur.fr> Message-ID: <00ec01bfea9d$d85557e0$be5dba8c@bizzaro> Raynald wrote: > However, it seems that the Piper XML is not yet completely defined. Here are some of the DTD's made by Brad: https://alpha.bioinformatics.org/cgi-bin/cvsweb.cgi/piper/dtd/ It is a work-in-progress, since Piper is so complex and we are just now trying to glue some very big pieces together. > In two words, this is an interface builder. It makes HTML pages and CGI > scripts for unix programmes related with Biology. My point is it uses XML to > describe the program to be interfaced. This program is under GPL and well > documented. I think it would be great if Piper and Pise could share the same > DTD... > > Pise DTD > http://www-alt.pasteur.fr/~letondal/XML/pise.dtd It would be great, but I think Piper's DTD for wrapping command-line programs will be rather unique, based on the connection of Overflow nodes. Also, Piper is meant to be general-purpose, not biology-related (/Loci/ was originally supposed to be biology-related). I see some Pise tags like "Sequence", which we wouldn't want to use. I think Pise's DTD will be helpful to study, but without a lot of modification (which you said is not acceptable), we couldn't make direct use of it. I'd like to hear Brad's and Jean-Marc's opinions as well. It's good hearing from you again, Raynald :-) Cheers. Jeff From bizzaro at geoserve.net Mon Jul 10 17:02:17 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] Re: hello Message-ID: <396A39D9.397F2331@geoserve.net> Whew! I finally got my Linux box back on the Net! Damn Geoserve! Goodbye Windblows! I had to switch ISP's, but my e-mail will remain the same for a while. Anyway, this message is from Catherine Letondal, who is now on this list. I hope you don't mind me posting it. ----------------------------------------- Sorry to answer so late... I have indeed read the papers about Piper and found it very interesting (I have just subscribed to the mailing list). I'm doing a PhD on End-user programming, with an application to biology (http://www.pasteur.fr/~letondal/Papers/chi_pp.html). The main aim is to integrate programming facilities into the biologist's sequence analysis tools, at least at a scripting level (I'm using Xotcl, which is an object extension of Tcl http://www.xotcl.org). For this purpose, I'm developping a scriptable alignment editor and sequence analysis environment, where GUI components as well as functional components are accessible from the user interface. In other words, I'm interested into Open implementation and meta-object protocols ideas, as well as usability issues, with biologists actively participating to the design (http://www.pasteur.fr/~letondal/These/links-ihm.html). ----------------------------------------- Cheers. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From bizzaro at geoserve.net Mon Jul 10 18:47:49 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] slashdot comment Message-ID: <396A5295.47CD8DC@geoserve.net> I posted a comment on Slashdot: http://slashdot.org/comments.pl?sid=00/07/10/1312245&cid=76 in response to an article at CNN about "distributed parallel computing": http://www.cnn.com/2000/TECH/computing/07/07/seti.implication.idg/index.html I call Piper "distributed serial computing" and say that it allows for distribution where the data cannot be parallelized. I think it's an interesting new twist on "whatever the heck Piper is". Follow the links, and if you have moderator status, push my score up :-) Cheers. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From rossini at biostat.washington.edu Tue Jul 11 12:46:33 2000 From: rossini at biostat.washington.edu (A.J. Rossini) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] Re: [GS-discuss] Piper In-Reply-To: "J.W. Bizzaro"'s message of "Sun, 9 Jul 2000 16:23:36 -0400" References: <008301bfe9e5$50a16400$be5dba8c@bizzaro> Message-ID: <87itucfwd2.fsf@alpha.cfas.washington.edu> >>>>> "JWB" == J W Bizzaro writes: JWB> you may want to take a closer look at Overflow, one of the JWB> collaborating projects: JWB>   JWB> http://freespeech.sourceforge.net/FreeSpeech/html/Overflow/home.html JWB> Overflow is a very fast, low-level, ``data flow'' system JWB> that operates on a local host.  It can be said that the goal JWB> of Piper is to distribute instances of Overflow around the JWB> Internet. Cool... A related project, which looks at workflow from a different perspective (more of a guidance mechanism than not) is ViSta, cf http://forrest.psych.unc.edu/, a visual (2-d) statistics system written over XLispStat. But of course, the goal is to distribute... -- A.J. Rossini Research Assistant Professor of Biostatistics Biostatistics/Univ. of Washington (Th) Box 357232 206-543-1044 (3286=fax) Center for AIDS Research/HMC/UW (M/F) Box 359931 206-731-3647 (3693=fax) VTN/SCHARP/FHCRC (Tu/W) Box 358080 206-667-7025 (4812=fax) rossini@(biostat.washington.edu|u.washington.edu|scharp.org) http://www.biostat.washington.edu/~rossini From bizzaro at geoserve.net Tue Jul 11 13:36:05 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] Re: Piper References: <008301bfe9e5$50a16400$be5dba8c@bizzaro> <87itucfwd2.fsf@alpha.cfas.washington.edu> Message-ID: <396B5B05.AF8275D9@geoserve.net> "A.J. Rossini" wrote: > > A related project, which looks at workflow from a different > perspective (more of a guidance mechanism than not) is ViSta, cf > http://forrest.psych.unc.edu/, a visual (2-d) statistics system > written over XLispStat. I haven't seen that one before. Thanks for the reference! I'll add it to my collection. Here is my collection of the various workflow and tree diagrams I've found around the Internet (needs to be updated soon): http://bioinformatics.org/loci/screenshots/wfd-examples/ Cheers. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From bizzaro at geoserve.net Tue Jul 11 15:17:02 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] Slashdot | JavaSpaces Principles, Patterns and Practice Message-ID: <396B72AE.70B4DB87@geoserve.net> In addition to Micrsoft's .NET/BizTalk and IBM's Websphere, Piper probably competes with Sun's Jini/JavaSpaces in some aspects. Here's a book review on a book about JavaSpaces: http://slashdot.org/article.pl?sid=00/07/02/2157205&mode=nocomment Jeff From rossini at biostat.washington.edu Tue Jul 11 17:30:42 2000 From: rossini at biostat.washington.edu (A.J. Rossini) Date: Fri Feb 10 19:20:18 2006 Subject: [Pipet Devel] XML-RPC Message-ID: <87ituce4n1.fsf@alpha.cfas.washington.edu> I'm slowly going over the archive of the mailing list for this project -- havn't seen XML-RPC mentioned, but lots of ties to projects where it should've fit. Is this under consideration as a mechanism for constructing glue or for serialization? best, -tony -- A.J. Rossini Research Assistant Professor of Biostatistics Biostatistics/Univ. of Washington (Th) Box 357232 206-543-1044 (3286=fax) Center for AIDS Research/HMC/UW (M/F) Box 359931 206-731-3647 (3693=fax) VTN/SCHARP/FHCRC (Tu/W) Box 358080 206-667-7025 (4812=fax) rossini@(biostat.washington.edu|u.washington.edu|scharp.org) http://www.biostat.washington.edu/~rossini From bizzaro at geoserve.net Tue Jul 11 17:43:19 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] [Fwd: XML-RPC] Message-ID: <396B94F7.AC5F8BA5@geoserve.net> Tony, attached are 6 messages that I have found (from the Loci/Tulip project) that contain a reference to XML-RPC. Jeff -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: [Pipet Devel] XML-RPC Date: Thu, 04 Feb 1999 08:22:56 +0000 Size: 593 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000711/9c2eba5c/attachment.mht -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: Re: [Pipet Devel] XML-RPC Date: Thu, 4 Feb 1999 09:44:48 +0100 Size: 994 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000711/9c2eba5c/attachment-0001.mht -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: Re: [Pipet Devel] and still more infrastructure things Date: Mon, 1 Mar 1999 10:55:37 +0100 Size: 2068 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000711/9c2eba5c/attachment-0002.mht -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: [Pipet Devel] XML-RPC Date: Fri, 25 Jun 1999 12:18:32 -0400 Size: 613 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000711/9c2eba5c/attachment-0003.mht -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: [Pipet Devel] E-Speak Date: Thu, 11 Nov 1999 01:02:46 +0000 Size: 1948 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000711/9c2eba5c/attachment-0004.mht -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: [Pipet Devel] Re: Sodipodi and Loci Date: Mon, 17 Jan 2000 07:52:46 +0000 Size: 6709 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000711/9c2eba5c/attachment-0005.mht From bizzaro at geoserve.net Tue Jul 11 18:18:53 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> Message-ID: <396B9D4D.38B8458F@geoserve.net> "A.J. Rossini" wrote: > > I'm slowly going over the archive of the mailing list for this project > -- havn't seen XML-RPC mentioned, but lots of ties to projects where > it should've fit. If you want to dig through more archives, there are the archives for Loci, which date back to April 1999: http://bioinformatics.org/pipermail/pipet-devel/ Before that, Loci was known as Tulip, and the discussions went back to late 1998. Those archives aren't on-line though. > Is this under consideration as a mechanism for constructing glue or > for serialization? The short answer is, the two guys involved in gluing things together (Brad and Jarl) are familiar with CORBA and seem to prefer it over other models. From discussions we've had, it seems CORBA is superior in terms of speed and security. We were seriously considering an XML-based RPC, and I even invented my own (with Brad's help). But, in the end, it proved to be too slow and in need of too much development. Cheers. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From rossini at biostat.washington.edu Tue Jul 11 18:57:23 2000 From: rossini at biostat.washington.edu (A.J. Rossini) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC In-Reply-To: "J.W. Bizzaro"'s message of "Tue, 11 Jul 2000 22:18:53 +0000" References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> Message-ID: <878zv8e0mk.fsf@alpha.cfas.washington.edu> >>>>> "JWB" == J W Bizzaro writes: JWB> If you want to dig through more archives, there are the JWB> archives for Loci, which date back to April 1999: JWB> http://bioinformatics.org/pipermail/pipet-devel/ JWB> Before that, Loci was known as Tulip, and the discussions JWB> went back to late JWB> 1998. Those archives aren't on-line though. Thanks for the information/context. AJR> Is this under consideration as a mechanism for constructing AJR> glue or for serialization? JWB> The short answer is, the two guys involved in gluing things JWB> together (Brad and Jarl) are familiar with CORBA and seem to JWB> prefer it over other models. From discussions we've had, it JWB> seems CORBA is superior in terms of speed and security. Agreed, in a sense (speed, yes; security, depends and is extremely contextual, so let's not go down that route...). JWB> We were seriously considering an XML-based RPC, and I even JWB> invented my own (with Brad's help). But, in the end, it JWB> proved to be too slow and in need of too much development. Thanks for the info! best, -tony -- A.J. Rossini Research Assistant Professor of Biostatistics Biostatistics/Univ. of Washington (Th) Box 357232 206-543-1044 (3286=fax) Center for AIDS Research/HMC/UW (M/F) Box 359931 206-731-3647 (3693=fax) VTN/SCHARP/FHCRC (Tu/W) Box 358080 206-667-7025 (4812=fax) rossini@(biostat.washington.edu|u.washington.edu|scharp.org) http://www.biostat.washington.edu/~rossini From bizzaro at geoserve.net Tue Jul 11 19:41:27 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> Message-ID: <396BB0A7.D22450E4@geoserve.net> "A.J. Rossini" wrote: > > JWB> From discussions we've had, it > JWB> seems CORBA is superior in terms of speed and security. > > Agreed, in a sense (speed, yes; security, depends and is extremely > contextual, so let's not go down that route...). I'm not an expert on any of the models. I am simply relying on what others here tell me. Jarl has made favorable comments regarding CORBA's security model. Also, regarding XML-RPC, SOAP and .NET (which, it appears, are now related), I have HEARD that the intention is to run them through HTTP in order to bypass firewalls. These are the reasons why I said CORBA is "superior" (bad choice of words -- it tends to cause more flame wars). Cheers. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From bizzaro at geoserve.net Tue Jul 11 20:10:31 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> Message-ID: <396BB777.A92BD56@geoserve.net> "A.J. Rossini" wrote: > > JWB> From discussions we've had, it > JWB> seems CORBA is superior in terms of speed... > > Agreed, in a sense (speed, yes;... This is ultimately more important to Piper than it may first appear. The initial impression that most people may get of Piper is that it is a system for /sequencial/ data flow (the data is passed to the next node only when the current node is done). If we didn't have Overflow at the core of Piper, this may be true. But Overflow is such a fast system that we want to include /streaming/ data flow. Here's an excerpt from one of Jean-Marc's (Overflow developer) recent e-mails: Jean-Marc wrote: > I just thought this might be interesting to some of you, as a demo of what > Overflow can do. I just "wrote" an Overflow program (.n) that performs real-time > audio processing. It reads the soundcard input, normalized the volume (lowers > louder sounds, amplifies lower sounds), and sends the result back to the > soundcard output. It takes less than 5% CPU on my Athlon 500 at 44.1kHz/stereo > (I use chunks of ~10 ms). Note, you need a full-duplex soundcard and the latest > version in CVS to try it. Any electric guitar player here would like to help me > write distortions and other effects? One may argue, however, that the lack of speed in transferring data across the Internet makes this a futile goal. But, we're ambitious, and things will only improve :-) Cheers. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From deanne at density.biop.umich.edu Tue Jul 11 19:56:15 2000 From: deanne at density.biop.umich.edu (deanne@density.biop.umich.edu) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC In-Reply-To: <396BB777.A92BD56@geoserve.net> Message-ID: Jeff wrote: > One may argue, however, that the lack of speed in transferring data across the > Internet makes this a futile goal. But, we're ambitious, and things will only > improve :-) And among "local nets", things are certainly fast enough for this to be true! Internal networks (like, at my University) are good examples. Yours, pedantically, Deanne From bizzaro at geoserve.net Tue Jul 11 20:17:47 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: Message-ID: <396BB92B.5B96879E@geoserve.net> deanne@density.biop.umich.edu wrote: > > Jeff wrote: > > > One may argue, however, that the lack of speed in transferring data across the > > Internet makes this a futile goal. But, we're ambitious, and things will only > > improve :-) > > And among "local nets", things are certainly fast enough for this to be > true! Internal networks (like, at my University) are good examples. That's a good point. And it brings up a good question for Jean-Marc (if you're reading this): What sort of (physical) network speeds would be acceptable for your work in audio processing? Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From rossini at biostat.washington.edu Tue Jul 11 20:24:14 2000 From: rossini at biostat.washington.edu (A.J. Rossini) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC In-Reply-To: "J.W. Bizzaro"'s message of "Wed, 12 Jul 2000 00:10:31 +0000" References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> Message-ID: <87em50ci1d.fsf@alpha.cfas.washington.edu> >>>>> "JWB" == J W Bizzaro writes: JWB> This is ultimately more important to Piper than it may first JWB> appear. The initial impression that most people may get of JWB> Piper is that it is a system for /sequencial/ data flow (the JWB> data is passed to the next node only when the current node is JWB> done). If we didn't have Overflow at the core of Piper, this JWB> may be true. But Overflow is such a fast system that we want JWB> to include /streaming/ data flow. Here's an excerpt from one JWB> of Jean-Marc's (Overflow developer) recent e-mails: JWB> Jean-Marc wrote: >> I just thought this might be interesting to some of you, as a >> demo of what Overflow can do. I just "wrote" an Overflow >> program (.n) that performs real-time audio processing. It reads >> the soundcard input, normalized the volume (lowers louder >> sounds, amplifies lower sounds), and sends the result back to >> the soundcard output. It takes less than 5% CPU on my Athlon >> 500 at 44.1kHz/stereo (I use chunks of ~10 ms). Note, you need >> a full-duplex soundcard and the latest version in CVS to try >> it. Any electric guitar player here would like to help me write >> distortions and other effects? No, it (speed) is quite important. I'd personally like to stream megabytes of data (in a statistical, "flat-file"-ish sense) and process it at the same time. The only rationale for XML-RPC is "simplicity". We'll keep it in quotes, since I'm saying that out of context and it implies a set decision-making criteria behind it. Again, I'm not interested in baiting, just finding out what has been explored/considered, and what the framework is. We (one of my "affiliations") are looking into wet-lab informatics/data processing, as well as cross-institution/group database collection (wet lab + epidemiological/health services data). Piper has the right requirements, if implemented reasonably quickly, to be a tool we could use (and hence consider developing for, or even just developing). Glue is good, and fast drying stable glue is better :-). best, -tony -- A.J. Rossini Research Assistant Professor of Biostatistics Biostatistics/Univ. of Washington (Th) Box 357232 206-543-1044 (3286=fax) Center for AIDS Research/HMC/UW (M/F) Box 359931 206-731-3647 (3693=fax) VTN/SCHARP/FHCRC (Tu/W) Box 358080 206-667-7025 (4812=fax) rossini@(biostat.washington.edu|u.washington.edu|scharp.org) http://www.biostat.washington.edu/~rossini From bizzaro at geoserve.net Tue Jul 11 20:47:29 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <87em50ci1d.fsf@alpha.cfas.washington.edu> Message-ID: <396BC021.1711B2E7@geoserve.net> "A.J. Rossini" wrote: > > We (one of my "affiliations") are looking into wet-lab > informatics/data processing, as well as cross-institution/group > database collection (wet lab + epidemiological/health services data). Interesting. Loci (now part of Piper) was originally developed as a bioinformatics application. We intended, and still do intend, to set up "collaboratories" based on the system. > Piper has the right requirements, if implemented reasonably quickly, By "if implemented reasonably quickly", do you mean (1) "if Piper becomes functional soon" or (2) "if Piper networks can be assembled quickly"? If you mean (1), we'd like to get things working soon too. Right now, the big challenge is merging Loci, GMS, and Overflow (not to mention BlueBox). Brad and Jarl have been veritable busy beavers, while I spend most of my time posting brain boogers to the list :-) But no-one has complained too much :-P > to be a tool we could use (and hence consider developing for, or even > just developing). Hmmm. I'd like to hear more :-) Cheers. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From jean-marc.valin at hermes.usherb.ca Tue Jul 11 20:54:56 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> Message-ID: <396BC1E0.B369F8DB@hermes.usherb.ca> > This is ultimately more important to Piper than it may first appear. The > initial impression that most people may get of Piper is that it is a system > for /sequencial/ data flow (the data is passed to the next node only when the > current node is done). If we didn't have Overflow at the core of Piper, this > may be true. But Overflow is such a fast system that we want to include > /streaming/ data flow. Here's an excerpt from one of Jean-Marc's (Overflow > developer) recent e-mails: > > Jean-Marc wrote: > > I just thought this might be interesting to some of you, as a demo of what > > Overflow can do. I just "wrote" an Overflow program (.n) that performs real-time > > audio processing. It reads the soundcard input, normalized the volume (lowers > > louder sounds, amplifies lower sounds), and sends the result back to the > > soundcard output. It takes less than 5% CPU on my Athlon 500 at 44.1kHz/stereo > > (I use chunks of ~10 ms). Note, you need a full-duplex soundcard and the latest > > version in CVS to try it. Any electric guitar player here would like to help me > > write distortions and other effects? I'd just like to explain a bit more here. There is no special "streaming feature" in Overflow. What allows me to process sound in real-time is that Overflow, as any "language", allows loops (it takes a while to get used to loops -- we call them "Iterator" networks -- in a data-flow). If you want to do streaming, it's your responsability to cut down your nodes so they can work on a chunk of data. If you do that, then you can stream. As for the speed of the network, it won't give you CD quality in real-time, but it can allow you to split an hour-long processing in chunks of 1 minute. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From bizzaro at geoserve.net Tue Jul 11 21:06:49 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <396BC1E0.B369F8DB@hermes.usherb.ca> Message-ID: <396BC4A9.3E29274@geoserve.net> Jean-Marc Valin wrote: > > I'd just like to explain a bit more here. There is no special "streaming > feature" in Overflow. What allows me to process sound in real-time is that > Overflow, as any "language", allows loops (it takes a while to get used to loops > -- we call them "Iterator" networks -- in a data-flow). If you want to do > streaming, it's your responsability to cut down your nodes so they can work on a > chunk of data. If you do that, then you can stream. As for the speed of the > network, it won't give you CD quality in real-time, but it can allow you to > split an hour-long processing in chunks of 1 minute. So, it's really a matter of "granularity". Overflow works sequencially but appears to be streaming because there are so many nodes, working on such little chunks, so quickly. Is that correct? Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From jean-marc.valin at hermes.usherb.ca Tue Jul 11 21:54:41 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <396BC1E0.B369F8DB@hermes.usherb.ca> <396BC4A9.3E29274@geoserve.net> Message-ID: <396BCFE1.500415ED@hermes.usherb.ca> > So, it's really a matter of "granularity". Overflow works sequencially but > appears to be streaming because there are so many nodes, working on such > little chunks, so quickly. Is that correct? Yes, although for users, it's almost transparent. Since all the audio nodes are made to work this way (though they can work on a big vector), if you edit the inside of the loop, it looks like the individual nodes are "streaming". Also, what do you mean by "because there are so many nodes"? Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From bizzaro at geoserve.net Tue Jul 11 21:56:08 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <396BC1E0.B369F8DB@hermes.usherb.ca> <396BC4A9.3E29274@geoserve.net> <396BCFE1.500415ED@hermes.usherb.ca> Message-ID: <396BD038.F60C4B43@geoserve.net> Jean-Marc Valin wrote: > > Also, what do you mean by "because there are so many nodes"? Many nodes => smaller chunks Few nodes => bigger chunks Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From jean-marc.valin at hermes.usherb.ca Tue Jul 11 22:04:47 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <396BC1E0.B369F8DB@hermes.usherb.ca> <396BC4A9.3E29274@geoserve.net> <396BCFE1.500415ED@hermes.usherb.ca> <396BD038.F60C4B43@geoserve.net> Message-ID: <396BD23F.FD850B34@hermes.usherb.ca> "J.W. Bizzaro" a ?crit : > Many nodes => smaller chunks > Few nodes => bigger chunks Well, in this case, there is the same number of nodes. It's just that each node receives 10000 chunks of data instead of 1. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From bizzaro at geoserve.net Tue Jul 11 22:08:51 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] GnoP - GNOME Properties Message-ID: <396BD333.5BE3A4D0@geoserve.net> There are some similarities here between GnoP and how I intended the UI XML to be assembled (will now be handled by BlueBox, which BTW Nile says will be out this week): http://www.5z.com/jirka/gnop.html Here's a sample XML file: http://www.5z.com/jirka/testgnop.gnop Cheers. Jeff From bizzaro at geoserve.net Tue Jul 11 22:18:26 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <396BC1E0.B369F8DB@hermes.usherb.ca> <396BC4A9.3E29274@geoserve.net> <396BCFE1.500415ED@hermes.usherb.ca> <396BD038.F60C4B43@geoserve.net> <396BD23F.FD850B34@hermes.usherb.ca> Message-ID: <396BD572.435B1D99@geoserve.net> Jean-Marc Valin wrote: > > "J.W. Bizzaro" a ?crit : > > Many nodes => smaller chunks > > Few nodes => bigger chunks > > Well, in this case, there is the same number of nodes. It's just that each node > receives 10000 chunks of data instead of 1. Yeah, I suppose you could break the data up however you want, regardless of the number of nodes. It may only take more time with less nodes. Thanks for the explanation :-) Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From jean-marc.valin at hermes.usherb.ca Tue Jul 11 22:28:14 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <396BC1E0.B369F8DB@hermes.usherb.ca> <396BC4A9.3E29274@geoserve.net> <396BCFE1.500415ED@hermes.usherb.ca> <396BD038.F60C4B43@geoserve.net> <396BD23F.FD850B34@hermes.usherb.ca> <396BD572.435B1D99@geoserve.net> Message-ID: <396BD7BE.B084782D@hermes.usherb.ca> > Yeah, I suppose you could break the data up however you want, regardless of > the number of nodes. It may only take more time with less nodes. Thanks for > the explanation :-) Well, I'm still not sure what you mean by "It may only take more time with less nodes". if I want to take the FFT of each frame, I only need one FFT block. I don't see how to do that with more than one FFT block, except by explicitly parallelizing. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From bizzaro at geoserve.net Wed Jul 12 00:42:39 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <396BC1E0.B369F8DB@hermes.usherb.ca> <396BC4A9.3E29274@geoserve.net> <396BCFE1.500415ED@hermes.usherb.ca> <396BD038.F60C4B43@geoserve.net> <396BD23F.FD850B34@hermes.usherb.ca> <396BD572.435B1D99@geoserve.net> <396BD7BE.B084782D@hermes.usherb.ca> Message-ID: <396BF73F.5C24652F@geoserve.net> Jean-Marc Valin wrote: > > Well, I'm still not sure what you mean by "It may only take more time with less > nodes". if I want to take the FFT of each frame, I only need one FFT block. I > don't see how to do that with more than one FFT block, except by explicitly > parallelizing. That's pretty much what I mean. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From bizzaro at geoserve.net Wed Jul 12 01:14:53 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB777.A92BD56@geoserve.net> <396BC1E0.B369F8DB@hermes.usherb.ca> <396BC4A9.3E29274@geoserve.net> <396BCFE1.500415ED@hermes.usherb.ca> <396BD038.F60C4B43@geoserve.net> <396BD23F.FD850B34@hermes.usherb.ca> <396BD572.435B1D99@geoserve.net> <396BD7BE.B084782D@hermes.usherb.ca> <396BF73F.5C24652F@geoserve.net> Message-ID: <396BFECD.7235E585@geoserve.net> "J.W. Bizzaro" wrote: > > Jean-Marc Valin wrote: > > > > Well, I'm still not sure what you mean by "It may only take more time with less > > nodes". if I want to take the FFT of each frame, I only need one FFT block. I > > don't see how to do that with more than one FFT block, except by explicitly > > parallelizing. > > That's pretty much what I mean. ......but I don't know how that would make Overflow appear to stream. You're right, breaking the data up is more important than adding nodes. Nevermind :-P Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ From chapmanb at arches.uga.edu Wed Jul 12 14:10:55 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML and Pise Message-ID: <200007121810.OAA180862@archa11.cc.uga.edu> Hi Raynald; Thanks for writing! Raynald wrote: > However, it seems that the Piper XML is not yet completely defined. I am > working in the Pasteur Institute and there is a software called Pise: > http://www-alt.pasteur.fr/~letondal/Pise/ written by Catherine Letondal > (letondal@pasteur.fr). I really like Pise a lot, and there was actually a bit of discussion on the Loci list about it back when it was first GPLed. I've spent a lot of time since then looking through the mounds of perl code that make it up :-). Raynald wrote: > In two words, this is an interface builder. It makes HTML pages and CGI > scripts for unix programmes related with Biology. My point is it uses XML to > describe the program to be interfaced. This program is under GPL and well > documented. I think it would be great if Piper and Pise could share the same > DTD... As Jeff mentioned, we do have a semi-permanent DTD working for Piper which is kind of a conglomeration between the old Loci XML format, and the way that Overflow does things. This is not yet permanent, but has been in place for a little while. That being said, there is still room for some flexibility (since I'm no stranger to rewriting code :-). A few months ago, I was really ready to just use the Pise DTD for Loci, actually. However, as I've got into Piper more and more I've realized that the Pise DTD isn't really meant for the kind of system Piper is trying to be. The big difference is that Pise is, as you mentioned, mostly concerned with being an interface generator. By contrast, Piper (in my mind) is less concerned with generating interfaces for specific programs, and a little more concerned with creating an interface to make connecting programs together more intutitive. The Piper DTD mainly focuses around three elements: Parameters, Inputs and Outputs. The Pise DTD is mainly focused on the Parameter part of this, which makes a lot of sense, since it is designed to create specific GUIs. Pise does have a pipe element, which provides some support for piping output from one program into another, but this seems less developed then Piper is trying to be. This right here seems to be the major sticking point, since Pise would have to add on quite a few changes here to deal with this. You mentioned the possibility of "some" modifications of the part of Pise, but I would imagine these would be quite small, considering that Pise is already in use in quite a few places. It seems like adding on whole another Input and Output parts might not be favorable, but I don't know... There could probably be some reconciliation on the parameter attribute on our part, but the thing I'm most worried about here is that Jeff has been wanting to use BlueBox to do the generation of user interfaces, so I don't know if the Pise model would fit with this, since we haven't seen BlueBox yet. Did you have any ideas how the Pise and Piper models could be combined, now that you've seen both DTDs? Jeff wrote: >> Also, Piper is meant to be general-purpose, not biology-related (/Loci/ was >> originally supposed to be biology-related). I see some Pise tags like >> "Sequence", which we wouldn't want to use. Well, this isn't a big concern to me, since we are not forced to use all of the elements, and specific nodes that wanted to use biology related terms could use them. I guess we would have a danger of developing some huge monsterous DTD with lots of domain specific information in it, which would be a bad thing. Raynald: > What do you think of that ? I think it is a really good idea, but am worried about the technical difficulties of doing it because of the differences in the goals of the two projects. Having read what I said, what do you think? Brad From bizzaro at geoserve.net Wed Jul 12 14:28:06 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:19 2006 Subject: [Pipet Devel] XML and Pise References: <200007121810.OAA180862@archa11.cc.uga.edu> Message-ID: <396CB8B6.CE04E3E3@geoserve.net> Brad Chapman wrote: > > The big > difference is that Pise is, as you mentioned, mostly concerned with > being an interface generator. By contrast, Piper (in my mind) is less > concerned with generating interfaces for specific programs, and a > little more concerned with creating an interface to make connecting > programs together more intutitive. > The Piper DTD mainly focuses around three elements: Parameters, > Inputs and Outputs. One important point here: It is via "connecting programs together" that an interface is generated. As Brad suggests, interface generation or, more correctly, "construction" is secondary to program or node connection. > There could probably be some reconciliation on the parameter > attribute on our part, but the thing I'm most worried about here is > that Jeff has been wanting to use BlueBox to do the generation of > user interfaces, :-P > so I don't know if the Pise model would fit with > this, since we haven't seen BlueBox yet. Nile wrote to me the other day and said that BlueBox code will be available this week. So, hang on to your hats. > Well, this isn't a big concern to me, since we are not forced to use > all of the elements, and specific nodes that wanted to use biology > related terms could use them. I guess we would have a danger of > developing some huge monsterous DTD with lots of domain specific > information in it, which would be a bad thing. I would rather not have unused elements in the DTD. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From chapmanb at arches.uga.edu Wed Jul 12 14:41:46 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] XML-RPC Message-ID: <200007121841.OAA130628@archa11.cc.uga.edu> Tony Rossini wrote: > I'm slowly going over the archive of the mailing list for this project > -- havn't seen XML-RPC mentioned, but lots of ties to projects where > it should've fit. > > Is this under consideration as a mechanism for constructing glue or > for serialization? Hi, I should butt in on this discussion since I'm probably the one who pushed for CORBA the most out of anyone. I should also preface this all by saying that I like CORBA *a lot* and so everything I say is probably baised by this :-) I've spent some time looking at XML-RPC and SOAP, and from all of my impressions, it seems like they are primarily designed to be used in web applications. This seems to be kind of reflected in the languages for which XML-RPC implementations are available: Java, Python, Perl, PHP, etc. By contrast, CORBA is designed for application usage. This is why I sort of immediately invision CORBA as being better suited for use for Piper. > No, it (speed) is quite important. I'd personally like to stream > megabytes of data (in a statistical, "flat-file"-ish sense) and > process it at the same time. I don't know if speed is really an issue many people know much about yet. There was a big argument on comp.object.corba about SOAP versus CORBA speed issues, and it seemed like most of the discussions was more opinions and not hard facts. XML-RPC and SOAP are relatively new (compared to CORBA), so it is difficult to say how well they will scale right now. But I guess I can't really say that XML based protocols are slower than IIOP. > The only rationale for XML-RPC is "simplicity". We'll keep it in > quotes, since I'm saying that out of context and it implies a set > decision-making criteria behind it. You are right, XML-RPC is probably simpler to start with, but as you mention, it is hard to say how it will keep this simplicity as an application grows. Although CORBA is big and harder to get into at first, I think it contains tons of hooks for just about every situtation, which might make it ultimately simpler over time. Hard to say, I guess. > Again, I'm not interested in baiting, just finding out what has been > explored/considered, and what the framework is. Yeah, I don't think the whole situation has been completely hashed out on the list. CORBA and XML based communication protocols seem to me to be "somewhat" interchangable. Probably personal preference has played the largest role in choosing CORBA for the communication protocol for Piper. I guess the largest voice against CORBA has been because of the learning curve to get started with it. However, I have found it really nice for development (very flexible to API changes etc), so once you are over the learning curve, it can be very nice. > We (one of my "affiliations") are looking into wet-lab > informatics/data processing, This is actually what I want to use Piper for myself, so it better be able to handle this :-). My grad research is all wet-lab based, and I would like Piper to help enhance the things I'll be doing there. The sooner it is implemented, the better it'll be for me as well! Brad From bizzaro at geoserve.net Wed Jul 12 14:48:41 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] XML and Pise References: <200007121810.OAA180862@archa11.cc.uga.edu> Message-ID: <396CBD89.9C870C25@geoserve.net> Brad Chapman wrote: > > That being said, there is still room for some flexibility (since > I'm no stranger to rewriting code :-). A few months ago, I was really > ready to just use the Pise DTD for Loci, actually. Before Brad came along, Gary and I were considering basing application wrapping on AppLab: http://industry.ebi.ac.uk/applab/ Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From rossini at biostat.washington.edu Wed Jul 12 15:00:57 2000 From: rossini at biostat.washington.edu (A.J. Rossini) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] XML-RPC In-Reply-To: Brad Chapman's message of "Wed, 12 Jul 2000 14:41:46 -0400" References: <200007121841.OAA130628@archa11.cc.uga.edu> Message-ID: <87vgybuqae.fsf@alpha.cfas.washington.edu> >>>>> "BC" == Brad Chapman writes: BC> Hi, I should butt in on this discussion since I'm probably the BC> one who pushed for CORBA the most out of anyone. I should also BC> preface this all by saying that I like CORBA *a lot* and so BC> everything I say is probably baised by this :-) Good to have that prior. BC> I've spent some time looking at XML-RPC and SOAP, and from all BC> of my impressions, it seems like they are primarily designed BC> to be used in web applications. This seems to be kind of BC> reflected in the languages for which XML-RPC implementations BC> are available: Java, Python, Perl, PHP, etc. By contrast, BC> CORBA is designed for application usage. This is why I sort of BC> immediately invision CORBA as being better suited for use for BC> Piper. Hmm... I think I agree with your conclusions, but not your assessment (i.e. I think considering CORBA is definitely the right approach, but I'm not sure if the logic you used is a good argument :-). BC> I don't know if speed is really an issue many people know much BC> about yet. There was a big argument on comp.object.corba about BC> SOAP versus CORBA speed issues, and it seemed like most of the BC> discussions was more opinions and not hard facts. XML-RPC and BC> SOAP are relatively new (compared to CORBA), so it is BC> difficult to say how well they will scale right now. But I BC> guess I can't really say that XML based protocols are slower BC> than IIOP. I think you probably can say that they are slower, just from a "non-compressed, plus parse-tree needed" point of view. BC> You are right, XML-RPC is probably simpler to start with, but BC> as you mention, it is hard to say how it will keep this BC> simplicity as an application grows. Although CORBA is big and BC> harder to get into at first, I think it contains tons of hooks BC> for just about every situtation, which might make it BC> ultimately simpler over time. Hard to say, I guess. Exactly. BC> Yeah, I don't think the whole situation has been completely BC> hashed out on the list. CORBA and XML based communication BC> protocols seem to me to be "somewhat" interchangable. They are. We (you) could provide an XML-RPC interface, without too much difficulty (probably would take time to think it through, though...). BC> personal preference has played the largest role in choosing BC> CORBA for the communication protocol for Piper. I guess the BC> largest voice against CORBA has been because of the learning BC> curve to get started with it. However, I have found it really BC> nice for development (very flexible to API changes etc), so BC> once you are over the learning curve, it can be very nice. I have to agree with this (all of the statements above). best, -tony -- A.J. Rossini Research Assistant Professor of Biostatistics Biostatistics/Univ. of Washington (Th) Box 357232 206-543-1044 (3286=fax) Center for AIDS Research/HMC/UW (M/F) Box 359931 206-731-3647 (3693=fax) VTN/SCHARP/FHCRC (Tu/W) Box 358080 206-667-7025 (4812=fax) rossini@(biostat.washington.edu|u.washington.edu|scharp.org) http://www.biostat.washington.edu/~rossini From rossini at biostat.washington.edu Wed Jul 12 15:50:13 2000 From: rossini at biostat.washington.edu (A.J. Rossini) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] XML-RPC In-Reply-To: Jarl van Katwijk's message of "Wed, 05 Jan 2000 21:19:25 -0900" References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB0A7.D22450E4@geoserve.net> <387433ED.735C2882@casema.net> Message-ID: <87puoj3z7u.fsf@alpha.cfas.washington.edu> JvK> Just a small note on 'corba & security' : I didn't say corba JvK> is secure, I said we need a security mechanism, implemented JvK> by corba. But the actual security has nothing to do with JvK> corba. Corba just gives the option to do thing right. That is correct. There are ways to achieve comparable methods (SSL, etc) with the other approaches, however they aren't (and there aren't) "standards". I believe there is a freely usable CORBA-security implementation; I'll have to look up my mail archives for the exact location. (currently, I've done most of my work/experiments with Java/ORBacus, but have looked at Orbit/PyOrbit for various reasons). best, -tony -- A.J. Rossini Research Assistant Professor of Biostatistics Biostatistics/Univ. of Washington (Th) Box 357232 206-543-1044 (3286=fax) Center for AIDS Research/HMC/UW (M/F) Box 359931 206-731-3647 (3693=fax) VTN/SCHARP/FHCRC (Tu/W) Box 358080 206-667-7025 (4812=fax) rossini@(biostat.washington.edu|u.washington.edu|scharp.org) http://www.biostat.washington.edu/~rossini From bizzaro at geoserve.net Wed Jul 12 15:56:53 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] XML-RPC References: <87ituce4n1.fsf@alpha.cfas.washington.edu> <396B9D4D.38B8458F@geoserve.net> <878zv8e0mk.fsf@alpha.cfas.washington.edu> <396BB0A7.D22450E4@geoserve.net> <387433ED.735C2882@casema.net> <87puoj3z7u.fsf@alpha.cfas.washington.edu> Message-ID: <396CCD85.C18A047B@geoserve.net> "A.J. Rossini" wrote: > > (currently, I've done most of my work/experiments with Java/ORBacus, > but have looked at Orbit/PyOrbit for various reasons). ^^^^^^^ Developed here, although it seems to have died. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From bizzaro at geoserve.net Wed Jul 12 16:44:47 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] NOTICE: Server Switchover on Saturday, July 15 Message-ID: <396CD8BF.9DA679CD@geoserve.net> Bioinformatics.org: The Open Lab will be switching its primary addresses... bioinformatics.org bio-informatics.org theopenlab.org theopenlab.net www.bioinformatics.org www.bio-informatics.org www.theopenlab.org www.theopenlab.net bioinformatics.org ...to a new computer system THIS Saturday, July 15. If you have any work to do with the current computer, PLEASE do NOT do it this Saturday! And, PLEASE do NOT send any messages to the mailing lists on that day! Hopefully, by Monday, everything will be functioning normally, with the exception of some new software and features that will be announced that day. Thank you. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From bizzaro at geoserve.net Thu Jul 13 12:35:15 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] man to man^H^H^Hnode parameters Message-ID: <396DEFC3.A7850A00@geoserve.net> I just wanted to re-introduce Brad's very cool idea of parsing Unix man pages for the purpose of automatically determining the parameters to be used with command-line programs imported to run under Piper (as nodes). We should develop a parser that can grab both the OPTIONS and their descriptions and XML-ize them. Descriptions can then be used by the UI as tooltips or the equivalent. Here's an excerpt from the man page for ls: -a, --all do not hide entries starting with . -A, --almost-all do not list implied . and .. -b, --escape print octal escapes for nongraphic characters --block-size=SIZE use SIZE-byte blocks -B, --ignore-backups do not list implied entries ending with ~ -c with -lt: sort by, and show, ctime (time of last modification of file status information) with -l: show ctime and sort by name otherwise: sort by ctime A few weeks ago, I described Piper to a comp sci prof at UMass. This idea actually brought a smile to his face. He didn't say anything about it. He just smiled :-) This also goes along with our plan to make Piper as Unix-centric as possible. By this, I mean to make use of all the excellent ideas that have been developed over the decades and, more importantly, all the thousands of tools in existence. It will give Piper an ENORMOUS head-start over systems like .NET. Why should Unix be Windows-ized or Mac-ized?! I want to fight this trend, not just for the sake of fighting, but because the ideas behind these OSes are not necessarily superior to those behind Unix. I wonder if Piper will actually breath new life into the development of traditional command-line programs :-) Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jean-marc.valin at hermes.usherb.ca Thu Jul 13 12:48:21 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] man to man^H^H^Hnode parameters References: <396DEFC3.A7850A00@geoserve.net> Message-ID: <396DF2D5.277064EB@hermes.usherb.ca> "J.W. Bizzaro" a ?crit : > > I just wanted to re-introduce Brad's very cool idea of parsing Unix man pages > for the purpose of automatically determining the parameters to be used with > command-line programs imported to run under Piper (as nodes). > > We should develop a parser that can grab both the OPTIONS and their > descriptions and XML-ize them. Descriptions can then be used by the UI as > tooltips or the equivalent. Just a thought: why not validata these by searching the getopt calls in the actual source code (it all OSS isn't it?). Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From bizzaro at geoserve.net Thu Jul 13 13:17:14 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:20 2006 Subject: [Pipet Devel] man to man^H^H^Hnode parameters References: <396DEFC3.A7850A00@geoserve.net> <396DF2D5.277064EB@hermes.usherb.ca> Message-ID: <396DF99A.42C8439F@geoserve.net> Jean-Marc Valin wrote: > > Just a thought: why not validata these by searching the getopt calls in the > actual source code (it all OSS isn't it?). That's a good point. The problems are (1) the program has to be OSS, (2) the source code has to be on the computer (not everyone bothers to get the source), (3) the source code has to be in a place where Piper can find it (unless the user specifies the location), and (4) the program has to be in a language that uses getopt (C/C++). I suppose that's why you said getopt would be for validation and not the sole source of this information. On the other hand, not every program has a man page, but it's still more likely to be present, and more easily located, than the source code. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From bizzaro at geoserve.net Thu Jul 13 13:33:44 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] man to man^H^H^Hnode parameters References: <396DEFC3.A7850A00@geoserve.net> Message-ID: <396DFD78.4CFEECAE@geoserve.net> "J.W. Bizzaro" wrote: > > I just wanted to re-introduce Brad's very cool idea of parsing Unix man pages > for the purpose of automatically determining the parameters to be used with > command-line programs imported to run under Piper (as nodes). I imagine that when Piper is started on a system, it finds all of the programs in $PATH. It then proceeds to find the man pages and generate the XML that defines the parameters. Why not do this at compile time? Two reasons: (1) The programs available should be specific to the $PATH of the user running Piper and to the permissions that the user has, and (2) if additional programs are added to the computer, Piper needs to import them without requiring a rebuild (imagine having to rebuild Piper everytime you put a new program on your computer!); Piper should just be restarted. So, everytime Piper starts up, it should do a quick comparison between what programs are in $PATH and what programs it last recorded being in $PATH. I think this approach would be most convenient for the user, certainly better than an interactive program-by-program approach. What do you think? Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From bizzaro at geoserve.net Thu Jul 13 14:17:13 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] more info on WebSphere Message-ID: <396E07A9.E18CB0D8@geoserve.net> ``[IBM] WebSphere Transcoding Publisher is a tool that is considered "protocol agnostic," meaning that this technology will soon allow any Web content to be easily ported on any "display" device.... Meantime, IBM is now keen on promoting "open standards" and "open source" computing as it declared that WebSphere will run on any platform, or that is at least 30 hardware platforms, and a list of computing environments such as Windows 2000, Linux, HP-UX, among others. "IBM is particular about not building a suite of Web applications but components that can be modular and can scale up," said Dutton.'' http://www.newsalert.com/bin/story?StoryId=CowVTqc4bmdaWnd I consider WebSphere, along with M$.NET, JavaSpaces, E-Speak, etc. to be similar enough to Piper to be considered "commercial competitors". A big difference seems to be the reliance of these systems on the web browser, perhaps a mistake. Also, looking at the bunch we have on this list, I'd say Piper will be a bit more tuned toward scientific and technical computing than e-commerce and personal productivity ;-) Cheers. Jeff From bizzaro at geoserve.net Thu Jul 13 15:21:18 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] man to man^H^H^Hnode parameters References: <396DEFC3.A7850A00@geoserve.net> <396DFD78.4CFEECAE@geoserve.net> Message-ID: <396E16AE.F7AC6B4E@geoserve.net> "J.W. Bizzaro" wrote: > > I imagine that when Piper is started on a system, it finds all of the programs > in $PATH. It then proceeds to find the man pages and generate the XML that > defines the parameters. That first sentence should read... I imagine that when a /new session/ is started in Piper, Piper finds all of the programs in the /user's/ $PATH. This will be the responsibility of the DL. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jean-marc.valin at hermes.usherb.ca Thu Jul 13 15:32:33 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] man to man^H^H^Hnode parameters References: <396DEFC3.A7850A00@geoserve.net> Message-ID: <396E1951.4CFB1676@hermes.usherb.ca> "J.W. Bizzaro" a ?crit : > > I just wanted to re-introduce Brad's very cool idea of parsing Unix man pages > for the purpose of automatically determining the parameters to be used with > command-line programs imported to run under Piper (as nodes). > > We should develop a parser that can grab both the OPTIONS and their > descriptions and XML-ize them. Descriptions can then be used by the UI as > tooltips or the equivalent. This actually gives me an idea for Overflow: parse all the headers in /usr/include, and automatically generate Overflow Nodes for all the functions (and maybe Overflow types for all teh structs). I'm not yet sure how to make that work for all cases (especially when values are returned from pointers), but for functions like 'sin', 'cos', ... I think it wouldn't be hard. For those who have tried VisualWorks (Smalltalk), this could be similar to the DLL/C Connect process. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From bizzaro at geoserve.net Thu Jul 13 16:07:06 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] man to man^H^H^Hnode parameters References: <396DEFC3.A7850A00@geoserve.net> <396E1951.4CFB1676@hermes.usherb.ca> Message-ID: <396E216A.80D51806@geoserve.net> Jean-Marc Valin wrote: > > This actually gives me an idea for Overflow: > parse all the headers in /usr/include, and automatically generate Overflow Nodes > for all the functions (and maybe Overflow types for all teh structs). I'm not > yet sure how to make that work for all cases (especially when values are > returned from pointers), but for functions like 'sin', 'cos', ... I think it > wouldn't be hard. For those who have tried VisualWorks (Smalltalk), this could > be similar to the DLL/C Connect process. Wow, what a super idea if it can be implemented! Keep us updated on your plans and progress. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From bizzaro at geoserve.net Thu Jul 13 18:29:55 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] [Fwd: BOSC talk] Message-ID: <396E42E3.A9D469DE@geoserve.net> -------------- next part -------------- An embedded message was scrubbed... From: Nomi Harris Subject: BOSC talk Date: Thu, 13 Jul 2000 15:13:51 -0700 (PDT) Size: 2182 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000713/c5fad087/attachment.mht From bizzaro at geoserve.net Thu Jul 13 18:50:37 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] [Fwd: BOSC talk] References: <396E42E3.A9D469DE@geoserve.net> Message-ID: <396E47BD.5506CCBA@geoserve.net> "J.W. Bizzaro" wrote: > > You will have 20 minutes for your talk, with > additional time allotted for questions. Hey, I've got to describe Piper /and/ The Open Lab in 20 minutes! :-P That's about enough time to read the abstracts :-P Every talk I've given since being a grad student has taken at least an hour. Whoo-hoo for me :-P Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From gvd at penguin.pharmacy.ualberta.ca Thu Jul 13 19:12:19 2000 From: gvd at penguin.pharmacy.ualberta.ca (Gary Van Domselaar) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] [Fwd: BOSC talk] In-Reply-To: <396E47BD.5506CCBA@geoserve.net> Message-ID: On Thu, 13 Jul 2000, J.W. Bizzaro wrote: > "J.W. Bizzaro" wrote: > > > > You will have 20 minutes for your talk, with > > additional time allotted for questions. > > Hey, I've got to describe Piper /and/ The Open Lab in 20 minutes! :-P That's > about enough time to read the abstracts :-P Every talk I've given since being > a grad student has taken at least an hour. Whoo-hoo for me :-P Wow. I cant see you even giving an eagle's eye view of the open lab in under 18 minutes, so good luck packing piper into two ;-) g. -- Gary Van Domselaar gary@bioinformatics.org http://www.bioinformatics.org/~gary ---------------------------------------------------- bioinformatics.org: The Open Lab http://www.bioinformatics.org/ From bizzaro at geoserve.net Thu Jul 13 23:40:58 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] python-xml-nodom Message-ID: <396E8BCA.95BBE176@geoserve.net> >From INSTALL: Earlier versions will not work because of a change in where you import things from. Be careful, because now 4DOM wants to go in xml.dom, but pyDOM (that comes with the pyXML package) is also in xml.dom, so clean out the old xml/dom with an 'rm -rf' before installing 4DOM! Note that there is a python-xml-nodom-0.5.5-1.i386.rpm (pyXML without DOM) on the 4DOM site. You may not like RPMS, but perhaps we can mention this in INSTALL. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jean-marc.valin at hermes.usherb.ca Fri Jul 14 15:34:50 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] Overflow 0.2.0 released Message-ID: <396F6B5A.AF665FF2@hermes.usherb.ca> This is the moment you've been waiting for... I jsut released version 0.2.0 of Overflow. You can download from http://download.sourceforge.net/freespeech/ Along with *many* bug fixes, it is now easier to build Piper with it. I have provided linux (x86) binaries. Brad, can cou build FreeBSD binaries? (donwnload the source, configure with --disable-static and tar the install directory). Have fun! Jean-Marc P.S. Don't forget to use gcc 2.95 to compile and have fftw compiled with --enable-float -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jean-marc.valin at hermes.usherb.ca Fri Jul 14 20:43:32 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] Overflow XML generator Message-ID: <396FB3B4.464ABDBB@hermes.usherb.ca> Hi, I just added to Overflow a script that generates XML node definitions when installing. The XML files are split in different toolboxes. The change is in CVS, but NOT in the 0.2.0 release. The next step is to change the overflow lib to use that info instead of the current "NODE_INFO" macro. Brad, I think we should handle that together. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From bizzaro at geoserve.net Fri Jul 14 21:41:41 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] REMINDER: Server Down Time Saturday Message-ID: <396FC155.B8487F21@geoserve.net> Just a reminder, we will be switching servers during the afternoon Saturday. Please do not login, use CVS, the mailing lists, etc. during that time. I will send an announcement when we are done. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From bizzaro at geoserve.net Sun Jul 16 02:52:41 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] cvs Message-ID: <39715BB9.FD83FF4B@geoserve.net> It should be safe to use CVS on the server now. Be sure to use www.bioinformatics.org as the address for the piper module. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From bizzaro at geoserve.net Mon Jul 17 17:41:46 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] Developing Gnome Application with Python (Part 1) Message-ID: <39737D9A.9BF49A06@geoserve.net> http://linuxfocus.org/English/July2000/article160.shtml Jeff From bizzaro at geoserve.net Sat Jul 22 20:24:27 2000 From: bizzaro at geoserve.net (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] piper website in cvs Message-ID: <397A3B3B.91BF1129@geoserve.net> Greetings. I made a CVS module containing the Piper website. It is aptly named... piper-website Feel free to make changes and contribute. (I can hear Brad now: "Hey Jeff, how about contributing to Pied!" ;-) Very soon now.) Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From letondal at pasteur.fr Mon Jul 24 13:38:31 2000 From: letondal at pasteur.fr (Catherine Letondal) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] XML and Pise In-Reply-To: Your message of "Wed, 12 Jul 2000 14:10:55 EDT." <200007121810.OAA180862@archa11.cc.uga.edu> Message-ID: <200007241738.e6OHcVJ27727@nefertiti.pasteur.fr> Brad Chapman writes: > Hi Raynald; > Thanks for writing! > > Raynald wrote: > > However, it seems that the Piper XML is not yet completely defined. > I am > > working in the Pasteur Institute and there is a software called Pise: > > http://www-alt.pasteur.fr/~letondal/Pise/ written by Catherine > Letondal > > (letondal@pasteur.fr). > > I really like Pise a lot, and there was actually a bit of discussion > on the Loci list about it back when it was first GPLed. I've spent a > lot of time since then looking through the mounds of perl code that > make it up :-). > > Raynald wrote: > > In two words, this is an interface builder. It makes HTML pages and > CGI > > scripts for unix programmes related with Biology. My point is it > uses XML to > > describe the program to be interfaced. This program is under GPL and > well > > documented. I think it would be great if Piper and Pise could share > the same > > DTD... > > As Jeff mentioned, we do have a semi-permanent DTD working for Piper > which is kind of a conglomeration between the old Loci XML format, and > the way that Overflow does things. This is not yet permanent, but has > been in place for a little while. > That being said, there is still room for some flexibility (since > I'm no stranger to rewriting code :-). A few months ago, I was really > ready to just use the Pise DTD for Loci, actually. However, as I've > got into Piper more and more I've realized that the Pise DTD isn't > really meant for the kind of system Piper is trying to be. The big > difference is that Pise is, as you mentioned, mostly concerned with > being an interface generator. That's right. > By contrast, Piper (in my mind) is less > concerned with generating interfaces for specific programs, and a > little more concerned with creating an interface to make connecting > programs together more intutitive. > The Piper DTD mainly focuses around three elements: Parameters, > Inputs and Outputs. The Pise DTD is mainly focused on the Parameter > part of this, which makes a lot of sense, since it is designed to > create specific GUIs. Pise does have a pipe element, which provides > some support for piping output from one program into another, but this > seems less developed then Piper is trying to be. This right here seems > to be the major sticking point, since Pise would have to add on quite > a few changes here to deal with this. You mentioned the possibility of > "some" modifications of the part of Pise, but I would imagine these > would be quite small, considering that Pise is already in use in quite > a few places. It seems like adding on whole another Input and Output > parts might not be favorable, but I don't know... The pipe element in Pise is understood as an Input for InFile/Sequence parameters and as an Output for OutFile/Results parameters, so it would be easy to add an information, maybe as an attribute of the pipe element? I don't know Piper enough for now, but I think the main problem would be for other kinds of parameters (Integer, List, ...) which are not connected at all in Pise. There is a student project in Bielefeld to extend the dataflow model of Pise this way. > There could probably be some reconciliation on the parameter > attribute on our part, but the thing I'm most worried about here is > that Jeff has been wanting to use BlueBox to do the generation of > user interfaces, so I don't know if the Pise model would fit with > this, since we haven't seen BlueBox yet. Did you have any ideas how > the Pise and Piper models could be combined, now that you've seen both > DTDs? > > Jeff wrote: > >> Also, Piper is meant to be general-purpose, not biology-related > (/Loci/ was > >> originally supposed to be biology-related). I see some Pise tags > like > >> "Sequence", which we wouldn't want to use. > > Well, this isn't a big concern to me, since we are not forced to use > all of the elements, and specific nodes that wanted to use biology > related terms could use them. I guess we would have a danger of > developing some huge monsterous DTD with lots of domain specific > information in it, which would be a bad thing. > > Raynald: > > What do you think of that ? > > I think it is a really good idea, but am worried about the > technical difficulties of doing it because of the differences in the > goals of the two projects. Having read what I said, what do you think? > > Brad > > > _______________________________________________ > pipet-devel maillist - pipet-devel@bioinformatics.org > http://bioinformatics.org/mailman/listinfo/pipet-devel -- Catherine Letondal -- Pasteur Institute Computing Center From jeff at bioinformatics.org Mon Jul 24 18:19:27 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] [Fwd: Important information about your poster at ISMB 2000] Message-ID: <397CC0EF.E15C12D8@bioinformatics.org> The Piper poster will be displayed Monday, August 21 at ISMB. You can find the short abstract here: http://ismb2000.sdsc.edu/posters/poster-list.html#B Jeff -------------- next part -------------- An embedded message was scrubbed... From: Ann Redelfs Subject: Important information about your poster at ISMB 2000 Date: Sun, 23 Jul 2000 08:43:39 -0800 Size: 4106 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000724/61f08eba/attachment.mht From jeff at bioinformatics.org Mon Jul 24 22:56:53 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] [Fwd: [Bosc-attendees] BOSC'2000 Schedule & Speakers (v1.0)] Message-ID: <397D01F5.B2A676FA@bioinformatics.org> I have to describe Piper and The Open Lab in 20 minutes /AND/ follow Tim O'Reilly (O'Reilly & Associates publishers). No pressure. Jeff -------------- next part -------------- An embedded message was scrubbed... From: bosc-attendees-admin@bubbles.sonsorol.org Subject: [Bosc-attendees] BOSC'2000 Schedule & Speakers (v1.0) Date: Mon, 24 Jul 2000 21:47:27 -0400 Size: 4458 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/20000725/69c2dc22/attachment.mht From jeff at bioinformatics.org Tue Jul 25 08:53:00 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] BlueBox source code Message-ID: <397D8DAC.89342719@bioinformatics.org> The source code for BlueBox, showing the architecture and including some GUI widgets, is available here: http://www.dloo.com/bluebox/download.html Pipers, PLEASE download it, look it over, and post comments. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Tue Jul 25 08:54:51 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] BlueBox source code References: <397D8DAC.89342719@bioinformatics.org> Message-ID: <397D8E1B.8C0A83ED@bioinformatics.org> "J.W. Bizzaro" wrote: > > The source code for BlueBox, showing the architecture and including some GUI > widgets, is available here: > > http://www.dloo.com/bluebox/download.html > > Pipers, PLEASE download it, look it over, and post comments. Okay, except for the fact that the server is refusing connections :-P Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From chapmanb at arches.uga.edu Tue Jul 25 09:13:30 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] BlueBox source code Message-ID: <200007251313.JAA32980@archa13.cc.uga.edu> Jeff wrote: >> >> The source code for BlueBox, showing the architecture and including some > GUI >> widgets, is available here: >> >> http://www.dloo.com/bluebox/download.html >> >> Pipers, PLEASE download it, look it over, and post comments. Okay :-). I grabbed this over the weekend, but never finished installing it. The install is pretty vicious right now, with hard-coded libraries and install directories buried in the Makefile.am's. Some of the problems could be fixed using *-config files (I don't anything about wxWindows to know how to not have to hardcode their stuff) or just from hacking the Makefile.am's, of course. Howerve, my impression was that things are changing really drastically, making me pretty unexcited to get this fixed up just to have the next source release be completely different :-). Also, right now BlueBox uses libxml2, while Overflow uses libxml. These are incompatible, so be sure to keep 'em separate. This caught me for a while since I forgot I installed libxml2 for BlueBox, and started getting all kinds of problems in Overflow. Of course, probably everyone else isn't as slow as I am... > Okay, except for the fact that the server is refusing connections :-P Use an ftp client (and not your browser) and you shouldn't have any problems. Brad From jeff at bioinformatics.org Tue Jul 25 09:38:14 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] BlueBox source code References: <200007251313.JAA32980@archa13.cc.uga.edu> Message-ID: <397D9846.697F2E65@bioinformatics.org> Brad Chapman wrote: > > I grabbed this over the weekend, but never finished installing it. Brad, you are ALWAYS one step ahead of me :-) ...which doesn't say much :-P Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Wed Jul 26 07:26:03 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:21 2006 Subject: [Pipet Devel] Text Processing Pipelines Message-ID: <397ECACB.25DDE0FA@bioinformatics.org> ``Sure the command line is evil, but mastering it will unlock the powers of a Unix box that remain unrealized under modern graphical user interfaces. This article details the construction of text processing pipelines, using ordinary GNU utilities, to accomplish a few fairly challenging tasks.'' http://www.linuxnewbie.org/nhf/intel/programming/text_processing_pipes.html Jeff From chapmanb at arches.uga.edu Wed Jul 26 17:13:18 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs Message-ID: <200007262113.RAA73684@archa10.cc.uga.edu> Hey all; I've been doing a lot of thinking about how the Network/Composite loci deal with getting inputs and outputs. To summarize the situation right now (based on the last discussion we had on this): Whenever a new node is added inside of a Composite Locus (or Network, for Overflow) all of the unconnected inputs and outputs are "inherited" by the parent Composite Locus, so that they can potentially be connected with other loci in the same level as the parent Composite Locus. This is the situation of things right now if you grab the latest Piper from CVS. Now that I've got this way implemented, however, I really don't like it :-<. There are a few problems I see with it: 1. The composite loci end up with a ton of NetInputs and NetOutputs (to use Overflow terminology), and I think this will make it really tough to see what NetInputs and NetOutputs go with which internal Loci, even when viewing the names of inputs and outputs is implimented. 2. Having been thinking more about it, it seems like this behavior isn't really what I would want as a default. It seems like that 90% of the time when I add a new locus I'm planning on connecting it to other loci in the same network. So, based on this we are doing a lot of extra work with adding and removing inputs for maybe 10% of the cases when an input or output will feed to the outside of a network. In addition, this behavior may be confusing to a user, who wouldn't expect to see inputs appearing and dissappearing randomly. 3. The situation gets messed up when you have a composite inside of another composite. Then if you add a new locus to the inner composite, the new inputs are inherited into that composite, and then since they aren't connected there, are then inherited again into the outer composite. 4. All of the above stuff makes this kind of a pain to maintain, and also slows things down quite a bit. What I'd like to propose is that we change this behavior to be similar to that used in Overflow. A user would have to click on an input or output and then specifically request that it be inherited to the parent locus. The user would then also supply a name for it (like in Overflow) and then this name would help associate the NetInput with the internal locus connector to which it is associated. What do people think about this? Should I do it? Better suggestions? Defenders of the current way? Brad From jeff at bioinformatics.org Thu Jul 27 07:06:37 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs References: <200007262113.RAA73684@archa10.cc.uga.edu> Message-ID: <398017BD.6CE45785@bioinformatics.org> Brad Chapman wrote: > > Whenever a new node is added inside of a Composite Locus (or Network, > for Overflow) all of the unconnected inputs and outputs are > "inherited" by the parent Composite Locus, so that they can > potentially be connected with other loci in the same level as the > parent Composite Locus. This is the situation of things right now if > you grab the latest Piper from CVS. The idea that I had was that the I/O's (links) you DIDN'T want mapped to the parent could be explicitly HIDDEN. This is the opposite of what you propose below (where you bring up some good points), because everything gets mapped by default. I just wanted to mention this, because it is not as though the user could not (when implemented) "unmap" a link. > 1. The composite loci end up with a ton of NetInputs and NetOutputs > (to use Overflow terminology), and I think this will make it really > tough to see what NetInputs and NetOutputs go with which internal > Loci, even when viewing the names of inputs and outputs is implimented. When building a large subnet, I can see how you might end up with dozens of links mapped to the parent, and that could get ugly. > 2. Having been thinking more about it, it seems like this behavior > isn't really what I would want as a default. It seems like that 90% of > the time when I add a new locus I'm planning on connecting it to other > loci in the same network. So, based on this we are doing a lot of > extra work with adding and removing inputs for maybe 10% of the cases > when an input or output will feed to the outside of a network. In > addition, this behavior may be confusing to a user, who wouldn't > expect to see inputs appearing and dissappearing randomly. Good point. > 3. The situation gets messed up when you have a composite inside of > another composite. Then if you add a new locus to the inner composite, > the new inputs are inherited into that composite, and then since they > aren't connected there, are then inherited again into the outer > composite. That's an excellent point and something I didn't consider. > 4. All of the above stuff makes this kind of a pain to maintain, and > also slows things down quite a bit. I can see how slow it is. And we do want to keep communication to a minimum when working through an object broker can make things this slow. > What I'd like to propose is that we change this behavior to be similar > to that used in Overflow. A user would have to click on an input or > output and then specifically request that it be inherited to the > parent locus. Great. I say we go ahead and do that, and we can nix the option to hide a link (explicitly unmapping a link). I think what we can do is place a "P" in the dot, indicating that communication is from/to Parent. In addition, the dot can turn green when its mapped counterpart on the parent is also green. Can the DL send that information (whether the link on the parent is connected or not) to all the child nodes? > The user would then also supply a name for it (like in > Overflow) and then this name would help associate the NetInput with > the internal locus connector to which it is associated. Shouldn't links have names by default? And then the user can change it if they want. BTW, I'm impressed with how far you've gone toward adding Overflow features to Pied. You guys have to check this out! I will be taking the job of developing Pied over from Brad now, which will give him more time for other things. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Thu Jul 27 07:12:43 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs References: <200007262113.RAA73684@archa10.cc.uga.edu> <398017BD.6CE45785@bioinformatics.org> Message-ID: <3980192B.6E241AA2@bioinformatics.org> "J.W. Bizzaro" wrote: > > Can the DL send that information (whether the link on the parent is connected > or not) to all the child nodes? Not literally to the nodes. The UI will get information from the DL about the state of a link, correct? BTW, all of this should not make any real difference to Peep. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Thu Jul 27 12:21:19 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] name changes in pied Message-ID: <3980617F.FD6DCF9@bioinformatics.org> As discussed, I'm making the following changes to names in the Pied UI to match names in Overflow: Loci Name Overflow Name ========= ============= Workspace ---> Network Locus ---> Node Loci ---> Nodes (or Pied, depending on meaning) Connector ---> Link Composite ---> Subnet Brad, I hope the use of "node" does not cause any problems. You mentioned the conflict with DOM, but I'm assuming Pied is far enough removed. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From chapmanb at arches.uga.edu Thu Jul 27 12:28:36 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs Message-ID: <200007271628.MAA115106@archa12.cc.uga.edu> Jeff wrote: > The idea that I had was that the I/O's (links) you DIDN'T want mapped to the > parent could be explicitly HIDDEN. This is the opposite of what you propose > below (where you bring up some good points), because everything gets mapped > by > default. I just wanted to mention this, because it is not as though the user > could not (when implemented) "unmap" a link. Yup, I'm with you completely on your ideas. I thought they were plenty snazzy when you wrote about 'em (which is why I spend the time implementing them!), and it really took seeing them in action to find the problems that I noted. You know the paradigm -> build one to throw away (well, in our case, build 12 to throw away :-). [...my proposal for making NetInput/Outputs work like Overflow...] > Great. I say we go ahead and do that, and we can nix the option to hide a > link (explicitly unmapping a link). Sounds good, I'll do it then, unless anyone else raises an objection soon. > I think what we can do is place a "P" in > the dot, indicating that communication is from/to Parent. In addition, the > dot can turn green when its mapped counterpart on the parent is also green. Sounds good! > Can the DL send that information (whether the link on the parent is connected > or not) to all the child nodes? > Not literally to the nodes. The UI will get information from the DL about the > state of a link, correct? Right now I give connection information (telling the UI when to connect things) through the following callback function: void showConnection(in Locus inLocus, in Connector inConnector, in Locus outLocus, in Connector outConnector); (there is an equivalent "hideConnection"). I guess for this case what we should do is just pass a single Locus and Connector to "be connected" (and pass the other set as NULL). Since I can't really think of another case where this will happen, the UI can interpret this as being due to a "Parent" connector being mapped. Sound okay? >> The user would then also supply a name for it (like in >> Overflow) and then this name would help associate the NetInput with >> the internal locus connector to which it is associated. > > Shouldn't links have names by default? And then the user can change it if > they want. Okay, but I'm not sure what the "default" name should be so that I'll be easily recognizable. What I've been doing internally is munging together the "name" of the locus inside of a composite (ie. Constant3) with the name of the link (so ending up with something like Constant3--VALUE as the name in the parent). But this "Constant3" info isn't available to the user in the ui (all they see is constant) so I don't think this name'll be very meaningful. How do you want to handle the default? Brad From chapmanb at arches.uga.edu Thu Jul 27 12:37:27 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] name changes in pied Message-ID: <200007271637.MAA51740@archa13.cc.uga.edu> Jeff wrote: > As discussed, I'm making the following changes to names in the Pied UI to > match names in Overflow: > > Connector ---> Link Actually, connector is more analagous to "Terminal" in Overflow (I like connector better, tho :-). In Overflow, as in Piper, a Link is an actual connection between two connectors (or terminii). Jeff wrote: > Brad, I hope the use of "node" does not cause any problems. You mentioned > the > conflict with DOM, but I'm assuming Pied is far enough removed. Well, I've given up my "no Node" crusade and resigned myself to the fact that people like it :-). So feel free to change away! Brad From jeff at bioinformatics.org Thu Jul 27 13:05:13 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] name changes in pied References: <200007271637.MAA51740@archa13.cc.uga.edu> Message-ID: <39806BC9.42903379@bioinformatics.org> Brad Chapman wrote: > > > Connector ---> Link > > Actually, connector is more analagous to "Terminal" in Overflow (I > like connector better, tho :-). In Overflow, as in Piper, a Link is an > actual connection between two connectors (or terminii). Hmmmmmmmm. In Overflow, the little dots on the edges of the nodes are "terminals" then? Terminal is a better name for the way it appears in Overflow. In Peep, "Link" may actually be a better name, since it is something like a symlink on the Unix command-line. "Terminal" and "Connector" may actually be confusing terms in Peep. I don't know, I just want both UI's to have names that match those used in the DL and other parts of Piper. You tell me the best name. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Thu Jul 27 13:15:06 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs References: <200007271628.MAA115106@archa12.cc.uga.edu> Message-ID: <39806E1A.2E69DFCE@bioinformatics.org> Brad Chapman wrote: > > > I think what we can do is place a "P" in > > the dot, indicating that communication is from/to Parent. In > > addition, the dot can turn green when its mapped counterpart on > > the parent is also green. > > Sounds good! I'm thinking now about a small black dot inside of the green dot, which can be interpreted as a line coming up out of the main dot and into the parent. > void showConnection(in Locus inLocus, in Connector inConnector, > in Locus outLocus, in Connector outConnector); > > (there is an equivalent "hideConnection"). I guess for this case what > we should do is just pass a single Locus and Connector to "be > connected" (and pass the other set as NULL). Since I can't really > think of another case where this will happen, the UI can interpret > this as being due to a "Parent" connector being mapped. Sound okay? Yeah, we can give that a try. > > Shouldn't links have names by default? And then the user can change > > it if they want. > > Okay, but I'm not sure what the "default" name should be so that I'll > be easily recognizable. What I've been doing internally is munging > together the "name" of the locus inside of a composite (ie. Constant3) > with the name of the link (so ending up with something like > Constant3--VALUE as the name in the parent). But this "Constant3" info > isn't available to the user in the ui (all they see is constant) so I > don't think this name'll be very meaningful. How do you want to handle > the default? Well, Piper should know what /type/ of information each connector relays, or what the connection represents, so how about the default name simply being the connector type? Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From chapmanb at arches.uga.edu Fri Jul 28 09:40:16 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] name changes in pied Message-ID: <200007281340.JAA46952@archa11.cc.uga.edu> > Hmmmmmmmm. In Overflow, the little dots on the edges of the nodes are > "terminals" then? Terminal is a better name for the way it appears in > Overflow. In Peep, "Link" may actually be a better name, since it is > something like a symlink on the Unix command-line. I think "Link" is confusing because a link is a connection between two different connectors (or terminii). If we are deciding on naming, what I'd like to have is: Connector -> One part of a thing that will make up a link (ie. inputs and outputs of a node). So this is a generic name to refer to Inputs and Outputs. Link -> The connection between two connectors. > "Terminal" and > "Connector" > may actually be confusing terms in Peep. Well, even in Peep nodes have inputs and output. You might not explicitly see them (ie. you only see links or pipes or whatever, as in the command line) but they are there. We need a way to refer to the things that make up the connections, and this should be separate from the name of a connection. Does this make any sense? What do you all think? Brad From chapmanb at arches.uga.edu Fri Jul 28 09:46:42 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs Message-ID: <200007281346.JAA67536@archa13.cc.uga.edu> [...what to name connectors that get mapped to the parent locus...] > Well, Piper should know what /type/ of information each connector relays, or > what the connection represents, so how about the default name simply being > the > connector type? Well, the name of a connector has to be unique (this is the id of each connector) so just the type wouldn't work properly, since if you had two "Constant" node types in a network, and then mapped both of their "VALUE" connectors to the parent locus, you would get a clash. As I mentioned I have been munging together the type (ie. VALUE) with a sort of unique identifier (ie. Constant1) to get something like 'Constant1--VALUE' for a name. This works okay now, but I guess I just worry how easy it will be to maintain. This is also not very helpful to a user, since right now they only see 'Constant' as the name of the node, and so can't tell which Constant is Constant1 and which is Constant2. Overflow gets around all of this by just making the user pick a name to identify the connector which is being mapped to the parent, so this is another choice to make things easier. This is kind of a pain, but I don't know which way is better. What do people think? Brad From jarl at casema.net Sat Jul 29 08:01:48 2000 From: jarl at casema.net (Jarl van Katwijk) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs References: <200007281346.JAA67536@archa13.cc.uga.edu> Message-ID: <3982C7AB.D71F233E@casema.net> > As I mentioned I have been munging together the type (ie. VALUE) > with a sort of unique identifier (ie. Constant1) to get something > like 'Constant1--VALUE' for a name. This works okay now, but I guess I > just worry how easy it will be to maintain. This is also not very > helpful to a user, since right now they only see 'Constant' as the > name of the node, and so can't tell which Constant is Constant1 and > which is Constant2. > Overflow gets around all of this by just making the user pick a > name to identify the connector which is being mapped to the parent, so > this is another choice to make things easier. This is kind of a pain, > but I don't know which way is better. What do people think? > Maybe base the naming on the manes of the nodes the link connects? So when node_A and node_B are connected, name the link by default Constant_nodeA_nodeB. bye, jarl From jeff at bioinformatics.org Fri Jul 28 10:40:49 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] name changes in pied References: <200007281340.JAA46952@archa11.cc.uga.edu> Message-ID: <39819B71.27B6E9D9@bioinformatics.org> Brad Chapman wrote: > > I think "Link" is confusing because a link is a connection between two > different connectors (or terminii). If we are deciding on naming, what > I'd like to have is: Yet a "link" in a chain forms a "link". Even if there were only one, you would still call it a "link" ;-) > Connector -> One part of a thing that will make up a link (ie. inputs > and outputs of a node). So this is a generic name to refer to Inputs > and Outputs. > > Link -> The connection between two connectors. If you insist, I'll call it "connector". > > "Terminal" and > > "Connector" > > may actually be confusing terms in Peep. > > Well, even in Peep nodes have inputs and output. You might not > explicitly see them (ie. you only see links or pipes or whatever, as > in the command line) but they are there. We need a way to refer to the > things that make up the connections, and this should be separate from > the name of a connection. How about a "linker"? Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Fri Jul 28 10:43:20 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs References: <200007281346.JAA67536@archa13.cc.uga.edu> Message-ID: <39819C08.61B2FEF@bioinformatics.org> Brad Chapman wrote: > > Overflow gets around all of this by just making the user pick a > name to identify the connector which is being mapped to the parent, so > this is another choice to make things easier. This is kind of a pain, > but I don't know which way is better. What do people think? I think we can start with a munged name for a default, as you suggest, and let the user change it to whatever they want. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Fri Jul 28 10:46:46 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Thoughts on Network/Composite Inputs and Outputs References: <200007281346.JAA67536@archa13.cc.uga.edu> <3982C7AB.D71F233E@casema.net> Message-ID: <39819CD6.80897495@bioinformatics.org> Jarl van Katwijk wrote: > > Maybe base the naming on the manes of the nodes the link connects? So > when node_A and node_B are connected, name the link by default > Constant_nodeA_nodeB. But, I think that only individual connectors (linkers) should be named, not the entire connection (link) that they form. If connected connectors all shared one name, it would be hard to distinguish them. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Sat Jul 29 12:00:40 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Avoiding Bad Software Design Message-ID: <3982FFA8.FF0FAC0F@bioinformatics.org> Step 1. Only re-invent the wheel if you truly have a better wheel. Step 2. Know what you're going to do before you do it. Step 3. Know the rules before you break them. Step 4. Be serious. Step 5. Listen to your elders. Step 6. Steal liberally. Step 7. There *is* a difference between beta and production code! Step 8. Maintenance is not a chance for a do-over. Step 9. Know when to kill it. Step 10. Know when to get away from it. Step 11: Be humble. http://www.osopinion.com/Opinions/MontyManley/MontyManley12.html Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Sat Jul 29 14:00:16 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Natural Language CLIs? Message-ID: <39831BB0.415A8D3A@bioinformatics.org> Deanne, You might be interested in this thread at Slashdot, which is about Natural Language Interfaces and the command-line. This is something I thought a great deal about with respect to the Peep interface prior to you actually starting on it. I also wrote a bit about it here in the past. http://slashdot.org/article.pl?sid=00/07/16/1713207&mode=flat >From the referenced article: ``Blackcomb will be the first fully .Net-enabled Windows release from Microsoft, and the version of Windows that will include "the most profound changes in the UI (user interface)," Gates told PDC attendees. Blackcomb will be the Windows release where Microsoft will incorporate the "type in-line" feature, which Microsoft first demonstrated at its .Net Forum 2000 rollout in late June. Type in-line is a natural-language interface that will allow users to more quickly and centrally search the Web, access documents and answer simple questions.'' This is yet another Piper feature to show up in .NET. Sometimes, I swear Bill Gates is reading my mind, or subscribed to this list. ``Yes, I can read your mind. And shame on you!'' Jeff From chapmanb at arches.uga.edu Sat Jul 29 18:36:10 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Getting rid of modification of xml definition files Message-ID: <200007292236.SAA05698@archa13.cc.uga.edu> Hello all; Jean-Marc and I have been having a lot of debate back and forth over the way the Piper should represent the desription of the workflow diagram. The way that Piper has worked up until this point is that it stores the description of each locus/node added to a network as a separate XML file in the filesystem (a *.def file). These files are stored in a heirarchial system, with each composite locus (network) having a directory with all of the xml files describing the children loci/nodes inside of it. Every time something is modified in the user interface, the xml file for representing the modified object gets changed to reflect this. Similary, when we need to retrieve informationa bout an object, we get this from the XML file. Recently in addition to this "permanent" storage layer, I added an "in-memory" storage layer on top of this. This was due to the fact that all of the file accesses were really slowing things down, and it was a *huge* speed improvement to add this "in-memory" layer on top. Even more recently (well, I'm still working on it right now :-) I've been merging this in-memory layer with the Overflow UI* library, and using this library as the in-memory layer. So anyways (you knew I would have to get to a point somewhere in here, didn't you :-), Jean-Marc has been suggesting that we get rid of the "permanent" storage layer and manipulation of the XML files, and instead use the Overflow '*.n' file format as the permanent storage format for Piper. When we hashed this back and forth we came up with a number of reasons to do this: 1. Speed improvement by not having to access the filesystem so much. 2. The "individual XML files for each locus" format will not scale well to large networks (Jean-Marc has some with *tons* of nodes). 3. The *.n file format is more compact and thus easier to transport between compouters. 4. The naming of the *.def files is ugly and hard to handle, especially when there are tons of nodes. 5. Storing these *.def files could get to be a beast, especially if you are working with tons of different networks at once. On the other side, the best reason to retain the '*.def' file manipulation strategy is that if we start having tons and tons of options that could be set in *.def files (ie. having to do with XML descriptions of user interface components) this could make the *.n files get huge, bloated and hard to comprehend. After a lot of discussion, I guess I'm on the side of Jean-Marc (which is why I'm writing this mail, after all) and think it might be best to do away with the copying and manipulation of *.def files. These *.def files will still be used to define a node (as they are being used for right now in Overflow and Piper), but will just be like C header files -- descriptions. Internally, the dl will manipulate the information about the work flow diagram in the "in-memory" layer and use the *.n format for permanent storage. Since the dl runs as a separate process from the UI, crash recovery will still be possible since we can just save a *.n file as a crash file, and then re-load from that. So at any rate, this is a proposal to do away with the *.def file manipulation in the dl. What do people think about this? Are there any proponents of leaving it the old way? Other things that should be considered? Comments are very welcome! Brad From jeff at bioinformatics.org Sun Jul 30 06:28:56 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Getting rid of modification of xml definition files References: <200007292236.SAA05698@archa13.cc.uga.edu> Message-ID: <39840368.8E15B474@bioinformatics.org> I recall the IRC conversation Brad and Jean-Marc first had about this a couple months ago. Basically, Overflow keeps class definitions in C++, which are then inherited by the XML-based .n files. The .n files also describe node connectivity/linkage. Jean-Marc asked during the conversation if the .def files were class definitions or instances. Brad replied that they were both. Brad also mentioned that the advantages of .def and the use of the filesystem hierarchy were (1) better access to class definitions (XML is used rather than C++), (2) crash recovery (the DL is always writing to the filesystem), and (3) the manipulation of networks (requiring a great deal of node and link shuffling). It seemed to me that Jean-Marc agreed about the advantages of .def. The advantages to .n that I can see are (1) clear distinction between class definition and instance, and (2) the use of /less/ files (perhaps a network can always be described in a single XML file). Brad, if you want to switch to the use of .n files and Overflow's overall approach, can you explain (1) how class definitions will be stored and accessed, and if they'll be distinguished any better from class instances, (2) how crash recovery will take place (will .n always reflect the state of the networks?), and (3) if writing to .n all the time is any better or quicker than writing to .def's all the time. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jeff at bioinformatics.org Sun Jul 30 06:39:39 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Getting rid of modification of xml definition files References: <200007292236.SAA05698@archa13.cc.uga.edu> <39840368.8E15B474@bioinformatics.org> Message-ID: <398405EB.F6B2714F@bioinformatics.org> "J.W. Bizzaro" wrote: > > (3) if writing to .n all the time is any better or quicker > than writing to .def's all the time. Isn't it true that, with .n, a single link change will require the entire .n file (with "tons" of nodes) to rewritten while, with .def, only one small file will have to be rewritten? Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From chapmanb at arches.uga.edu Sun Jul 30 08:34:32 2000 From: chapmanb at arches.uga.edu (Brad Chapman) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Getting rid of modification of xml definition files Message-ID: <200007301234.IAA71884@archa13.cc.uga.edu> Jeff wrote: > Brad, if you want to switch to the use of .n files and Overflow's overall > approach, can you explain (1) how class definitions will be stored and > accessed, and if they'll be distinguished any better from class instances, I'm not sure what you mean by `class` here, I guess you're talking about node definitions and node instanaces, right? If so, then nodes will still be defined using the *.def files. The information in a *.def file is stored in the filesystem where Piper can read the information from them when a new node type is added. Jean-Marc likes to think of them as being analagous to C header files. Anyways, that is how nodes are defined. When a node is loaded, the information just gets loaded into "in-memory" storage and is manipulated from there. Jean-Marc just added the code allowing Overflow to read in information from *.def files (previously the info came from a macro call). So in this case, the definitions of nodes (in the *.def files) are static and unchanging, and so they always define a "base state" for the node (or something like that). The it is once it is in memory that things change (like setting parameters, adding inputs and outputs, etc). > (2) > how crash recovery will take place (will .n always reflect the state of the > networks?), Yup, since you can load a workflow diagram from a *.n file, it has enough info to reflect the state of the networks at any time. So to do crash recovery, the dl would just make periodic saves of *.n files while running, and then make this available should a crash occur. I already have a separate thread devoted to filesystem access, so I can just set this up to do saves every once in a while. >> (3) if writing to .n all the time is any better or quicker >> than writing to .def's all the time. Well, it is probably about the same time frame to make an individual change to a *.def file or to save a *.n file. The big slowdown in the *.def system really comes when you are doing something like loading a network from a *.n file. In this case, you have to copy over the *.def files for every node in the network, and this really slows things down. This is really where I see a big bottleneck in the *.def file usage situation. > Isn't it true that, with .n, a single link change will require the entire .n > file (with "tons" of nodes) to rewritten while, with .def, only one small > file > will have to be rewritten? Yup, this is true. What I would just do is overwrite the saved *.n file (and not modify the file, as I have been doing with the *.def files). I can't really remark on the speed for saving a huge *.n file (Jean-Marc?) but for the type of networks I'm messing around with it is very fast. As I mentioned, this will also occur in a separate thread so it shouldn't block the ability of the dl to respond to user interface requests. More questions? What are your thoughts? Brad From jeff at bioinformatics.org Sun Jul 30 09:05:41 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Getting rid of modification of xml definition files References: <200007301234.IAA71884@archa13.cc.uga.edu> Message-ID: <39842825.B590D4B7@bioinformatics.org> Brad Chapman wrote: > > I'm not sure what you mean by `class` here, I guess you're talking > about node definitions and node instanaces, right? That's right. I'm just using the OO analogy. > Yup, since you can load a workflow diagram from a *.n file, it has > enough info to reflect the state of the networks at any time. So to do > crash recovery, the dl would just make periodic saves of *.n files > while running, and then make this available should a crash occur. I > already have a separate thread devoted to filesystem access, so I can > just set this up to do saves every once in a while. The .n file works well with the another analogy I've been making, that the network can be treated as some sort of "document" (Jean-Marc's actual name for it), where the user can do "new", "open", "close", "save" and "save as" (perhaps even "undo") with the network. Periodic saving of .n is then akin to any prgram creating a temp file (Emacs, for example, keeps a temp file named ~). Then, should .n be considered a temp file until the user explicitly calls "save"? I, for one, have no objections to Brad's and Jean-Marc's plan here. Cheers. Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jean-marc.valin at hermes.usherb.ca Sun Jul 30 13:02:19 2000 From: jean-marc.valin at hermes.usherb.ca (Jean-Marc Valin) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] Getting rid of modification of xml definition files Message-ID: <39845F9B.19636700@hermes.usherb.ca> > Isn't it true that, with .n, a single link change will require the entire .n > file (with "tons" of nodes) to rewritten while, with .def, only one small file > will have to be rewritten? It is true that the whole .n will have to be rewritten. However, I've been dealing with .n files of more than 50 nodes and the save operation is done instantly. I believe you could save a .n with 200 nodes in no more time than it takes to save a Word Document. Also, from my experience, you aren't likely to use networks of more than 50-100 nodes. At that point the "program" simply gets too hard to read. Past that, the simplest to to is to break the .n into "functions" in different .n files... the same way as you split a C program into multiple .c files. There's also another thing I'd like to say about the .n format in Overflow. Unlike Piper, for which (if I'm correct) the in-memory storage went after the permanent storage, the ideas between Overflow were the other way (is that correct english?). When I wrote the permanent storage, I used XML simply because I had a library that was doing all the work for me. I did not care what the .n would look like. The fact that I could save and load was enough for me. This is all to say that for me, the permanent storage is just a "nice feature to have" that allows you to keep your work, but it's nothing fundamental in the system. Jean-Marc -- Jean-Marc Valin Universite de Sherbrooke - Genie Electrique valj01@gel.usherb.ca From jarl at casema.net Mon Jul 31 01:01:40 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:22 2006 Subject: [Pipet Devel] BL status References: <200007301234.IAA71884@archa13.cc.uga.edu> Message-ID: <39850833.39F08247@casema.net> Hi All, I've been pretty quiete last few weeks, very buzy with my day-time job & getting GMS ready to forfill the BL tasks. 1. I've been working on getting the BL to be highly extendable and flexible, this work seems acceptable. 2. I've removed all gnome dependancies. This was the hardest TODO entry I guess.. not technically, but mentally, and I hope gnome will be accepted into the PIper project again soon, once people will see it's advantages. 3. Based the corba on cpp-orbit. This implies Piper will need Orbit AND the cpp-orbit bindings. 4. Porting the sources to c++. Currenly all sources do compile with a c++ compiler, but they do not yet link. This is mainly because the corba generated sources that are now c++ object bases, so I've to rewrite much lines of code in the GMS sources. Once I've c++ BL binairies I'll start working on lining the Overflow libries to GMS and develope an BL->PL api. If Jean-Mark has some time to spare we'll work together I hope. bye, jarl From jeff at bioinformatics.org Sun Jul 30 16:30:38 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:23 2006 Subject: [Pipet Devel] BL status References: <200007301234.IAA71884@archa13.cc.uga.edu> <39850833.39F08247@casema.net> Message-ID: <3984906E.D1C6486A@bioinformatics.org> That's great. It sounds like you've been very busy :-) > 3. Based the corba on cpp-orbit. This implies Piper will need Orbit AND the > cpp-orbit bindings. Do you mean omniORB or ORBit? Jeff -- J.W. Bizzaro jeff@bioinformatics.org Director, Bioinformatics.org: The Open Lab http://bioinformatics.org/~jeff "Let the machine do the dirty work." -- Kernighan and Ritchie -- From jarl at casema.net Mon Jul 31 02:27:09 2000 From: jarl at casema.net (jarl van katwijk) Date: Fri Feb 10 19:20:23 2006 Subject: [Pipet Devel] BL status References: <200007301234.IAA71884@archa13.cc.uga.edu> <39850833.39F08247@casema.net> <3984906E.D1C6486A@bioinformatics.org> Message-ID: <39851C3D.C75E054B@casema.net> > > > 3. Based the corba on cpp-orbit. This implies Piper will need Orbit AND the > > cpp-orbit bindings. > > Do you mean omniORB or ORBit? ORBit. and the cpp binding (http://orbitcpp.sourceforge.net/). There bindings are needed to generate cpp code out of the IDL. bye, jarl From jeff at bioinformatics.org Mon Jul 31 18:16:10 2000 From: jeff at bioinformatics.org (J.W. Bizzaro) Date: Fri Feb 10 19:20:23 2006 Subject: [Pipet Devel] SIMPL Message-ID: <3985FAAA.2C3EA777@bioinformatics.org> ``SIMPL, an acronym for Synchronous Interprocess Messaging Project for Linux, espouses the mission to bring `the powerful QNX Send/Receive/Reply messaging paradigm' to the open source movement. According to SIMPL documents, the QNX messaging scheme is `an intuitive and simple to understand, yet immensely powerful, way of designing distributed software applications.''' http://www.linuxdevices.com/news/NS6327780008.html Jeff