From Alan at TheWilliamsFamily.org Tue Jun 1 15:16:10 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:35 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <374EF7D7.8A334ECD@bc.edu> Message-ID: Hello Locians! I am working on a simmilar package to Loci called the General Sequence Toolkit (gst). Currently I am designing the storage interface for the gst package. I thought I'd share some of my ideas on this list for the benifit of both GST and Loci. Warning, I am not a computer science person, but a molecular biology person, so I may miss the obvious (but hopefully not :o). I am loosly defining the "Data Manager" or DM as the interface and backend or server side code for managing bioinformatics data of various types. Some of the design goals for the DM: 1) Allow for extendibility (ie DM plugins) 2) Simple, general, but sufficient interface 3) Minimize transfere of un-needed data 4) Allow for relocation of data 5) Allow for read only access 6) Enable wrappers for common non-xml, non-gst data sets (ie genbank) 7) Allow multi user access 8) The interface should not assume anything regarding the data except that it is in an XML format. 9) Enable network transparency 10) Simple and robust xreferencing 11) ??? In the most basic sense, GST would not have a DM but rather a DM interface to DM plug-ins. Some examples: 1) Genbank/Entrez DM would consist of a local plug-in that provide the program with read only access to NCBI's databases over the internet. The non-xml genbank entries would automatically be wrapped/converted into xml by the DM. 2) Intranet Oracle or AceDB database DM would consist of a local plug-in (as well as a server for the plug-in possibly). The plug-in would handle the network transparency as well as wrapping/converting the data to xml. 3) A file system DM would consist of a plugin for GST as well as a server for the plug-in. Transfere of data from the file system to GST would be handled by a socket b/w the plugin and the server. When a user starts up GST, if a filesystem DM for his/her personal GST directory is not running, one is automatically started. If the user is on another computer, they can still access their personal GST directory as long as the file system DM is running on the same computer as the personal GST directory tree. So rather than working with files straight off the filesystem (as in GDE), a file/data would have to be imported into one of the available DM's before use in GST. So with that overall model in mind, there are several issues that I can think of: 1) Is this a good model? 2) What are the functions that the interface must handle? 3) How will robust cross referencing be done (ie I am working on a project and all the sequences are available through the oracle DM and the blast results are available through the filesystem DM.) Thoughts on the cross referencing issue: Using URI's along with XLinks and Xpointers is probably the best way given that the data model is centered on XML. Along those lines, the URI might look like: gstoracle://oracle.server.bio.com:85/genbank/pri/U29875 where gstoracle defines the data access method (which DM plugin to use), oracle.server.bio.com:85 is the host computer for the data (and port number), and genbank/pri/U29875 is the remaining portion of the reference to get to the desired sequence. So working from this, we should allow for aliases of the protocol://hostname:port portion. This is necessary if we are going to simplify the process of relocating data. The question is where and how should these alias lists be maintained. One possibility is to have an alias list maintained by each client side gst installation. Another posibility is to have the server side of the DM's maintain their own alias lists. A third option is to have both the client side installations and the DM server sides maintain alias lists. The forth option (which I am leaning toward) is that each DM would provide it's own alias upon request and the gst client program when started would update it's own person list of aliases on startup. This way the DM's could request the alias for cross references and store just the alias. example: gstfile://server.bio.com/g349f7 would be converted to gstserver1/g349f7 before storing this reference. if the gstserver1 data was then moved to an oracle database on a different computer, the administrator would only need to edit the personal lists on the DM and client installations inorder to force the initial lookup. No editing of the actual data would be needed. As for encoding these links, it would make sense to use xlink's to refer to xml entities and xpointers to reference within entities. In addition, the DM would be responsible for assigning new ID's so that within each DM ID's are unique. Hence the URI for a GST xml resource would include the ID and would be unique. So the next issue is the interface: This first question in my mind is whether the DM should handle parsing of the XML files or should it just pass/take whole xml files. The interface would be simpler if the DM just passed the whole xml file (in other words the interface deals with xml entities only and not parts of entities). On the other hand, if a user just needs one part of the entity, the whole entity must still be transfered over a potentially slow network connection. The alternative is to have the xml parsing take plase on the DM side of the interface. This would allow for only the requested element to be transfered. In addition, only the requested portion from a non xml data set would have to be wrapped in xml. So despite the additional complexity, it would seem that having the interface be aware of xml parts and not just entities would be highly advantageous. So what are some of the necessary interface functions: 1) Reserve/request unique ID/URI 2) lock/unlock URI 3) Revoke a URI lock 4) Initialize/Login 5) Close/Shutdown 6) Delete Entity/Element 7) Update Entity/Element 8) Add Element 9) Search which returns an XML document with extended xlinks for hits 10) Ability to return browse lists 11) Retrieve alias 12) Retrieve DM Info: * Name (URI) * Description * Read only or Read/Write * ??? Sorry about the lengthing posting, but hopefully it wills stimulate some profitable discussion. -Alan ************************************************************************ Alan Williams (finger alan@avocado.ucr.edu for pgp public key) ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From bizzaro at bc.edu Tue Jun 1 19:44:59 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:35 2006 Subject: [Pipet Devel] Re: Comment on Re: The Loci Project References: Message-ID: <3754707B.C2492E91@bc.edu> This has turned into an interesting discussion on the purpose and use of XML. It would be nice to hear more opinions. Your point that XML "is to represent data that is obvious to humans but hard for computers" is well understood, but how about visa versa? Can XML be used to represent data that is hard for humans but obvious for computers? I can think of one case where a complete GUI is represented not by C or Java, but by XML. Here is a simple GUI, a button in a window, made with an IDE called "GLADE"": ------------------8<--------------------- project1 pixmaps C False False gladesrc.c gladesrc.h gladesig.c gladesig.h GtkWindow window1 window1 GTK_WINDOW_TOPLEVEL GTK_WIN_POS_NONE True True False GtkFixed fixed1 GtkButton button1 32 16 112 40 True ------------------8<--------------------- If you know anything about even HTML, you've got an idea as to how this works. And it is LANGUAGE INDEPENDENT! Could it be argued that this use of XML is "illegal" and should not have been done? XML/SGML is a fantastic middle ground between human and computer and works both ways. Limiting it to document and database markup for Web-based applications would be a shame. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Tue Jun 1 20:22:25 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:35 2006 Subject: [Pipet Devel] pygtools tutorial - intro Message-ID: <37547941.EFB3C86D@bc.edu> Locians, It has become painfully obvious to me that most of the Loci developers do not know Python or the bindings to GTK/GNOME (pygtools). I will therefore start writing tutorials on this list, covering the basics and whatever else I am learning. This will also help to illustrate the development of Loci itself. The tutorials will be of use to others beside us, because I am helping with the pygtools documentation project. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Tue Jun 1 21:09:55 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:35 2006 Subject: [Pipet Devel] pygtools tutorial - python & unix basics Message-ID: <37548463.9440FC23@bc.edu> We will begin with some Python and UNIX basics. Python is run as a script, just like a Perl, Tcl/Tk or UNIX shell script. It therefore must run through another program, an interpreter. We can run a Python script as a file to be read by the interpreter: $ python hello.py Or we can run it as an executable: $ ./hello.py But to do so, we must change the mode of the program: $ chmod u+x hello.py tell the UNIX shell where the interpreter is located. This is done by adding this line to the top of the script: #!/usr/bin/env python From bizzaro at bc.edu Tue Jun 1 21:10:20 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:35 2006 Subject: [Pipet Devel] pygtools tutorial - hello world Message-ID: <3754847C.35333320@bc.edu> Below is the classic "Hello World" written in PyG. I will add comments to each line. -------------8<-------------- # Tell UNIX to run this script through Python. #!/usr/bin/env python # Tell Python to import all of the classes and methods found in the # file gtk.py. This is another Python script and was put in the Python # module library when you installed PyG Tools. This script tells Python # what many of the gtk objects are that are written below. from gtk import * # This is a Python "method". It is just like a procedure or function # in older programming languages. def hello(*args): # The name of the method is "hello". print "Hello World" # This prints to the console/terminal. window.destroy() # This tells GTK to close the window # after printing. # Another Python method. def destroy(*args): # The name of the method is "destroy". window.hide() # This makes the window non-visible. mainquit() # This exits the whole script. # This block of code creates the window for us. window = GtkWindow(WINDOW_TOPLEVEL) # first we make an instance of # the GtkWindow() class. window.connect("destroy", destroy) # This connects the destroy event # to the event handler, the destroy # method we defined above. window.set_border_width(10) # Some asthetics. window.show() # Make the window visible. # This block creates a button, and nearly all widgets follow this pattern. button = GtkButton("Hello World") # Make an instance. button.connect("clicked", hello) # Connect it to an event handler. window.add(button) # Add it to its parent (window is # button's parent). button.show() # And show it. # A loop is created to "catch" widget events, which are of course sent to # the connect event handlers. mainloop() -------------8<-------------- Here is the same script, without all the comments: -------------8<-------------- #!/usr/bin/env python >from gtk import * def hello(*args): print "Hello World" window.destroy() def destroy(*args): window.hide() mainquit() window = GtkWindow(WINDOW_TOPLEVEL) window.connect("destroy", destroy) window.set_border_width(10) window.show() button = GtkButton("Hello World") button.connect("clicked", hello) window.add(button) button.show() mainloop() -------------8<-------------- Next -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Tue Jun 1 21:41:45 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:35 2006 Subject: [Pipet Devel] pygtools tutorial - some definitions Message-ID: <37548BD9.99370C8E@bc.edu> In case you are unfamiliar with some of the terminology of modern object oriented programming (OOP), here are some plain English definitions and examples: "Class" - This is how "objects" are classified. - Each object belongs to a class or category. - There is a hierarchical arrangement of objects "methods" and other classes, under classes. "Object" - An object is a data structure that can "inherit" the properties of its parents. - Objects fall under classes. "Method" - A method is a function or procedure, which means it actually does something to something. - Methods also fall under classes. "Instance" - A unique copy of a "class" with all of its methods and objects. - A change in one instance does not affect other instances. "Event" - An event is when something happens in a GUI. - It can be a mouse movement, button click, etc. - Event handlers are "methods". Here is a Python example: MyClass: # A new class is defined. def my_method(self, my_number): # A new method is defined # with the parameter "my_number". self.my_object = number # A new object is defined. my_intance = MyClass() # An instance is made. my_instance.my_method(2) # The method is called. print my_instance.my_object # This will print "2". (Let me know if something is wrong here.) From dwrice at indiana.edu Wed Jun 2 10:55:50 1999 From: dwrice at indiana.edu (Danny Rice) Date: Fri Feb 10 19:18:35 2006 Subject: [Pipet Devel] pygtools tutorial - some definitions In-Reply-To: <37548BD9.99370C8E@bc.edu> References: <37548BD9.99370C8E@bc.edu> Message-ID: <14165.17237.437074.985107@siren.bio.indiana.edu> >Here is a Python example: > >MyClass: # A new class is defined. > > def my_method(self, my_number): # A new method is defined > # with the parameter "my_number". > self.my_object = number # A new object is defined. > What is "self" here? A generic parameter or something special? Has "my_object" been declared somewhere else? Should number be my_number? > >my_intance = MyClass() # An instance is made. >my_instance.my_method(2) # The method is called. Why is only 1 parameter passed? >print my_instance.my_object # This will print "2". It seems like you would need something like "print my_instance.my_method.my_object" in case you had multiple methods under the class. This is all pretty confusing to me. Danny Rice From bizzaro at bc.edu Wed Jun 2 14:12:14 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:35 2006 Subject: [Pipet Devel] [Fwd: Comment on Re: The Loci Project] Message-ID: <375573FE.C87A59EA@bc.edu> -------------- next part -------------- An embedded message was scrubbed... From: Guy Hulbert Subject: Re: Comment on Re: The Loci Project Date: Wed, 2 Jun 1999 07:45:55 -0400 (EDT) Size: 5076 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/19990602/72e5173e/attachment.mht From bizzaro at bc.edu Wed Jun 2 14:33:26 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] [Fwd: DTD for PDB] Message-ID: <375578F6.D2A73850@bc.edu> FYI: This is Peter Murray Rust's message about converting PDB to XML: -------------- next part -------------- An embedded message was scrubbed... From: Peter Murray-Rust Subject: Re: DTD for PDB Date: Wed, 02 Jun 1999 15:55:57 +0100 Size: 4872 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/19990602/226cb85b/attachment.mht From bizzaro at bc.edu Wed Jun 2 15:11:43 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] pygtools tutorial - some definitions References: <37548BD9.99370C8E@bc.edu> <14165.17237.437074.985107@siren.bio.indiana.edu> Message-ID: <375581EF.8363E43C@bc.edu> Danny Rice wrote: > > >Here is a Python example: > > > >MyClass: # A new class is defined. > > > > def my_method(self, my_number): # A new method is defined > > # with the parameter "my_number". > > self.my_object = number # A new object is defined. > > > > What is "self" here? A generic parameter or something special? Self == MyClass or whatever class the code happens to fall under. So, self.my_object = number is the same as MyClass.my_object = number It's "self-referring". It is put in as the first parameter passed when the method is declared so that my_method inherits MyClass's objects. But it is not, as I mention below, passed as a parameter by the method call. > Has "my_object" been declared somewhere else? Should number be > my_number? You're right, it is my_number. Me bad. self.my_object = my_number _is_ the declaration of the object. It does not need to be type-defined as with C. In C you might type int my_object = my_number; > >my_intance = MyClass() # An instance is made. > >my_instance.my_method(2) # The method is called. > > Why is only 1 parameter passed? The other parameter, "self", is not passed here because it refers to MyClass, which is not the class of _this_ block of code but the class of my_method. So, confusingly, 1 paramter is passed, but the method lists 2. It really only expects one to be passed though (a number or whatever). > >print my_instance.my_object # This will print "2". > > It seems like you would need something like "print > my_instance.my_method.my_object" in case you had multiple methods > under the class. Remember that self.my_object = my_number is the same as MyClass.my_object = my_number Therefore, my_object belongs "under" MyClass in terms of hierarchy, and not my_method, even though I wrote it in under my_method. Using "self" makes this object global to all methods under MyClass. Finally, my_intance = MyClass() So, my_instance.my_object is the "instance of the object". > This is all pretty confusing to me. This is really all there is to it. In the tutorial I sent so far, I explained most of the basics of PyGTools/Python OOP (PyG POOP?) If you just get this stuff down, the rest is pretty simple. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Thu Jun 3 23:49:11 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] looking good Message-ID: <37574CB7.3BC113EA@bc.edu> Locians, Attached is the latest screenshot of the workspace. You will see that loci/locuses are represented as colored buttons, which when double-clicked, pop up the gui of the locus. When double-clicked again, the gui is hidden. Again, these can be dragged and dropped, and the children loci all move with their parents. The arrows shown are actually turned off in the working version, because I found a related bug in the gnome canvas widget. This bug will be fixed in the gnome-libs 1.0.10 release. Any comments? I'm thinking about an elaborate gui-building scheme where loci represent widgets that can be connected in a hierarchy and then "fused" or combined on command to make higher level widgets. This will allow just about any compatible combination of loci to be fused to form larger, customized loci. How's that sound? This is of course on the CVS. If you want a gzipped snapshot, just ask me. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- -------------- next part -------------- A non-text attachment was scrubbed... Name: loci-sshot-19990603.gif Type: image/gif Size: 6605 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/19990604/8f6140a8/loci-sshot-19990603.gif From bizzaro at bc.edu Sun Jun 6 08:21:00 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] constructing the command-line Message-ID: <375A67AC.F8BC6C9B@bc.edu> Locians, As the development continues, it appears the users will be able to construct their own gui's by connecting loci together. This will involve the use of an XML similar to Damon Chaplin's Glade XML, which was really made just to save projects. Here is an example of Glade XML. It's simply a button placed in a container and in a window. -----------------8<------------------ project1 pixmaps C False False gladesrc.c gladesrc.h gladesig.c gladesig.h GtkWindow window1 window1 GTK_WINDOW_TOPLEVEL GTK_WIN_POS_NONE True True False GtkFixed fixed1 GtkButton button1 32 16 112 40 True -----------------8<------------------ Note: James Henstridge already made a Python script to generate a PyGTK gui from Glade XML. You can see that the widgets here are Gtk widgets; of course, this is for Gtk. But let's say we make higher level, composite widgets (the "megawidgets" I've been talking about). Instead simple gui widgets, some of Loci's composite widgets will generate the options of a command-line. (Some composite widgets will be biowidgets of a sort.) For example, in the build phase we start off with a windowlet/locus containing a composite widget that is an option to choose the name of a program. This may simply have a text entry box where the user can type in the name. Next, the user can add a flag option. A new locus appears, and they are presented with a pull down list of all flags, -a to -Z. Underneath that is a text entry box that goes with the flag, because sometimes a flag option expects a term to be added. And the user can actually specify types and ranges for any option. Say the program that the gui is being built for requires an integer after the -n flag. In the build phase, the user can set this using some radio button or something. At this point the user will see several composite widgets, each with its own locus, all connected with lines. Selecting the parent locus (the first one made), the user can pull down a menu and cause them all to be merged into a functional, gui representation of a command-line. Now that the locus is out of the build phase, only the options that were not previously set are visible. I mentioned a couple months ago that there are some programs meant to wrap command-lines in a gui. GGUI is one that uses wizards: http://www.greysite.com/ggui/ I also found TkWorld, which is like GGUI: http://www.tkworld.org/ But Loci is not meant to be just a command-line generator. It will do this, but much more will be included. For one, Loci will keep biodata in a standardized XML format. This will require conversion utilities: one for LocusML -> program-format, and one for program-format -> LocusML. This is because, unlike simple UNIX shell utilities, most bioinfo programs have specific format requirements. Another issue I don't believe is addressed so nicely by GGUI and TkWorld, is piping. Loci's loci will, by the nature of the bioinfo programs used, make extensive use of piping or connecting output to input. And this is supposed to occur seamlessly over the Internet; not something normally done in the UNIX shell. So what will the resultant gui XML look like? I think that when the loci are combined, the XML's can be concatonated. And the user can take them back apart, recombine them as they wish, and even save/store them. A really neat result is that a user can make some unique concoction and then send it across the Internet to someone else using Loci. Rapid locus development and sharing will be important to Loci's success. Well, if you read this far, thank you. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Wed Jun 9 09:11:07 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] questions about Sketch Message-ID: <375E67EB.1D399102@bc.edu> Bernhard, I wrote to you about a month ago about The Loci Project and possibly integrating Sketch so that schematics generated by Loci can be further manipulated by Sketch. Loci uses PyGTK, and is so far 100% Python. http://bioinformatics.org/loci/ BTW, Sketch is a very nice drawing program. We have some questions for you about Sketch: (1) You are switching to PyGTK from Tkinter. Was the Sketch/Tkinter version meant to run on Windows and Mac? Of course you know that PyGTK hasn't been ported to those platforms. Win/Mac compatibility is not a big issue for Loci, but what do you think you are gaining or losing by making the switch? (2) Sketch is licensed under the GPL. However, two components, Pax and Filters, are not. What I find confusing is that they are all written and licensed by you. Why use different licenses? Why not use just the GPL? (3) When I built Sketch on my system, I noticed that some parts (Pax and others) were written in C and compiled into binaries. Exactly what needs to be compiled and can Sketch work without it? IOW, is Sketch really 100% Python? (4) Is there a way Loci or any other Python script can import Sketch as a module and have a drawing area appear in its window or a child window? If not, would you consider implementing this feature, as sort of a plug-in mechanism? Or should we make our own drawing applet via PIL (Python Imaging Library), or just copy/paste the Sketch source directly into Loci? (5) What are your thoughts on using GNOME or PyGNOME? Do you plan to make use of the GNOME canvas at all? Do you think GNOME and other GNU/Linux imaging libraries could substitute for PIL? That's all for now :-) Thanks. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Wed Jun 9 18:53:24 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] from Justin Message-ID: <375EF064.26C890B6@bc.edu> This was originally sent to me...Jeff ------------------- I'm going to work a little late tongiht and start on the first implementation of the underlying loci infrastructure. Here's my plan: 1. simple command-line digest program. feed it a peptide sequence (1 letter abreviations), and a digestion enzyme selected from a few hardcoded in the program. 2. python wrapper from paos object to 1 and back, merged with paos server 3. merged paos server and client, combined with a mini tool address book. 4. paos client with simple text interface to allow entering a peptide sequence and an enzyme name, as well as outputing a list of digested fragments. 2 and 3 will then run on their own ports as daemons. a user starts up 4, types in the seq/enzyme. 4 makes an object with entered data, and connected to 3 3 accepts object from 4, and looks at destination/tool property. if dest/tool prop. is a specific network address, have the client aspect of 3 contact 2. if a generic tool, consult address book to find appropriate address, then contact 2 2 accepts the object from 3, formats the data so 1 can use it. run 1, and 2 takes output, stores in object. 2 then updates 3, and 3 then updates 4. 4 outputs the data. I already have a number of things I want to change, but I'll just build that first. Then, we have to decide how the network layer will work, and how to abstract objects (particularly XML) and how they are transferred. I actually have quite a bit of free-time now with school out, so once I have a home network connection again, I hope to get a lot of the lowlevel loci done this summer. I'm on the xml-mol list, which is also rather silent. Hopefully we can hash out the DTD's for our various datatypes soon. Justin From bizzaro at bc.edu Wed Jun 9 19:28:05 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] from Justin References: <375EF064.26C890B6@bc.edu> Message-ID: <375EF885.58AF944B@bc.edu> Justin Bradford wrote: > I'm going to work a little late tongiht and start on the first > implementation of the underlying loci infrastructure. Here's my plan: > > 1. simple command-line digest program. feed it a peptide sequence (1 > letter abreviations), and a digestion enzyme selected from a few > hardcoded in the program. Great. I was thinking that I could get Loci to wrap a GUI around "ls", as a test ;-) > > 2. python wrapper from paos object to 1 and back, merged with paos server I'm at the point in the WFD (Work Flow Diagram) where I need to communicate with a server, so I need to get into PAOS. > > 3. merged paos server and client, combined with a mini tool address book. What do you have in mind for the tool address book? Right now the WFD uses this generic block of code to define a locus: --------8<-------- locus1 = Locus() locus1.first = locus0 locus1.parent = locus0 locus1.group = locus1.parent.group.add('group') locus1.type = 'Widget' locus1.label = 'This is locus 1' from testwidget1 import * locus1.gui = TestWidget1() locus1.x = 100 locus1.y = 10 locus1.placed_x = locus1.x locus1.placed_y = locus1.y self.add_locus(locus1) --------8<-------- locus.first is the root canvas group, and .parent and .group help define the placement in the hierarchy. We do need to have each locus identify its type and label/name. Next, the actual GUI for the locus is "imported", which in the case of the command-line program is a concatonation of GUI XML's that will help construct the command-line. Since a locus is a concatonation, I need to make a generic XML parser / GUI displayer, which really comes from "pyglade" by James Henstridge. x and y placement is not specified by the locus but is determined by the WFD. I'm working on it. > > 4. paos client with simple text interface to allow entering a peptide > sequence and an enzyme name, as well as outputing a list of digested > fragments. Probably a GtkText widget? > > 2 and 3 will then run on their own ports as daemons. Good point. I realized that the Loci server will have to run as a daemon, just like a Web server. This is because signals can come from many places at any time. How about "locid" for Loci-daemon? I suppose the daemon can include the tool address book? I'm looking for one program that everything can call on for services. > > a user starts up 4, types in the seq/enzyme. > 4 makes an object with entered data, and connected to 3 > 3 accepts object from 4, and looks at destination/tool property. > if dest/tool prop. is a specific network address, have the client aspect > of 3 contact 2. if a generic tool, consult address book to find > appropriate address, then contact 2 I'm thinking that the deamon is the same program on the local and remote machines. They both have address books of what loci/tools are local to them. But when the local daemon contacts the remote daemon, it downloads the remote daemon's address book. So the local daemon acts as if EVERYTHING is local. > 2 accepts the object from 3, formats the data so 1 can use it. > run 1, and 2 takes output, stores in object. > 2 then updates 3, and 3 then updates 4. > 4 outputs the data. > > I already have a number of things I want to change, but I'll just build > that first. Then, we have to decide how the network layer will work, > and how to abstract objects (particularly XML) and how they are > transferred. Yep. Good. > I actually have quite a bit of free-time now with school out, so once I > have a home network connection again, I hope to get a lot of the lowlevel > loci done this summer. Yeah, I have a lot of time to spend on Loci. Well, that is if I don't want to get my Master's by August ;-) BTW, I switched from the Ph.D. to the Master's at BC because I want to get into a more compuational grad program...on a personal note. > I'm on the xml-mol list, which is also rather > silent. Hopefully we can hash out the DTD's for our various datatypes > soon. So I take it you saw my little debate about the uses of XML? Any thoughts? Again, I think most people have one big app in mind when they're talking about XML: the Web browser. In all fairness, XML was adopted by the W3C for the Web, but I think it is suitable as a simpler offspring of SGML. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From justin at ukans.edu Thu Jun 10 22:28:19 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] from Justin In-Reply-To: <375EF885.58AF944B@bc.edu> Message-ID: > > I'm going to work a little late tongiht and start on the first > > implementation of the underlying loci infrastructure. Here's my plan: Well, paos is giving me problems. I can't seem to get clients to communicate to each other via a server. I'm starting some deeper debugging to figure out what's going on. > > 2. python wrapper from paos object to 1 and back, merged with paos server > > I'm at the point in the WFD (Work Flow Diagram) where I need to communicate with > a server, so I need to get into PAOS. We've never really settled the details of "inter-node" communication, which I talk about at the bottom. > What do you have in mind for the tool address book? Right now the WFD uses this > generic block of code to define a locus: I was just thinking concept. I haven't gotten down to any details. However, my rant on DOM below does concern the method of storing this data in a python object. > We do need to have each locus identify its type and > label/name. Probably version, too, so we know what capabilities are available. > So I take it you saw my little debate about the uses of XML? Any thoughts? > Again, I think most people have one big app in mind when they're talking about > XML: the Web browser. In all fairness, XML was adopted by the W3C for the Web, > but I think it is suitable as a simpler offspring of SGML. I agree. It's a useful way to express things. However, I've been thinking about how to express XML data in python. I was thinking about using lists to make a DOM like structure. There are already DOM tools for python, but they use classes which I (and many of the authors) think are too much overhead for a DOM tree. They've proposed rewriting the module in C and as lists, too. data.doc = [NODE, 'dna', [NODE, 'sequence', [ATTR, 'source', 'strain 4'], [DATA, 'ATGGGTCAAGTCCAATGATCCAGTAGTAAGCCAGTT'], ... ] ] A DOM interface can be built on this easily, and normal python code can access it directly, or use the DOM interface. A SAX interface could also be done. The benefit is that individual node/tools/components/etc don't ever have to worry about parsing XML and making up their own structures to hold it. They always get these interfaces. Plus it will make it easy to export interfaces (via CORBA or some simpler plug-in interface). Paos could also be modified to use a DOM->XML module rather than pickle for storage (and even transport). Furthermore, it would make it simpler (possible?) for non-python nodes to communicate in the system. Additionally, we could move to a providing an interface over the network, rather than transferring whole objects. DOM over CORBA, DOM over IP, etc. I know this is much more complicated than just using normal Python structures for simple things, but it will be necessary to be able to express arbitrary XML in a Python object. Also, we can take advantage of Python to make these things more versatile. For instance we could have: obj1.doc = [NODE, 'foo', [NODE, 'bar', [OBJECT, obj2], # a reference to another python object ] ] obj2.doc = [NODE, 'stuff', [DATA, 'zxzxzxzxzxzx'], .... ] Plus, Paos can keep track of these things automatically, and we could do things like only send a part of the whole across the network. I might make up a quick DOM interface, and try this out in the test model I described early (once it's running of course). Thoughts? Justin From David.Lapointe at umassmed.edu Fri Jun 11 09:17:31 1999 From: David.Lapointe at umassmed.edu (Lapointe, David) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] XML Parsing Comparisons Message-ID: <93307F07DE63D211B2F30000F808E9E501644CA9@edunivexch02.umassmed.edu> Clark Cooper just published some comparisons of parsing XML using different parsers (C, Java, Perl, and Python based). The results are not too surprising. http://www.xml.com/xml/pub/Benchmark/exec.html David Lapointe Manager - Research Computing Services UMass Medical School Worcester, MA 01655 508/856-5141 From bizzaro at bc.edu Fri Jun 11 14:48:46 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] Data Storage Interfaces References: Message-ID: <37615A0E.60AC2D43@bc.edu> Hi Alan. I'm sorry your message went unanswered for a while. I consider myself lucky when someone answers me on this list :-) Some of the terminology you use here is a little unclear to me. And I've got to say that it is a little difficult for me to get into a design so different from Loci, since I have spent so much time trying to refine the Loci design. IOW, I'm happy with Loci's design, and I'd probably just recommend the same thing to you ;-) We've discussed licensing, which is a basic distinction between GST and Loci. But what are some other differences? From what you have told me, I've come up with the following: (1) GST uses C; Loci uses Python (2) GST requires Oracle DB; Loci uses PAOS and a simple DB However, both GST and Loci use XML extensively. And the GST DM is something like Loci's server/daemon, "Locid". I don't see many differences there. > In the most basic sense, GST would not have a DM but rather a DM interface > to DM plug-ins. Some examples: Hmmm. But the DM's you listed are required, right? Otherwise there would be no XML parsing, file system access, etc. > 1) Is this a good model? It seems pretty good to me :-) Are you planning on using an object server, like CORBA? How do you hope to get multiple components to communicate independently? > 3) How will robust cross referencing be done (ie I am working on a project > and all the sequences are available through the oracle DM and the blast > results are available through the filesystem DM.) In Loci, a single XML document "travels" a workpath, so everything is done serially (within one path). The document will collect various XML's (different DTD's) along the path, some of which will be understood by certain loci, some of which will be understood by others. > Using URI's along with XLinks and Xpointers is probably the best way given > that the data model is centered on XML. Along those lines, the URI might > look like: > > gstoracle://oracle.server.bio.com:85/genbank/pri/U29875 > > where gstoracle defines the data access method (which DM plugin to use), > oracle.server.bio.com:85 is the host computer for the data (and port > number), and genbank/pri/U29875 is the remaining portion of the reference > to get to the desired sequence. This is something like the scheme Humberto came up with for obtaining widgets that were needed but not present on the user's machine. The remote locus returns an XML with the results of an analysis and specifies the best "viewer" to examine the result graphically, and even where to get the widgets. If the user doesn't have the vidgets, a dialog asks if they would like to get them from whatever URL. > So working from this, we should allow for aliases of the > protocol://hostname:port portion. This is necessary if we are going to > simplify the process of relocating data. The question is where and how This may be akin to what Justin recently called an "address book". Loci has to keep track of what loci are available, locally and remotely, including version numbers. > As for encoding these links, it would make sense to use xlink's to refer > to xml entities and xpointers to reference within entities. In addition, > the DM would be responsible for assigning new ID's so that within each DM > ID's are unique. Hence the URI for a GST xml resource would include the > ID and would be unique. Yeah. Each XML object in Loci will need a unique ID. This should help with cross referencing between different XML's. But again, the XML's will all be in the same document, unless there is a good reason to split the document up (e.g. an analysis could be performed on a subset of the biodata, rather than the whole thing). > This first question in my mind is whether the DM should handle parsing of > the XML files or should it just pass/take whole xml files. The interface Heh. That's an issue we've been discussing for a while. We're using PAOS to pass Python objects, but PAOS doesn't handle XML. We do need XML, but can we use XML and do without PAOS? What if PAOS included an XML parser? That would make things simpler. PAOS is actually good for letting the user graphically monitor the progress of analyses. XML is of course best for data management. We're working on a system that uses both. But as far as Loci is concerned, can we make it so that the XML types (DTD's) are not hard-coded into Loci? What if each locus were responsible for finding its own XML parser/translator? That would pretty much make Loci a general purpose command-line wrapper. Just thinking out loud. > would be simpler if the DM just passed the whole xml file (in other words > the interface deals with xml entities only and not parts of entities). On > the other hand, if a user just needs one part of the entity, the whole > entity must still be transfered over a potentially slow network > connection. That's a good point. We haven't considered slow networks or very large files. > The alternative is to have the xml parsing take plase on the > DM side of the interface. This would allow for only the requested element > to be transfered. In addition, only the requested portion from a non xml > data set would have to be wrapped in xml. So despite the additional > complexity, it would seem that having the interface be aware of xml parts > and not just entities would be highly advantageous. Hmmm. I'd like to hear what Justin, Humberto et al think about that. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From justin at ukans.edu Fri Jun 11 17:08:42 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <37615A0E.60AC2D43@bc.edu> Message-ID: > > would be simpler if the DM just passed the whole xml file (in other words > > the interface deals with xml entities only and not parts of entities). On > > the other hand, if a user just needs one part of the entity, the whole > > entity must still be transfered over a potentially slow network > > connection. > > That's a good point. We haven't considered slow networks or very large files. I have, and that's my problem with 1) how Paos passes objects -- it sends the whole thing. I would prefer just sending updates. Breaking up the data into linked objects could be an adequate compromise. 2) the independently roaming object concept where it's passed directly from tool to tool. Without a "home" everything has to be passed, and by the end of a complex series, that could be a large object. > > The alternative is to have the xml parsing take plase on the > > DM side of the interface. This would allow for only the requested element > > to be transfered. In addition, only the requested portion from a non xml > > data set would have to be wrapped in xml. So despite the additional > > complexity, it would seem that having the interface be aware of xml parts > > and not just entities would be highly advantageous. > > Hmmm. I'd like to hear what Justin, Humberto et al think about that. I think I was addressing similar issues in my previous mail. Your second alternative is roughly a description for DOM (including level 2, however), except DOM doesn't make the programs directly write and parse it's own XML whenever it crosses an interface. At the lowest level, it might actually be straight XML passed across the network, but the interface should abstract that. I'm beginning to think the optimal solution is a virtual interface (or set of optional interface) across all junctions. It's the most efficient (only what the receiving end wants is sent [and only the receiving end really knows what it wants]). It's completely language independent, as well as "junction" indepedent (each end has a standard interface, regardless of whether a C, Python, or Perl script is on the other end, or whether the two are communication via CORBA, TCP/IP, UDP/IP, shared memory, a pipe, a dynamically-loaded plug-in interface). This interface method requires a home location where the object resides throughout its processing life-time. This is what I had envisioned the work flow system to be (ie. coordinating it's various objects, where and when they connected, etc). This could be located on the client machine, and it allows the various other loci to be really dumb (which means small). Justin From bizzaro at bc.edu Fri Jun 11 18:12:53 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:36 2006 Subject: [Pipet Devel] Data Storage Interfaces References: Message-ID: <376189E5.86EABF4D@bc.edu> Justin Bradford wrote: > > That's a good point. We haven't considered slow networks or very large files. > > I have, and that's my problem with > 1) how Paos passes objects -- it sends the whole thing. I would prefer > just sending updates. Breaking up the data into linked objects could be > an adequate compromise. > 2) the independently roaming object concept where it's passed directly > from tool to tool. Without a "home" everything has to be passed, and by > the end of a complex series, that could be a large object. I think the confusion lies in thinking that the XML file is physically passed. In reality, it would reside in the local filesystem while loci read/write from/to the file. But this is not to mention "passing" across filesystems or the Internet. > This interface method requires a home location where the object resides > throughout its processing life-time. This is what I had envisioned the > work flow system to be (ie. coordinating it's various objects, where > and when they connected, etc). This could be located on the client > machine, and it allows the various other loci to be really dumb (which > means small). I agree that this big blob of a data object should reside on the local filesystem. IOW, you're right. What would be involved in making such a system? Does it include PAOS, or are you thinking of making something from scratch? :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Sun Jun 13 14:46:11 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] [Fwd: questions about Sketch] Message-ID: <3763FC73.7C59B1D7@bc.edu> Below is Bernhard's reply to my questions about Sketch. Recall that a large part of the Loci GUI is presenting the results of an analysis as publication-quality illustrations or figures. If the results can be rendered as vector graphic objects, they become much more usable. As you may have guessed, the results can then be manipulated in a vector graphics drawing program and combined with other results, etc. On a related note, I am using the GNOME canvas for the workflow diagram. It has proven to be just what I needed. However, as Bernhard mentions, it may make Loci more difficult to install on non-GNOME, non-Linux systems. I tried using straight GTK for the WFD, but I don't think I could ever match the features and speed of the GNOME canvas with Python drawing primitives. So, Sketch and what Bernhard was able to do with Python, are of great interest to me. The question I must answer now is whether I should stick with the GNOME canvas for almost everything (which may mean making our own PyGNOME vector drawing program) or wait and see if we can implement Sketch. As Bernhard says below, he is working on making Sketch more as we would need it, of course not by our request, but he too finds the GNOME canvas an attractive toolset. Jeff -------------- next part -------------- An embedded message was scrubbed... From: Bernhard Herzog Subject: Re: questions about Sketch Date: 13 Jun 1999 19:23:41 +0200 Size: 7533 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/19990613/aa842e8f/attachment.mht From hortiz at neurobio.upr.clu.edu Mon Jun 14 15:16:34 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: Your message of "Fri, 11 Jun 1999 16:08:42 EST." Message-ID: <199906141916.PAA29606@chimbo.neurobio.upr.clu.edu> justin@ukans.edu said: > That's a good point. We haven't considered slow networks or very > large files. > I have, and that's my problem with 1) how Paos passes objects -- it > sends the whole thing. I would prefer > just sending updates. Breaking up the data into linked objects > could be > an adequate compromise. Paos makes me nervous too. It looks complex, and I can't see what it buys us over CORBA. Orbit is already a standard part of gnome, and we may as well leverage as much as we can from other efforts. > 2) the independently roaming object concept > where it's passed directly > from tool to tool. Without a "home" everything has to be passed, > and by > the end of a complex series, that could be a large object. > I'm beginning to think the optimal solution is a virtual interface (or > set of optional interface) across all junctions. It's the most > efficient (only what the receiving end wants is sent [and only the > receiving end really knows what it wants]). So, data objects have an URI, and a loci can request the data it needs by URI. The local locid can fetch remote data objects, and cache them. Each part of a pipleline of loci can request only the data objects it needs. Your local locus requests it be sent the results that it wants, and only those, and displays them for you. This way only the necessary data objects need be transferred. Imagine a service that annotates a blast search: your locus sends the sequence data to the blast server, the blast server sends the matching genbank UID's to the annotation server, the annotation server may have a local copy of genbank, and gets the sequences from there, then sends the UID's and the feature annotations back to your local locus, which may have to fetch some of the UID's from genbank, then applies the annotations and displays the result. > It's completely language > independent, as well as "junction" indepedent (each end has a standard > interface, regardless of whether a C, Python, or Perl script is on the > other end, or whether the two are communication via CORBA, TCP/IP, UDP/ > IP, shared memory, a pipe, a dynamically-loaded plug-in interface). This sounds good, and can help make sure we don't overcommit to PAOS. We just need a simple way of communicating between loci, "here's this data, please run foo v2 on it", "have your results, formatted for bar v1" > This interface method requires a home location where the object > resides throughout its processing life-time. This is what I had > envisioned the work flow system to be (ie. coordinating it's various > objects, where and when they connected, etc). This could be located on > the client machine, and it allows the various other loci to be really > dumb (which means small). Data objects can be identified by URI's with special URI's for data on a local disk (the locid will have to have some way to service requests for your local data, possibly from multiple loci). But now say we want to run a five step pipeline on 2GB worth of genomic sequences, each of the five loci may want a copy of the sequence, which means our machine will send the file five times. Try that over a modem! Caching at loci hubs can help solve this problem. -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From hortiz at neurobio.upr.clu.edu Mon Jun 14 15:48:26 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] from Justin In-Reply-To: Your message of "Thu, 10 Jun 1999 21:28:19 EST." Message-ID: <199906141948.PAA29832@chimbo.neurobio.upr.clu.edu> > However, I've been thinking about how to express XML data in python. > I was thinking about using lists to make a DOM like structure. There are > already DOM tools for python, but they use classes which I (and many of > the authors) think are too much overhead for a DOM tree. They've proposed > rewriting the module in C and as lists, too. If the current xml module is as slow as indicated in recent posts, we sure need a faster module, especially if loci uses the "everything is XML" data model. > The benefit is that individual node/tools/components/etc don't ever have > to worry about parsing XML and making up their own structures to hold > it. They always get these interfaces. This would be a big benefit. > Paos could also be modified to use a DOM->XML module rather than pickle > for storage (and even transport). Storage isn't a problem, but I think transport should definitely be in XML > Furthermore, it would make it simpler > (possible?) for non-python nodes to communicate in the system. Another big reason for doing transport in a standard format. > Additionally, we could move to a providing an interface over the network, > rather than transferring whole objects. DOM over CORBA, DOM over IP, etc. Like "fetch the exons from sequence M12345" and getting just the exons? Wouldn't this be much more complicated than just fetching a data object? Now we'll need a locid that not only can fetch and transport objects, but one that can understand their internal structure. Transport should just worry about getting the objects from one place to another, if you want something to chop up a data object, write an analysis locus that implements the feature, and returns the object to be transported: Local loci Remote loci ---------- ----------- I want to frob uid m12345 please send the exons gimme the exons from uid m12345 [exon extractor locus runs locally] here's the exons here's the frobbed sequence -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From bizzaro at bc.edu Mon Jun 14 17:56:55 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] Data Storage Interfaces References: <199906141916.PAA29606@chimbo.neurobio.upr.clu.edu> Message-ID: <37657AA7.DF21DB1E@bc.edu> Humberto Ortiz Zuazaga wrote: > > Paos makes me nervous too. It looks complex, and I can't see what it buys us > over CORBA. Orbit is already a standard part of gnome, and we may as well > leverage as much as we can from other efforts. I guess we do need to determine just how much an object broker/server will be used and what services are needed. The problem with Orbit is that it is in C, and there are no Python bindings to it. In fact there is no GPL or LGPL licensed CORBA implementation for Python. So, are our requirements worth the effort of making bindings to Orbit? If we are talking about only Python->Python communication for loci, it does seem like a bit of overkill to use CORBA. On the other hand, CORBA would let Loci use non-Python components, at the price of anarchy IMHO. > > It's completely language > > independent, as well as "junction" indepedent (each end has a standard > > interface, regardless of whether a C, Python, or Perl script is on the > > other end, or whether the two are communication via CORBA, TCP/IP, UDP/ > > IP, shared memory, a pipe, a dynamically-loaded plug-in interface). > > This sounds good, and can help make sure we don't overcommit to PAOS. We just > need a simple way of communicating between loci, "here's this data, please run > foo v2 on it", "have your results, formatted for bar v1" I'd be satisfied if we come up with some system that is (1) small, (2) simple and (3) language independent. Don't get me wrong; Loci's design should not be dictated by a toolset that was not made with Loci in mind. IOW, I am flexible. I came across PAOS and thought that it (or something like it) would be suitable for Python->Python communication. This may be more important for the GUI than for TCP/IP-based communication; I don't know. > Data objects can be identified by URI's with special URI's for data on a local > disk (the locid will have to have some way to service requests for your local > data, possibly from multiple loci). I don't know if this is what you're addressing, but let's say this is how the user will contact a remote command-line program: Workspace ------> Local Locid ----> Net -----> Remote Locid ----> CL Prog I think the local and remote locids are really the same program. So by a network loopback, the Workspace contacts the local locid the same way any two locids would communictate. The above pathway then really looks like this: Workspace ---> Net ---> Local Locid ----> Net -----> Remote Locid ----> CL Prog This is something like making an httpd connection to Apache running on your local machine. This is the key: ----> Net -----> Locid ----> Net ----> If all locids are the same and communicate the same way, the Loci network becomes seamless and limitless. This would mean, of course that command-line programs installed on the local machine would be in something akin to a httpd/cgi-bin directory. > But now say we want to run a five step pipeline on 2GB worth of genomic > sequences, each of the five loci may want a copy of the sequence, which means > our machine will send the file five times. Try that over a modem! > > Caching at loci hubs can help solve this problem. Right, local and remote locids will have the same capabilities. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Mon Jun 14 18:38:53 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] from Justin References: Message-ID: <3765847D.4F7C89EC@bc.edu> Justin Bradford wrote: > > The benefit is that individual node/tools/components/etc don't ever have > to worry about parsing XML and making up their own structures to hold > it. They always get these interfaces. Plus it will make it easy to export > interfaces (via CORBA or some simpler plug-in interface). Related to this, I have been thinking about the different XML's needed to markup all sorts of biodata. Should Loci have a static set of DTD's for biodata? What if someone is working in a more unusual area of biology? Can new DTD's/XML's be "plugged" into Loci? I think if individual loci are responsible for supplying or identifying their own XML translators, Loci will be much more flexible. The way I have been developing the Workspace, functional loci are built up from smaller components. I mentioned earlier how a command-line can be built by connecting widgets/forms together. In the same fashion, an analysis locus will have to be constructed from smaller components. What are they? Biodata in XML format | | |/ XML parser (generic?) | | |/ Translator of biodata to CLP usable format | | |/ command-line program | | |/ Translator of CLP biodata to XML format | | |/ XML writer (generic?) | | |/ Biodata in XML format These components are represented in the Workspace and can be connected as part of a WFD. It is up to the user to put compatible parts together in the right order. This is actually locus development, but pre-built loci can be stored and shared. The translators are unique and may be the most difficult part to engineer. The input translators must know (1) what the input XML/DTD is and (2) what format the CLP requires. The output translator works in the opposite manner. Loci will actually need a huge collection of translators for it to be very usable; the more flexible the translator, the better. Fortunately, like everything else in Loci, they can be retrieved from a remote repository. > Also, we can take advantage of Python to make these things more versatile. > For instance we could have: > obj1.doc = [NODE, 'foo', > [NODE, 'bar', > [OBJECT, obj2], # a reference to another python object > ] > ] > > obj2.doc = [NODE, 'stuff', > [DATA, 'zxzxzxzxzxzx'], > .... > ] Are you saying we should have a text file formatted like this, to be parsed by a Python interpreter? Cheers. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Mon Jun 14 22:03:09 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] from Justin References: <3765847D.4F7C89EC@bc.edu> Message-ID: <3765B45D.2FAEA263@bc.edu> "J.W. Bizzaro" wrote: > > I think if individual loci are responsible for supplying or identifying their > own XML translators, Loci will be much more flexible. Of course if we use DOM, as Justin suggested, then I'm talking about DOM->CPL (command-line program) and CPL->DOM translators. Humberto wrote: > > Paos could also be modified to use a DOM->XML module rather than pickle > > for storage (and even transport). > > Storage isn't a problem, but I think transport should definitely be in XML > > > Furthermore, it would make it simpler > > (possible?) for non-python nodes to communicate in the system. > > Another big reason for doing transport in a standard format. Why? Isn't Justin talking about a standardized DOM? I'm a bit confused as to where we should draw the line between DOM and XML. I'll take a guess...DOM isn't on the filesystem while XML is :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From hortiz at neurobio.upr.clu.edu Mon Jun 14 22:22:21 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] from Justin In-Reply-To: Your message of "Tue, 15 Jun 1999 02:03:09 GMT." <3765B45D.2FAEA263@bc.edu> Message-ID: <199906150222.WAA32413@chimbo.neurobio.upr.clu.edu> > Humberto wrote: > > > > Paos could also be modified to use a DOM->XML module rather than pickle > > > for storage (and even transport). > > > > Storage isn't a problem, but I think transport should definitely be in XML > > Why? Isn't Justin talking about a standardized DOM? I meant xml vs pickling. Paos uses pickling to write data across the network. I argue that loci should use xml, to enable reading the data streams with other tools. Again, we probably be storing xml files on disk as well, in case we want other tools to manipulate the files. > I'm a bit confused as to > where we should draw the line between DOM and XML. I'll take a guess...DOM > isn't on the filesystem while XML is :-) As far as I can tell, DOM is a method for representing XML files in internal storage, and an API for navigating inside the parsed file. Right? From hortiz at neurobio.upr.clu.edu Mon Jun 14 22:37:18 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] from Justin In-Reply-To: Your message of "Mon, 14 Jun 1999 22:38:53 GMT." <3765847D.4F7C89EC@bc.edu> Message-ID: <199906150237.WAA32533@chimbo.neurobio.upr.clu.edu> > Related to this, I have been thinking about the different XML's needed to markup > all sorts of biodata. Should Loci have a static set of DTD's for biodata? What > if someone is working in a more unusual area of biology? Can new DTD's/XML's be > "plugged" into Loci? I think Lincoln Stein's boulder file format is a good example here. In the documentation he suggests that programs doing io on boulder files should ignore and pass through any records it doesn't understand. In the same vein, loci should just operate on the parts of a stream they understand. > I think if individual loci are responsible for supplying or identifying their > own XML translators, Loci will be much more flexible. an analysis locus should say "I need v3.2 foo objects as input" or return a list of acceptable input formats. If your local locus can't export that format object it should request the services of a translator. The locid can perhaps modify the dialog if it already has translators available. From bizzaro at bc.edu Mon Jun 14 23:48:52 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] from Justin References: <199906150237.WAA32533@chimbo.neurobio.upr.clu.edu> Message-ID: <3765CD24.28148BD3@bc.edu> Humberto Ortiz Zuazaga wrote: > > I think Lincoln Stein's boulder file format is a good example here. In > the documentation he suggests that programs doing io on boulder files > should ignore and pass through any records it doesn't understand. Do you have a reference or URL for this? I do understand what you're suggesting. > an analysis locus should say "I need v3.2 foo objects as input" or > return a list of acceptable input formats. If your local locus can't > export that format object it should request the services of a > translator. The locid can perhaps modify the dialog if it already has > translators available. Interesting. It seems as though we have to choose between 3 controllers. Who makes the final decision about components to be added to the workpath? (1) The locus (2) The server/locid (3) The user Maybe we can choose a chain of command, as suggested by the order of the list: (1) The locus tries to find an immediately available solution if that fails... (2) The locid tries to find a remotely available solution if that fails... (3) The user is notified There is a difference, however, in how we see the locus functioning. I think you're saying the locus waits until _runtime_ to check for availability, when the next component in the workpath is needed. It then decides on its own what to do next. But I'm saying that this should be decided at _design_time_, and ultimately by the locus developer. There are a several advantages to checking availability at design time: (1) The workpath and all components can be displayed in the WFD (2) The workflow won't be interrupted constantly at runtime (3) The developer has more control (4) The locus will require less "AI" to make decisions. So the chain of command would be more like this: (1) The locus finds all immediately available solutions (2) The locid finds all remotely available solutions (3) The developer is given a list and decides what to do next What I'd like to see is the locus respond ("I need v3.2 foo objects as input") to the Workspace at design time to let the developer know what is needed. Sure, it's simpler not to get the user involved in this, but we are talking about a developer constructing a command-line, something that may not be done by everyone. But the strongest argument is that design-time availability-checks match my more recent thinking about the use of the GUI for both analysis _and_ locus construction ;-) I hope the GUI can represent, as well as allow for the control of, every Loci component. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Tue Jun 15 01:24:25 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] gnoghurt Message-ID: <3765E389.C3A0D982@bc.edu> For anyone interested in the development of the Loci Workspace, this caught my attention. Gnoghurt (rhymes with yogurt) is a GNOME application that pipelines video feeds and filters. The feeds and filters can be connected together using a GUI, where components are represented by icons and connected with lines. Sound familiar? The use of pixmap icons is something I once considered, but I'm using buttons now. Also, click on mouse button 3 over an icon, and a configuration window pops up (bottom right of screenshot). Interesting. Anyway, a screenshot is attached. Gnoghurt requires gnome-libs-1.0.10, which took me almost a half-hour to compile. I was hoping to glean some ideas from it. Does anyone have a preference for icons in the WFD? I may reconsider it. Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: gnoghurt.gif Type: image/gif Size: 10869 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/19990615/0cd32816/gnoghurt.gif From David.Lapointe at umassmed.edu Tue Jun 15 09:03:49 1999 From: David.Lapointe at umassmed.edu (Lapointe, David) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] FW: ANNOUNCE: new XML+VRML application Message-ID: <93307F07DE63D211B2F30000F808E9E501644CB2@edunivexch02.umassmed.edu> I think this might be of interest to some of you. David Lapointe Manager - Research Computing Services UMass Medical School Worcester, MA 01655 508/856-5141 "What we obtain too cheap, we esteem too lightly." - T. Paine -----Original Message----- From: arun@csa.iisc.ernet.in [mailto:arun@csa.iisc.ernet.in] Sent: Tuesday, June 15, 1999 12:20 AM To: Perl-XML Mailing List Subject: ANNOUNCE: new XML+VRML application Forwarded message: > From ashes@indigo.csa.iisc.ernet.in Sat Jun 12 15:39:57 1999 > Date: Sat, 12 Jun 1999 15:44:22 +0530 (IST) > From: Ashes Dhanna Ganguly > To: www-vrml@vrml.org > cc: arun@csa.iisc.ernet.in > Subject: VRML and Molecular Dynamics > Message-ID: > MIME-Version: 1.0 > Content-Type: TEXT/PLAIN; charset=US-ASCII > content-length: 755 > > Hello Everyone, > > I would like to invite you to our project page. Our project work was > developing an XML application for visualizing chemical simulations on the > web. We have designed a new XML application - Molecular Dynamics Language > (MoDL) and the visualization is in VRML. > > The url is http://violet.csa.iisc.ernet.in/~ashes/MoDL/page1.html > > Some of the examples have simulations while some are static structures > only. The downloads are not yet ready, will be so in a day or two. > > It is still under development and we would appreciate your feedback and > suggestions very much. If there are chemists on this list, we would > particularly like to hear from you about the new language we are > developing, among other things. > > Thank you, > > Ashes & Arun > hello, The conversion from MoDL to VRML was done using XML::Parser with the Subs style. Interested people can have a look at the mentioned url. Comments and suggestions are welcome. So long and thanks for the modules. regards, B.Arun --- You are currently subscribed to perl-xml as: [david.lapointe@umassmed.edu] To unsubscribe, forward this message to leave-perl-xml-65677R@lyris.activestate.com For non-automated Mailing List support, send email to ListHelp@ActiveState.com From David.Lapointe at umassmed.edu Tue Jun 15 09:20:57 1999 From: David.Lapointe at umassmed.edu (Lapointe, David) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] FW: CGI performance issues was Re: Xmerge or XSL... Message-ID: <93307F07DE63D211B2F30000F808E9E501644CB4@edunivexch02.umassmed.edu> We are addressing the same issues on the Loci list as the Perl-XML list is, interestingly. This thread concerns heavy loading of web servers with XML parsing of large files. David Lapointe Manager - Research Computing Services UMass Medical School Worcester, MA 01655 508/856-5141 "What we obtain too cheap, we esteem too lightly." - T. Paine -----Original Message----- From: Tim Bray [mailto:tbray@textuality.com] Sent: Monday, June 14, 1999 6:13 PM To: Perl-XML Mailing List Cc: Perl-XML Mailing List Subject: Re: CGI performance issues was Re: Xmerge or XSL... At 11:00 PM 6/14/99 +0000, Matt Sergeant wrote: >- 30 meg seems *way* high. I don't know what's going on there - but I've >been seeing about 10 maximum with mod_perl and XML stuff going on. I >guess our XML files are a lot smaller. Things to look out for: Using >XML::DOM (sorry Enno!) - it gobbles memory like no tomorrow. No need to apologize to Enno; it's an axiom that to to load N bytes of data into a tree, you're going to burn somewhere between 5 and 50 times N worth of memory, depending on how dense the tagging in, how clever your data structures are, and what programming language you're using. Basically, if you want to read chunks of XML of unconstrained size off the network, you simply cannot afford to use DOM-like APIs, that's all there is to it, or you're staring down the barrel of a loaded cannon. -Tim From David.Lapointe at umassmed.edu Tue Jun 15 09:32:23 1999 From: David.Lapointe at umassmed.edu (Lapointe, David) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] BoulderIO Message-ID: <93307F07DE63D211B2F30000F808E9E501644CB5@edunivexch02.umassmed.edu> This is a perl module. Documentation ( man page) can be found here as well as a link to the distro. http://www-genome.wi.mit.edu/genome_software/other/boulder.html David Lapointe, Ph.D. Research Computing Manager 6-5141 "What we obtain too cheap, we esteem too lightly." - T. Paine From hortiz at neurobio.upr.clu.edu Tue Jun 15 15:12:32 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] from Justin In-Reply-To: Your message of "Tue, 15 Jun 1999 03:48:52 GMT." <3765CD24.28148BD3@bc.edu> Message-ID: <199906151912.PAA05646@chimbo.neurobio.upr.clu.edu> > > I think Lincoln Stein's boulder file format is a good example here. > > Do you have a reference or URL for this? I do understand what you're > suggesting. David already posted the link, I hope everyone has a chance to look at it. Boulderio is a much simpler solution to the same problem we're looking at, how to pipeline biological data through a set of command line programs. It was designed for cgi based apps, and is written in perl. > There is a difference, however, in how we see the locus functioning. I think > you're saying the locus waits until _runtime_ to check for availability, when > the next component in the workpath is needed. It then decides on its own what > to do next. > > But I'm saying that this should be decided at _design_time_, and ultimately by > the locus developer. Yes, I was thinking of runtime checking for needed loci, when a user wishes to use a locus he's never used before. > There are a several advantages to checking availability at > design time: > > (1) The workpath and all components can be displayed in the WFD > (2) The workflow won't be interrupted constantly at runtime > (3) The developer has more control > (4) The locus will require less "AI" to make decisions. But you can't put together a locus that depends on the availability of a network resource (the net may not be up). I think we're talking about two different steps packaging (developing) a composite locus: you do want "compile time" checking for all needed subcomponents to transform your stated input to the correct output using a locus (composite or native): you want run time checking for any matching inputs and outputs this way, youre not limited by the ways the developer thought his locus could be used, as long as you can find a shim from your outputs to it's inputs. This is one of the biggest problems with current bioinformatics programs, why can't I just take the output program X and use it as input to program Y? > So the chain of command would be more like this: > > (1) The locus finds all immediately available solutions > (2) The locid finds all remotely available solutions > (3) The developer is given a list and decides what to do next Replace developer with user, and do this at run time when you run a locus and you've got my idea down too. > What I'd like to see is the locus respond ("I need v3.2 foo objects as input") > to the Workspace at design time to let the developer know what is needed. Sure, > it's simpler not to get the user involved in this, but we are talking about a > developer constructing a command-line, something that may not be done by > everyone. But that's the point, everyone is going to be constructing a command line, every time they run an analysis. In many cases, it will be the trivial command line input -> canned locus -> output, but often it won't be so simple. The people who really need help running the tools aren't the developers, it's the bench scientitsts. -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From bizzaro at bc.edu Tue Jun 15 18:14:04 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] yum, canned locus (was: from Justin) References: <199906151912.PAA05646@chimbo.neurobio.upr.clu.edu> Message-ID: <3766D02C.AB9765DA@bc.edu> Humberto Ortiz Zuazaga wrote: > > Boulderio is a much simpler solution to the same problem we're looking at, how > to pipeline biological data through a set of command line programs. It was > designed for cgi based apps, and is written in perl. I have to take a look at it, but what do you mean by "much simpler"? Do you mean simpler in design but less functional, or simpler in design with the same function? If it's the latter, why not take the same approach? > But you can't put together a locus that depends on the availability of a > network resource (the net may not be up). I agree, we have to consider that all of a canned (run-time) locus's resources may not always be available. This could be because (1) a server is down, (2) some remote component was removed, or (3) the canned locus came from elsewhere, and the user doesn't have permission to access the same resources. I think that we need BOTH design-time AND run-time checks. Are you saying that there should be no design-time check or that nearly all of the design should be a run-time decision by the locus? The user/developer should have control over the details of a locus canning, if he/she would like to can a locus. And the locus should first try to follow its canned path, deviating from the path only if resources are no longer available. Do you agree? > But that's the point, everyone is going to be constructing a command line, > every time they run an analysis. In many cases, it will be the trivial command > line input -> canned locus -> output, but often it won't be so simple. The > people who really need help running the tools aren't the developers, it's the > bench scientitsts. IMHO, we have to expect that bench scientists are no more computer-savvy than the general public. They will expect Loci to operate, on the surface, like any other GUI application. They'll want to choose File->Open, then select a sequence or whatever, and then push a button and get pretty pictures (well, something useful anyway :-) So, we must have lots of pre-canned command-lines too. I'd like the "learning curve" to be low. Later on, the user can peel away the layers of Loci to find an incredibly powerful and flexible system. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From stein at fmppr.fmnh.org Wed Jun 16 10:52:12 1999 From: stein at fmppr.fmnh.org (J. Steinbachs) Date: Fri Feb 10 19:18:37 2006 Subject: [Pipet Devel] yum, canned locus (was: from Justin) References: <199906151912.PAA05646@chimbo.neurobio.upr.clu.edu> <3766D02C.AB9765DA@bc.edu> Message-ID: <3767BA1C.AF064EB1@fmnh.org> Greetings: I'm new to this list. I'm not a programmer or developer, but I will be configuring, installing and (most importantly) maintaining two "spiffy" computer systems for the evolutionary biologists here at the Field Museum of Natural History in Chicago. These folks align small sequences (1000 base pairs), but they've usually got AT LEAST 50 species (I've heard rumors that some of the data sets actually contain upwards of 500 species). Once the alignment is done, they conduct phylogenetic analyses using Paup*, fastDNAml, etc. > IMHO, we have to expect that bench scientists are no more computer-savvy than > the general public. They will expect Loci to operate, on the surface, like any > other GUI application. They'll want to choose File->Open, then select a > sequence or whatever, and then push a button and get pretty pictures (well, > something useful anyway :-) So, we must have lots of pre-canned command-lines > too. I'd like the "learning curve" to be low. Later on, the user can peel away > the layers of Loci to find an incredibly powerful and flexible system. > This paragraph applies to my users very well. While there are some who love a unix prompt and running Paup* "batch" files, most of my users have *shudder* Macs sitting on their desks. The grant money for these spiffy computers also includes some funding for GCG. With these users in mind, I would love to get a system up and running focusing on GCG simply because it would be administratively simple for me and provides an instant friendly user interface for the novice computer users (mostly curators). The other system will likely be some sort of Linux Beowulf cluster without GCG, and for the time being, without a customized friendly user interface - unless of course Loci comes into fruition soon :) When I get one or both of these systems, I would be more than happy to alpha and/or beta test (and I can think of a few users who might be good guinea pigs for the user front-end). Cheers, -jennifer -------------------------- Jennifer Steinbachs Computational Biologist Dept. of Botany The Field Museum Chicago, IL 60605-2496 office: 312-665-7810 fax: 312-665-7158 email: stein@fmnh.org -------------------------- From bizzaro at bc.edu Wed Jun 16 14:38:47 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] yum, canned locus (was: from Justin) References: <199906151912.PAA05646@chimbo.neurobio.upr.clu.edu> <3766D02C.AB9765DA@bc.edu> <3767BA1C.AF064EB1@fmnh.org> Message-ID: <3767EF37.49BCAEF9@bc.edu> "J. Steinbachs" wrote: > > This paragraph applies to my users very well. While there are some who > love a unix prompt and running Paup* "batch" files, most of my users > have *shudder* Macs sitting on their desks. The grant money for these > spiffy computers also includes some funding for GCG. With these users > in mind, I would love to get a system up and running focusing on GCG > simply because it would be administratively simple for me and provides > an instant friendly user interface for the novice computer users (mostly > curators). Hi Jennifer. I'm a bit confused to see "GCG," "administratively simple," and "user friendly" used in the same sentence :-) Are we talking about the same GCG, the Wisconsin Package? I think there is a Web interface, if that's what you're referring to. > The other system will likely be some sort of Linux Beowulf cluster > without GCG, and for the time being, without a customized friendly user > interface - unless of course Loci comes into fruition soon :) Wow. Is the cluster going to be set up in-house or come pre-built by a Linux system builder? As you may have read, Loci can gains computational "power" by distributing analyses to more powerful systems. It would be nice to see it running on a Beowulf cluster. > When I get one or both of these systems, I would be more than happy to > alpha and/or beta test (and I can think of a few users who might be good > guinea pigs for the user front-end). Great. Do you have a system running GNOME at the moment? If so, it shouldn't be much to get the GUI running that I've been working on. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From stein at fmppr.fmnh.org Wed Jun 16 14:58:13 1999 From: stein at fmppr.fmnh.org (J. Steinbachs) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] computing at the Field Museum References: <199906151912.PAA05646@chimbo.neurobio.upr.clu.edu> <3766D02C.AB9765DA@bc.edu> <3767BA1C.AF064EB1@fmnh.org> <3767EF37.49BCAEF9@bc.edu> Message-ID: <3767F3C5.DB3CAC67@fmnh.org> "J.W. Bizzaro" wrote: > > > Hi Jennifer. I'm a bit confused to see "GCG," "administratively simple," and > "user friendly" used in the same sentence :-) Are we talking about the same > GCG, the Wisconsin Package? I think there is a Web interface, if that's what > you're referring to. I'm embarrassed to say that I have been too lazy to really learn HTML - my husband sorta taught me a few things, but why not let him do my HTML stuff for me? :) (his stuff is at http://alloy.net) So, the GCG SeqWeb would be administratively simple in comparison to the alternative: my own home-built web interface to software of our choice (a la http://bioweb.pasteur.fr/intro-uk.html#log). We don't have a yearly operating budget for the beast (doh!), hence my reluctance to completely take the GCG plunge. And then the users likely wouldn't make use of all the nifty tools - just sequence analysis and phylogenetic analysis... > > The other system will likely be some sort of Linux Beowulf cluster > > without GCG, and for the time being, without a customized friendly user > > interface - unless of course Loci comes into fruition soon :) > > Wow. Is the cluster going to be set up in-house or come pre-built by a Linux > system builder? As you may have read, Loci can gains computational "power" by > distributing analyses to more powerful systems. It would be nice to see it > running on a Beowulf cluster. We are currently contemplating two systems, both somewhere in-house: 1. A Sun E3500 or a comparable Alpha-based system, with ETA being the end of the summer - designed around GCG, for now. 2. Tentatively, a Microway Alpha-based 16-node Linux Beowulf cluster. Our serious constraint is space - or lack thereof, otherwise I'd just update UPenn's Phylofarm or NIH's LOBOS system. . o O (who gets money for an expensive system without thinking about where they're gonna put it or how they're gonna cool it?!?!) We had scoped out a small room, currently being used as a computer junkyard - but I'm getting the impression from the building engineers that a/c in that space will be very challenging given the fact that there is little to no direct ventilation currently. Oh, and then there is the problem with the teeny tiny LEAK in the ceiling. > > > When I get one or both of these systems, I would be more than happy to > > alpha and/or beta test (and I can think of a few users who might be good > > guinea pigs for the user front-end). > > Great. Do you have a system running GNOME at the moment? If so, it shouldn't > be much to get the GUI running that I've been working on. Right now, I don't have anything here (windowmaker on my linux box at home). They've got me in temporary office space (Nothing like arriving and being told "oh... don't unpack... you'll be moving in a few weeks...") and I don't even have my own computer - just this nasty NT thing. I'm waiting to hear back from Sun to see if they'll be giving me an UltraSparc 10 through their academic grant program. BTW, I'm on a gcg-managers mailing list (http://molbio.unmc.edu for details) and I mentioned Loci on there :) so you might see a few extra hits on the web page over the next couple of days. cheers, -j -------------------------- Jennifer Steinbachs Computational Biologist Dept. of Botany The Field Museum Chicago, IL 60605-2496 office: 312-665-7810 fax: 312-665-7158 -------------------------- From bizzaro at bc.edu Wed Jun 16 22:26:33 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] ORBit Message-ID: <37685CD9.80A2892F@bc.edu> This came from the GNOME update report. Apparently, a Python backend to ORBit/CORBA may be just around the corner. Something to think about. ORBit 0.4.90 === Elliot made a new release of ORBit; this has the rewritten IDL compiler, complete with a modular backend system so stubs can be generated for multiple languages. I noticed on orbit-list that Brian Wallis has already fixed a couple of bugs and coded up an Eiffel backend - language backends can't be too hard to write, so maybe we'll see a lot of them in short order. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From hortiz at neurobio.upr.clu.edu Thu Jun 17 13:47:53 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] yum, canned locus (was: from Justin) In-Reply-To: Your message of "Tue, 15 Jun 1999 22:14:04 GMT." <3766D02C.AB9765DA@bc.edu> Message-ID: <199906171747.NAA22109@chimbo.neurobio.upr.clu.edu> > Humberto Ortiz Zuazaga wrote: > > > > Boulderio is a much simpler solution to the same problem we're looking at > Do you > mean simpler in design but less functional, or simpler in design with the same > function? If it's the latter, why not take the same approach? Simpler in design, I guess it has about the same functionality, but the data stream is uglier than XML. Here's an example data record NAME=AFM036YB2 SEQUENCE=GATCCGAGATACCCCACACACACACACAC.... STS={ LEFT_START=0 LEFT_LENGTH=14 RIGHT_START=140 RIGHT_LENGTH=20 } = When read, this results in a perl object called a Stone, with tag, value pairs, sort of like a dict. If the stone is in $s, then $s-index(NAME) is AFM036YB2, and $s-> index('STS[0]') is another stone contining the STS data. I've used boulder to do some work on genetic mapping, and found it to be a little slow, but I'm not real good at OO perl, and may have been doing something dumb. The recent discussion on DOM makes me think it's a problem with the document-as-object model, however. > I think that we need BOTH design-time AND run-time checks. Are you saying that > there should be no design-time check or that nearly all of the design should be > a run-time decision by the locus? No, you're right. But the design time checks and the run time checks can be the same. At design time, failure of a check should be fatal, at run time, the locid should try to locate any missing components. > > But that's the point, everyone is going to be constructing a command line, > > IMHO, we have to expect that bench scientists are no more computer-savvy than > the general public. They will expect Loci to operate, on the surface, like any > other GUI application. They'll want to choose File->Open, then select a > sequence or whatever, and then push a button and get pretty pictures (well, > something useful anyway :-) So you think users won't want to do stuff like, "lets open this file, do this, then select all frobs, oh, wait, now do that instead, and give me the green frobs only"? I belive this kind of interactive, incremental analysis is the best feature of loci. Should users only get canned loci, with pushbutton doodads for setting options, and not get the entire loci toolkit, including the plumbing developers use for fitting loci together? -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From justin at ukans.edu Thu Jun 17 14:21:40 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] DOM speed/memory use In-Reply-To: <199906171747.NAA22109@chimbo.neurobio.upr.clu.edu> Message-ID: > I've used boulder to do some work on genetic mapping, and found it to be a > little slow, but I'm not real good at OO perl, and may have been doing > something dumb. The recent discussion on DOM makes me think it's a problem > with the document-as-object model, however. I think the problem with existing DOM implementations in Perl and Python is that they are using full objects for each node. DOM describes an interface, and the underlying structure can be implemented in simpler, and far more lightweight, ways than a full object. If we implement a DOM level 2-ish interface (or something with similar functionality) it can request certain subsection of the whole document. We can probably just generate a skeleton of the XML document in memory, with indices into the XML file. Then we just load the parts the client wants. We could probably do a caching implementation, too. All of this would still be transparent, however. Also, the whole concept of a client wanting all of the "foo" in "bar", or whatever, is simplified this way, too. The client doesn't have to know anything about the "encapsulating" XML structure. It just has to know the name of the substructure it wants (we'll probably have to use namespaces to prevent conflicts). XQL or some other XML query language would make it even more robust. Fundamentally, I think it might be a good idea to put a lot of power and functionality into this "Loci Object", because it makes things simpler for all of the other clients/tools. I think it would also reduce the amount of code, since without it, every tool will have to duplicate the ability to parse XML and somehow identify, extract, and internally represent the specific data it wants. This could be a rather vital component of the whole system, and it might be necessary to write this part in C and provide Python bindings. Definitely not right away, but if speed and memory use become problematic, it can be moved to C code, with no effect on the interface. Justin From bizzaro at bc.edu Thu Jun 17 20:19:07 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] Re: Meta-Application Interface: a proposal References: <199906171842.OAA19301@egret.cbl.umces.edu> Message-ID: <3769907B.C87B4638@bc.edu> Hello Ferdinando. I think you described The Loci Project to the letter :-) Loci is an Internet-based system of clients and servers, connected using a highly graphical user interface. Starting with a collection of the most basal components, the user builds a "workflow diagram", connecting icons and lines. In this manner, Unix command-lines and pipes can be constructed, even across a network. The system was designed for computational biology, where command-line programs are long-running processes and particularly abundant. Loci is being developed with the python-gnome toolset and makes extensive use of XML/DOM and (possibly soon) ORBit/CORBA. It is licensed under the GNU GPL, and is of course developed via the GNU model. We have more than one dozen volunteer developers and about two dozen on the mailing list. We hope to unify some of the scattered efforts to develop a free, comprehensive suite of computational biology applications. http://bioinformatics.org/loci/ Jeff jeff@bioinformatics.org Ferdinando Villa wrote: > > Hi guys and gals, > > I'm not a GNOME developer and I've never posted here, so I > hope you'll forgive my crossposting to a few different GNOME > lists. While thinking about one of my ecological modelling projects > I've come up with an idea for a general core GUI feature that I think deserves > sharing and needs feedback - also because, fingers crossed, I might > have funding by year 2000 to help its development, if done compatibly > with the (non-commercial, open-source) project's purposes. > > You can read about the original project at > http://iee.umces.edu/~villa/IMA. Basically it's a proposal to develop > a framework/set of tools to integrate independent ecological models by > making their semantics explicit (using XML DTD's) and developing sw to > perform connections on a graphical workbench, run models and 'probe' > the information passing along the connections. It's a big thing with > lots of difficulties, but very fascinating to me. > > This leads easily to the idea of a 'meta-application > interface' as a core desktop functionality, where users can 'patch' > applications together graphically (connecting their icons with the > mouse), save their meta-application to the desktop, and use it as a > single app. At the very simplest level this allows to use the UNIX > pipe mechanism graphically: e.g. to print text files as postscript, I > create a new meta-app (a box with input and optionally output ports), > drop the ascii-to-ps translator and the ps-printer icons into it, > connect the meta-app input to the translator and the translator to the > printer, assign an icon and save it. Now I can drop my ascii file on > the icon and have it translated and printed in one step. It's not much > of a deal but you get the idea - try and find something like that on > Windows or MacOS. One of the most powerful features of Unix comes to > the friendly desktop. Myself, I would love to have it. > > Going up with the complexity, you can carry this to a very high > and very new level of power. As a first stab at it: through an API, > applications can notify their 'output ports' and make them available > as outputs in a meta-application context. This translates in users > being able to select graphically the ports from the application's icon > in the meta-application editor. The 'semantics' of the info that > passes through there can be described in XML and the connections, of > course, can be based on CORBA, which makes it possible to use remote > apps ('services') as well. An application in the meta-application can > play a 'lead role' by showing its own interface when running the > meta-app, and is notified of the connected apps through the API, > adjusting its menus and functions (possibly directed from similar info > set in the connected apps) according to what services are available to > it in the meta-application context. Specific 'GUI' apps can offer > specific services like e.g. interactive output port selection: you > drop the Ascii file on the meta-printing app and you get the pop-up > asking which printer/viewer you want it to print on (all printers are > connected to the outputs of the switcher). > > Never mind my silly examples, but the beauty of this is that > the user does it all - if s/he wants - and can personalize the system > to very high levels of proficiency and customization, all without > writing a line of code. Think of the possibilities of using meta-apps > hierarchically within meta-apps. In the long run I can see apps > providing fundamental services (e.g. text composition, financial > conversion, you name it) and 'real' apps being made by users by > patching together the building blocks. And the development of this is > certainly non trivial but very scalable: something encapsulating just > UNIX pipes would be more than enough to make many people happy, and > simpler to do than most GNOME core libraries. I can't imagine > anything more ideal than GNOME for a similar functionality, and it > would be a smashy improvement over any other system, commercial or > not, that I know of. > > What do you think? When it comes to me: I'm paid to do > research as a theoretical ecologist and I've submitted a proposal for > the IMA project, which could be funded or not by the end of the > year. I won't know about funding for a while. If we get the funds, > I'll have to hire a programmer (since 2001 but possibly earlier) to do > the job with me and another colleague, and this person could be > developing the GNOME side as well if we make it fit in there. The > whole modelling project is probably much more complicated than this, > so it would be easy to fit it in ...:) > > If I don't get the funds, I don't think I'll ever have the > resources to do it, but I'd be happy to contribute if someone in the > Gnome community decides it's worth the effort. For now just think > about it and let me know what your feelings are (in case you decide to > email me, know that I won't read email until July 1st. Best address is > villa@cbl.umces.edu, i use this one for mailing lists only). > > In any case, thanks for reading, and best to you all > > ferdinando > > -- > Ferdinando Villa, Ph.D. Assistant Research Scientist > Institute of Ecological Economics University of Maryland > P.O. Box 38, 20688 Solomons MD Phone: (410) 326-7446 > URL: http://iee.umces.edu/~villa Fax: (410) 326-7354 > > -- > FAQ: Frequently-Asked Questions at http://www.gnome.org/gnomefaq > To unsubscribe: mail gnome-list-request@gnome.org with > "unsubscribe" as the Subject. -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From hortiz at neurobio.upr.clu.edu Fri Jun 18 16:41:26 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] ORBit In-Reply-To: Your message of "Thu, 17 Jun 1999 02:26:33 GMT." <37685CD9.80A2892F@bc.edu> Message-ID: <199906182041.QAA32496@chimbo.neurobio.upr.clu.edu> > This came from the GNOME update report. Apparently, a Python backend to > ORBit/CORBA may be just around the corner. Something to think about. http://www.python.org/download/Contributed.html#netobjects Lists a CORBA implementation in python, and two ILU bindings. Can we use these? -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From bizzaro at bc.edu Sat Jun 19 00:18:32 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] ORBit References: <199906182041.QAA32496@chimbo.neurobio.upr.clu.edu> Message-ID: <376B1A18.CD3F8D4E@bc.edu> Humberto Ortiz Zuazaga wrote: > > > This came from the GNOME update report. Apparently, a Python backend to > > ORBit/CORBA may be just around the corner. Something to think about. > > http://www.python.org/download/Contributed.html#netobjects > > Lists a CORBA implementation in python, and two ILU bindings. Can we use these? > Konrad wrote this a while back: "There are even two CORBA implementations for Python (ILU and FNOrb), but ILU is huge and difficult to install, and FNOrb has licensing restrictions." I checked the ILU Web site, and it appears that ILU has almost a freeware-type license, pretty much like Python's license. I could not make a connection to the SYLU Web site to check out its license. FNOrb is something we considered earlier, but it has one of those "free for non-commercial use" licenses, which is not compatible with the GPL and would restrict the commercial use of Loci. Does ILU seem all that huge to everyone? I never used it, but it appears to include all of CORBA, so it must be pretty big. I'm looking for some opinions from people who have used ILU. Should we wait for ORBit bindings or even make the bindings ourselves, if it is as easy as they say? BTW, we did at one time think that CORBA would be too difficult for new developers to pick up. However, if locus canning is done graphically, most people won't have to worry about it. But, on the other hand, translators and widgets will still be made the old-fashioned way and require the knowledge of PAOS, CORBA or whatever. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Sat Jun 19 04:47:52 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] I am pretty, oh so pretty... Message-ID: <376B5938.7E43CDFA@bc.edu> I merged the best of the GUI paradigms I've been toying with the past month or so. Now each locus has a box with a drop shadow, an icon, and a label. Box shape, color, icon and label can all be specified. Attached is a screenshot. You can see the popup menu (from a right mouse button click) over one of the loci, which is also black because it is selected. Below another locus is a windowlet, which you can show/hide by double-clicking in the box. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- -------------- next part -------------- A non-text attachment was scrubbed... Name: loci-sshot-19990619.gif Type: image/gif Size: 10088 bytes Desc: not available Url : http://bioinformatics.org/pipermail/pipet-devel/attachments/19990619/fbe7f523/loci-sshot-19990619.gif From bizzaro at bc.edu Sun Jun 20 23:00:00 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] GST, widgets & figure building Message-ID: <376DAAB0.6F0F938@bc.edu> Alan et al, I thinking about how people can write new widgets for Loci, particularly how GstMultiSeq and the editor can be implemented. One unique feature of Loci is (or will be) that data are displayed as vector graphics (or at least raster images) so that the user can create a figure for publication directly from Loci. Have you considered how this might be handled by your widgets, and do you have any suggestions for widgets doing this in general? The first widgets I planed on making for Loci were going to literally draw onto a (gnome) canvas widget. Since the drawing program to accompany Loci will use the same canvas, copying data will be trivial. However, more interactive widgets, such as GstMultiSeq, probably won't use the same canvas...I don't know. What are you using for your widgets? Any thoughts? :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Mon Jun 21 06:27:14 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] SVG Message-ID: <376E1382.87E6AE01@bc.edu> Some of you may already be aware of the W3C's SVG or Scalable Vector Graphics format. It is an XML for vector graphics. I found out that Raph, of GNOME fame, is working on a GNOME drawing program called Gill: http://www.levien.com/svg/ Check out the SVG file: http://www.levien.com/svg/tiger.svg I'll be looking into this stuff with respect to how we might make Loci's graphical widgets. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From Alan at TheWilliamsFamily.org Mon Jun 21 13:05:14 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <376DAAB0.6F0F938@bc.edu> Message-ID: Sorry for not responding to my own post sooner. I had to switch email addresses and in the process didn't recieve any of the follow-up posts Jeff wrote: > (2) GST requires Oracle DB; Loci uses PAOS and a simple DB I guess I didn't present this clear enough. My thought for GST was to define an interface for data retrieval and storage and then provide several standard data manager plug-ins that implement this interface. Iinitially it would probably be a local filesystem plug-in and a db plug-in [either mysql or acedb].) In many ways, it sounds simmilar to what is being talked about for Loci. I guess the only difference is that the GST approach would be limited to local plug-in based access (However, a plug-in that utilized CORBA could be implemented). The advantage from what I can tell of using a local plug-in approach as opposed to a PAOS approach is that you are not tied to a specific transfere technology, you are only limited in that the endpoint/startpoint from the client side must be a plug-in that implements the DM interface. Jeff wrote: > And the GST DM is something like Loci's server/daemon, "Locid". Sort of (I am probably not being careful about consistently using the term DM). So the approach would be: 1) A well defined DM interface (which opperates on the client side) 2) DM Plug-in(s) which implement the DM Interface (These also opperate on the client side) 3) Depending on the DM Plug-in, various backend or middle end "servers" may need to be implemented. So with Loci, it sounds like the middle end and backend "servers" technologies are fixed, where with a plug-in approach only the plug-in interface in fixed. Jeff wrote: > In Loci, a single XML document "travels" a workpath, so everything is > done serially (within one path). The document will collect various XML's So do you mean that the original data, and analysis results, etc will all accumulate in one xml document? This doesn't sound like a good idea, but I am probably misunderstanding what you are saying. Why might this be a bad idea (I am thinking as I write, so don't flame me too hard ;o): 1) Complicates data locking in a multi-user model 2) Increases server load by forcing parsing of un-needed data (See recent post from bioperl with comment on server side XML parsing). If parsing isn't done on the server side, then you have the issue of having to transfere all that combined data. It sounds better to me to just implement a robust cross referencing mechanism, assume each data object to be just one "item" (ie blast results, or a sequence, or a restriction map). Then let the backend server store the data as it sees fit (ie as one huge flat file, as individual database entries, as individual files, ...) This also simplifies the issue or redoing analysis (which I would assume would be a common task -- ie monthly blast searches on the same query sequence). Jeff wrote: > But as far as Loci is concerned, can we make it so that the XML types > (DTD's) are not hard-coded into Loci? What if each locus were > responsible for finding its own XML parser/translator? That would > pretty much make Loci a general purpose command-line wrapper. Definately! To take it to a further extreme, Loci should provide the xml parsing and a SAX like interface to the data. Then each loci doesn't have to implement an XML parser, only the SAX callbacks to handle the XML data. Then loci could handle any XML data for which a locus is provided and it could even handle data in the absence of a DTD. (The advantage of the SAX interface over passing the whole tree, is that memory isn't wasted on building parts of the tree that aren't needed.) I am not really familiar with the SAX interface, but hints could be provided so that only elements the locus wants are passed through the SAX callbacks. Justin's comments are basically what I was thinking. Rather than passing a program all the data, just pass it a reference (maybe an XLink/XPointer) to the input data. Under the plug-in model, plug-ins for networked data could implement a cache mechanism to speed up access. -Alan ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From Alan at TheWilliamsFamily.org Mon Jun 21 14:04:03 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: <376DAAB0.6F0F938@bc.edu> Message-ID: On Mon, 21 Jun 1999, J.W. Bizzaro wrote: > GstMultiSeq and the editor can be implemented. One unique feature of Loci is > (or will be) that data are displayed as vector graphics (or at least raster > images) so that the user can create a figure for publication directly from > Loci. Certainly inorder to be useful we must be able to output the data in a presentation quality manner. Given all the work on the gnome canvas (esp with anti-aliasing on text and the proposed gnome printing archetecture) that would be the place to start for building a presentation/output widget. This does tie you down to the gnome archetecture as opposed to just gtk+. Part of the problem comes in that sometimes we work with the data in a manner different from how we present it on paper. Multiple sequence data is a good example -- It is easier to work with the data for each sequence on a single line, but on paper we need to break the alignment into chunks. If I remeber correctly, the Loci way of doing data presentation is to have separate locus for analyzing data, editing datam, and presenting data; likewise I think we should have different widgets for each (only when necessary however). My plan for GstMultiSeq is to have a widget that allows one to work with and edit multiple sequence data, add markup (ie change fonts, colors, add boxes). The problem comes with redundancy: should the MultSequence widget also handle/allow for drawing arrows, adding arbitrary text, etc ... This is all stuff that the gnome canvas based figure builder would do. But from a user perspective, it would be nice to be able to add all that stuff in the sequence editor. An added benifit is that the seqence widget can handle moving extra objects around as sequence data is added or deleted, the figure builder presumably has no notion of sequence data, so if I deleted a chunk of sequence data, all my arrows in the figure builder would then be screwed up! So the best approach seems to be to implement as many position depended drawing functions into the editor as possible, and then write a companion widget that can take that data and display it on a gnome canvas. At any rate, this is a good example of where the markup for a figure is really part of the data as well, so the line between figure and data is yet again blured. As for my current implementation of GstMultiSeq (or more specifically GstMonoText) it is a GtkDrawingArea (no gnome stuff). The widget uses two off screen pixmaps (one with the data rendered, and another to do reverse drawing for showing selections). An advantage to this approach is that only the part of the sequence being looked at is rendered (so if we have a huge genomic fragment, we don't allocate a bunch of memory or waste time rendering all of it). The speed of this approach seems pretty good. So one basic approach for writing interactive widgets: Interactive Widget --> derive from whatever you want does whatever you want outputs XML data Figure Display Widget --> Derive from gnome-canvas-item widget Takes the XML data and displays it Allows for appropriate modification of the data So with a sequence editor example: Sequence Editor Widget: * Implements display and editing of sequences * Allows for cut and paste * Allows for sequence position depended markup - Color and Font Changes - Boxes and Shading of certain residues - Arrows or textual notes to certain residues - Allow for calls to other loci programs to further alter the data * Allow setting of display characteristics (which are stored in the sequences XML data Sequence Display Widget: * Will display seqence data on a gnome canvas * Will allow alteration of display settings - Number resudues per line - Ruler Display - Sequence Labels - Clipping of data (Only show residues 456 to 1043) With this approach, all changes under the sequence editor would be automattically reflected when the figure is refreshed. Now if you can implement your interactive widget as a gnome canvas item, then you only need to implement one main widget (display widget) as well as a wrapper widget which just has a gnome-canvas and displays the gnome cnavas item widget. This way the user can work with the data under its own display (with the wrapper) or under the figure builder. The plus with this approach is that the user gets the same interface under either environment. Well, not that I am a GUI expert or anything, just my ramblings on the topic. > The first widgets I planed on making for Loci were going to literally draw onto > a (gnome) canvas widget. Since the drawing program to accompany Loci will use > the same canvas, copying data will be trivial. However, more interactive > widgets, such as GstMultiSeq, probably won't use the same canvas...I don't > know. What are you using for your widgets? Any thoughts? > > > :-) > Jeff > -- > J.W. Bizzaro mailto:bizzaro@bc.edu > Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ > -- > > _______________________________________________ > pipet-devel maillist - pipet-devel@bioinformatics.org > http://bioinformatics.org/mailman/listinfo/pipet-devel > ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ------------------------------------------------------------------------ "But as for me and my house, we will serve the Lord." Joshua 24:15 ************************************************************************ From hortiz at neurobio.upr.clu.edu Mon Jun 21 14:41:52 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: Your message of "Mon, 21 Jun 1999 11:04:03 MST." Message-ID: <199906211841.OAA24896@chimbo.neurobio.upr.clu.edu> > My plan for GstMultiSeq is to have a widget that allows one to work with > and edit multiple sequence data, add markup (ie change fonts, colors, add > boxes). The problem comes with redundancy: should the MultSequence widget > also handle/allow for drawing arrows, adding arbitrary text, etc ... I vote no. The multi sequence widget should allow you to edit the alignment and define features. The fig builder should decide how to display the features, (mark feature A with a yellow box, put a red arrow under feature B, ...). > the figure builder presumably has no notion of sequence data, so if I > deleted a chunk of sequence data, all my arrows in the figure builder > would then be screwed up! Not anymore. We could also allow the fig builder to call back to the multiseq editor, so if we click on a base and add an arrow in the fig builder, the multiseq editor gets called to add a new feature to the multisequence, this is harder, but the right way to do it. > At any rate, this is a good example of where the markup for a figure is > really part of the data as well, so the line between figure and data is > yet again blured. Now the data is a multiseq object, with a list of features, and the markup is a list of visual attributes to apply to the data and the features. Another option is that the figbuilder simply calls the multiseq editor with a pointer to the canvas region where it should draw its output. The multiseq editor will own that piece of the canvas, and register callbacks for the canvas items it draws. All of the multiseq editor's display (draft and final mode) can be done to the gnome canvas. How do the gnome and KDE office suites handle embedding objects into other documents (like a graph in a paper)? This seems to be exactly the same kind of situation. -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From Alan at TheWilliamsFamily.org Mon Jun 21 17:07:11 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: <199906211841.OAA24896@chimbo.neurobio.upr.clu.edu> Message-ID: On Mon, 21 Jun 1999, Humberto Ortiz Zuazaga wrote: > > boxes). The problem comes with redundancy: should the MultSequence widget > > also handle/allow for drawing arrows, adding arbitrary text, etc ... > > I vote no. The multi sequence widget should allow you to edit the alignment > and define features. The fig builder should decide how to display the > features, (mark feature A with a yellow box, put a red arrow under feature B, > ...). I like your point which reinforces the idea that the editor should simply be used to define features, not how they are displayed. I would agree with you, in that the fig builder should provide basic drawing/rendering capabilities that the locus could use to render the data. This reduces redunency and improves consistency. So if we go with the idea of a rendering widget derived from gnome-canvas-item, the rendering widget could use calls to the standard figure builder canvas-item sets for rendering. So the basic set of widgets would be: Main Figure Builder Widget: Uses Gnome Canvas Standard Set of Builder Widget Items: Each Uses Gnome Canvas Items (ie box, arrow, star, line, circle, ... we can assume that any loci installation will know about these) Non-Standard Set of Builder Widget Items: Each Uses Gnome Canvas Items (These are items which Figure Builder Will manage, however they are essentially extensions that we cannot assume that an installation has) Specific Data Rendering Widget Items: Each Uses Gnome Canvas Item and is associated with a certain Loci Data Type So I could implement a MultiSeqRenderWidget(MSRW) which is derived from gnome-canvas-item. This widget must handle output and may optionally handle input. So the MSRW at the bare minimum must take data from a Multi Sequence Data Object (ie maybe a XML SAX interface) and render that data. Optionally, the MSRW may handle input from the user such as: Don't display this feature Display this feature with a yellow box Add a comment to the feature just below the sequene Display labels on the first line only Don't display a ruler Add an arrow to this residue All of these display characteristics could be stored in XML form in the Multi Sequence Data Object or maybe better yet, there should be a markup XML file that has a reference to the actual data. This would allow one to generate multiple "displays" or markup for the same data. So: Sequence Data Object: Sequences Sequence Labels List of Features and their location List of Markup Data Objects that Reference this object Markup Data Object: (or actually a Figure Builder Data Object) XLink to the Sequence Data Object an potentially other Data Objects List of characteristics on how to display each "feature" from each Object (The Figure Builder doesn't have to understand this data, it just passes the data off to the appropriate rendering widget) List of other markup to display (3 types) - Items unknown to Figure Builder: * Call the appropriate rendering widget and pass along any of the stored rendering data for the item which is stored here - Items known to Figure Builder: * Independent markup which isn't tied to one of the data objects * Dependent markup which is tied to one of teh data objects > Not anymore. We could also allow the fig builder to call back to the multiseq > editor, so if we click on a base and add an arrow in the fig builder, the > multiseq editor gets called to add a new feature to the multisequence, this is > harder, but the right way to do it. Another way (with the rendering widget idea) is that each rendering widget can register whether or not it can handle dependent markup. If it can, then when the user selects that object (lets say a multi sequence alignment) and adds an arrow, the figure builder will generate an id for the arrow and pass all the position info for the arrow to the rendering widget. The rendering widget will store the characteristics of the arrow in the markup object. The rendering widget will also create a new feature in the sequence data object using the position and id info in addition it will prompt the user for a label as well as a residue to anchor the arrow to. Now when an old Markup Data Object is to be rendered (or a Figure Builder Data Object), figure builder will go through the list of markup rendering each one. When it gets to an unknown item type, it will create the appropriate rendoring widget to handle the display. When it gets to independent items, it will render them directly. When it gets to a dependent item, it will pass the item to the rendering widget that the item is dependent upon. The rendering widget can then look up any info it needs from the data object (ie for the arrow, what is the position of the residue it is anchored to) modify any of the rendering data if necessary, and then pass the data back to the figure builder to be rendered. > Now the data is a multiseq object, with a list of features, and the markup is > a list of visual attributes to apply to the data and the features. Right. > Another option is that the figbuilder simply calls the multiseq editor with a > pointer to the canvas region where it should draw its output. The multiseq > editor will own that piece of the canvas, and register callbacks for the > canvas items it draws. All of the multiseq editor's display (draft and final > mode) can be done to the gnome canvas. I guess this was basically my idea with the rendering widget. > How do the gnome and KDE office suites handle embedding objects into other > documents (like a graph in a paper)? This seems to be exactly the same kind of > situation. I have no idea ;o( I am just babbling away as I think, so fire away at all the problems with this scheme. (I am sure there are many!) -Alan ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From bizzaro at bc.edu Mon Jun 21 19:20:52 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:38 2006 Subject: [Pipet Devel] GST, widgets & figure building References: Message-ID: <376EC8D4.AD62D4AA@bc.edu> "Alan J. Williams" wrote: > > I like your point which reinforces the idea that the editor should simply > be used to define features, not how they are displayed. Me too. > I would agree with you, in that the fig builder should provide basic > drawing/rendering capabilities that the locus could use to render the > data. This reduces redunency and improves consistency. So if we go with > the idea of a rendering widget derived from gnome-canvas-item, the > rendering widget could use calls to the standard figure builder > canvas-item sets for rendering. So the basic set of widgets would be: > > Main Figure Builder Widget: Uses Gnome Canvas > Standard Set of Builder Widget Items: Each Uses Gnome Canvas Items > (ie box, arrow, star, line, circle, ... we can assume that any loci > installation will know about these) > Non-Standard Set of Builder Widget Items: Each Uses Gnome Canvas Items > (These are items which Figure Builder Will manage, however they are > essentially extensions that we cannot assume that an installation > has) > Specific Data Rendering Widget Items: Each Uses Gnome Canvas Item and > is associated with a certain Loci Data Type Hmmm. I think we have basically the same idea. This is the way I see it: GnomeCanvas is a GtkLayout widget. It contains GnomeCanvasItems, which are GtkObjects. The items can be put into a GnomeCanvasGroup, which is also an item. The items are group, image, line, polygon, rectangle, ellipse, text, icon_text, and widget. Viewer is a GnomeCanvas (wraps GnomeCanvas) (what you're calling "Figure Builder"?). It inherits all the GnomeCanvas items, etc. It defines new high-level (predrawn) bioinformatics items. The items are derived from the standard GnomeCanvas items. The items are 1D nucleotide sequence, chromosome map, protein motif, etc. (the items are what you're calling "rendering widget"?) BTW, "traditionally", anything that displays publication-quality images from data is of the class "viewer". Anything that alters data is of the class "modifier". And data are of the class "document" or maybe just "data". I think we can also define subclasses for each of these (like your "rendering widget" is a subclass of "Figure Builder"?). "Figure Builder" referred to an actual drawing program (like Adobe Illustrator ;-) that also uses a viewer widget. > So I could implement a MultiSeqRenderWidget(MSRW) which is derived from > gnome-canvas-item. This widget must handle output and may optionally > handle input. So the MSRW at the bare minimum must take data from a Multi > Sequence Data Object (ie maybe a XML SAX interface) and render that data. I think the viewer should be a rather dumb widget that only draws what it is told to (draw a circle, for example). There should be a component(s) in-between the modifier and viewer ("translators" maybe) that will translate biodata into canvas items. It just makes things smaller, more modular, flexible and extensible (following the Loci paradigm). I believe a translator would be part of what you are calling "Figure Builder". Translators in general take the XML/DOM objects and call Gtk and derived Loci widgets. They are typically sandwiched between data and viewers/modifiers (loci). > Optionally, the MSRW may handle input from the user such as: > Don't display this feature > Display this feature with a yellow box > Add a comment to the feature just below the sequene > Display labels on the first line only > Don't display a ruler > Add an arrow to this residue Yes, connecting events to handlers. They will also have to be high-level derivatives. > All of these display characteristics could be stored in XML form in the > Multi Sequence Data Object or maybe better yet, there should be a markup > XML file that has a reference to the actual data. This would allow one to > generate multiple "displays" or markup for the same data. So: > > Sequence Data Object: > Sequences > Sequence Labels > List of Features and their location > List of Markup Data Objects that Reference this object Okay. > > Markup Data Object: (or actually a Figure Builder Data Object) > XLink to the Sequence Data Object an potentially other Data Objects > List of characteristics on how to display each "feature" from each > Object (The Figure Builder doesn't have to understand this data, > it just passes the data off to the appropriate rendering widget) > List of other markup to display (3 types) > - Items unknown to Figure Builder: > * Call the appropriate rendering widget and pass along any of > the stored rendering data for the item which is stored here > - Items known to Figure Builder: > * Independent markup which isn't tied to one of the data > objects > * Dependent markup which is tied to one of teh data objects Besides the terminology, I think we see it the same way. And I guess the viewer items should be separate modules from the viewer. > > Not anymore. We could also allow the fig builder to call back to the multiseq > > editor, so if we click on a base and add an arrow in the fig builder, the > > multiseq editor gets called to add a new feature to the multisequence, this is > > harder, but the right way to do it. Will we then be directly mixing graphical markup with biodata markup? > Another way (with the rendering widget idea) is that each rendering widget > can register whether or not it can handle dependent markup. If it can, > then when the user selects that object (lets say a multi sequence > alignment) and adds an arrow, the figure builder will generate an id for > the arrow and pass all the position info for the arrow to the rendering > widget. The rendering widget will store the characteristics of > the arrow in the markup object. The rendering widget will also create a > new feature in the sequence data object using the position and id info in > addition it will prompt the user for a label as well as a residue to > anchor the arrow to. How will the graphical feature be marked in the sequence XML? > Now when an old Markup Data Object is to be rendered (or a Figure Builder > Data Object), figure builder will go through the list of markup rendering > each one. When it gets to an unknown item type, it will create the > appropriate rendoring widget to handle the display. I'm a little foggy on how the viewer ("figure builder") will create a viewer item ("rendering widget") it knows nothing about. > When it gets to > independent items, it will render them directly. When it gets to a > dependent item, it will pass the item to the rendering widget that the > item is dependent upon. The rendering widget can then look up any info it > needs from the data object (ie for the arrow, what is the position of the > residue it is anchored to) modify any of the rendering data if necessary, > and then pass the data back to the figure builder to be rendered. Okay. > > Another option is that the figbuilder simply calls the multiseq editor with a > > pointer to the canvas region where it should draw its output. The multiseq > > editor will own that piece of the canvas, and register callbacks for the > > canvas items it draws. All of the multiseq editor's display (draft and final > > mode) can be done to the gnome canvas. > I guess this was basically my idea with the rendering widget. I agree. > > How do the gnome and KDE office suites handle embedding objects into other > > documents (like a graph in a paper)? This seems to be exactly the same kind of > > situation. I don't THINK that anything like XML is used for doing this in Gnome. I know I have seen a screenshot of a graph embedded in a spreadsheet: http://www.gnome.org/screenshots/gnumeric-bonobo.jpg It has something to do with "Bonobo", which may be an object model. There is A LOT of new functionality coming to Gnome and even Mozilla (SVG for example) that is very much like what we need for Loci. A point though that I would like to make is that these features are so general-purpose, it would be as difficult using them to embed a component into Loci, as making a new stand-alone program (a reason for not using Loci). In addition, they are made with C/C++, which would have to be wrapped in Python. It would be much simpler using only Python. (The same argument though can be used against CORBA and in favor of PAOS.) :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From Alan at TheWilliamsFamily.org Tue Jun 22 12:48:41 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: <376EC8D4.AD62D4AA@bc.edu> Message-ID: < The table of contents on the right side of the web pages is gone. Anybody know the URL to get to the Terminology Page. > On Mon, 21 Jun 1999, J.W. Bizzaro wrote: > It defines new high-level (predrawn) bioinformatics items. > The items are derived from the standard GnomeCanvas items. > The items are 1D nucleotide sequence, chromosome map, protein motif, etc. > (the items are what you're calling "rendering widget"?) Actually, I was thinking of two different types of derived canvas items (although the distinction is probably artificial): 1) Ones which are not connected to data (other than the rendering data such as position and size) 2) Ones which are connected to data (ie sequence data, plasmid map data) > BTW, "traditionally", anything that displays publication-quality images from > data is of the class "viewer". Anything that alters data is of the class > "modifier". And data are of the class "document" or maybe just "data". I think > we can also define subclasses for each of these (like your "rendering widget" is > a subclass of "Figure Builder"?). "Figure Builder" referred to an actual > drawing program (like Adobe Illustrator ;-) that also uses a viewer widget. So we have: +---------+ | Viewer: | Data Display +---------+ | +--> General Purpose Viewer (GnomeCanvas) | +--> Canvas Item Wrappers (This would contain general info on the item such as what to display in the pop up menu, is the item dragable, editable, ... The idea is that the General Purpose Viewer would be able to get all the info and function calls it needs to work with an arbitrary item from this info.) | +--> Items Provided by GnomeCanvas | (Item, Group, Line, Polygon, Rectangle, Ellipse, Text, | Image, Widget) | +--> Items We Provide which Are Not associated with data | (Arrows, ???) | +--> Items We Provide which are associated with data | +--> Sequence(s) [individual residue/graphical] | +--> Plasmid Map | +--> Linkage Map | +--> Gel | +--> .... +----------+ | Modifier | Data Conversion/Manipulation +----------+ +----------+ | Document | Data Storage +----------+ > I think the viewer should be a rather dumb widget that only draws what it is > told to (draw a circle, for example). There should be a component(s) in-between > the modifier and viewer ("translators" maybe) that will translate biodata into > canvas items. It just makes things smaller, more modular, flexible and > extensible (following the Loci paradigm). So would you consider a derived item widget and it's wrapper as I've laid out above to be a "component"? BioData >-------\ \ >----> ViewerItemWidget --> Display on Canvas / Markup Data >---/ > I believe a translator would be part of what you are calling "Figure Builder". > Translators in general take the XML/DOM objects and call Gtk and derived Loci > widgets. They are typically sandwiched between data and viewers/modifiers > (loci). Yes! This is exactly what I am thinking: So for a given figure or view, there is an associated xml data file with all the information the viewer needs to build the figure. The Viewer parses the file and for each element it creates the appropriate CanvasItemWrapper passing it the data contained in the XML element. This way the general purpose viewer doesn't need to know anything about the item it is constructing, only the wrapper does. The wrapper will process the data and create the appropriate canvas item with the appropriate parameters. Viewer Data File (XML): +---------------------------------------------------+ | | | | | | +---------+-----------------------------------------+ | | | | V Viewer ^\ / \\_________________________________________ _________/ \___________________________ \ / \ | / \ | | | | V | | | | | | | | V | | | | | | | | V | | | | | | | | V V V LineItemWrap LabelItemWrap SeqItemWrap | | | | | V V V 1. Get sequence data Create Gnome Create Loci 2. Create Loci Canvas Canvas Line Canvas Label Item Item Item > Will we then be directly mixing graphical markup with biodata markup? What do you mean by graphical? Graphical representation of sequences, pictures, random vector drawings? All of the above I would hope. > How will the graphical feature be marked in the sequence XML? > > So things are a little complicated in that data for a particular item on the figure may be spread across several xml files. So inorder to display a conserved motif on a sequence we have in the Viewer XML Data file an element for a Sequence Item with information needed by the Sequence Canvas Item on how to display it. With that information there may be a directive (an xml element) directing the display of a given feature. Probably a Bad Example: ... GnomeCanvasItem Rectangle Info ... So from this example, the Sequence Item would be derived from the gnome canvas group and it would minimally include a text item to display the sequence and a rectagle item to draw the box. The General Viewer would pass all the info between and including the sequence tags when creating the Sequence Item (through the sequence item wrapper). The Sequence Item Wrapper would request the sequence information form the given location and would then create a text item using that info. It would also create a gnome canvas rectange item passing everything between the rectangle to the rectangle item wrapper. It would set the location and size of the rectangle based on the sequence position for the feature from the data in the sequence file before passing the data on to create the rectangle. > I'm a little foggy on how the viewer ("figure builder") will create a viewer > item ("rendering widget") it knows nothing about. All it needs to know is that the element goes with the LociCanvasSequenceWrapper class. > It has something to do with "Bonobo", which may be an object model. I think so, and the world DOMination white paper as well (i think). I believe they are using a DOM approach to handling embedding. > There is A LOT of new functionality coming to Gnome and even Mozilla (SVG for > example) that is very much like what we need for Loci. A point though that I > would like to make is that these features are so general-purpose, it would be as > difficult using them to embed a component into Loci, as making a new stand-alone > program (a reason for not using Loci). In addition, they are made with C/C++, > which would have to be wrapped in Python. It would be much simpler using only > Python. (The same argument though can be used against CORBA and in favor of > PAOS.) The ability to at least imbed loci stuff into the gnome office package (once it exists) would be awesome, but we would probably be better off to focus on loci as a stand alone package for now. ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From Alan at TheWilliamsFamily.org Tue Jun 22 13:15:14 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: Message-ID: Does anyone know of any good references on the Gnome Canvas? ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From bizzaro at bc.edu Tue Jun 22 18:49:17 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building References: Message-ID: <377012ED.D59EE71@bc.edu> "Alan J. Williams" wrote: > > < The table of contents on the right side of the web pages is gone. > Anybody know the URL to get to the Terminology Page. > Sorry about that. I was getting some complaints that the links on the right were broken, which they were because there was too much that needed to be posted, and every page had to be changed when a link changed. It might have made sense to use an HTML preprocessor to give every page the same table on the right, but I just settled on the old fashioned approach. Anyway, here is the link. I didn't put it on the updated Loci page because some of the terminology is getting out of date. So, I caution you that some things have changed. http://bioinformatics.org/loci/documentation/gloss.html > > Actually, I was thinking of two different types of derived canvas items > (although the distinction is probably artificial): > 1) Ones which are not connected to data (other than the rendering data > such as position and size) > 2) Ones which are connected to data (ie sequence data, plasmid map data) The ones that are not connected to data, are they added by the user? I was thinking, to keep things simpler, the viewers that are part of the workspace or workflow diagram don't act as fully functional drawing programs. IOW, the workspace viewers can't get new items attached (like arrows) by the user. I think that the figure produced in the workspace should be exported to the Figure Builder (the actual drawing program) for that purpose. Of course the figure would still maintain its links to the biodata while in the Figure Builder; it's just that the FB would give the user more space and tools, and would allow multiple viewer items to be combined. > > So we have: > +---------+ > | Viewer: | Data Display > +---------+ > | > +--> General Purpose Viewer (GnomeCanvas) > | > +--> Canvas Item Wrappers > (This would contain general info on the item such as what to > display in the pop up menu, is the item dragable, editable, > ... The idea is that the General Purpose Viewer would be > able to get all the info and function calls it needs to work > with an arbitrary item from this info.) > | > +--> Items Provided by GnomeCanvas > | (Item, Group, Line, Polygon, Rectangle, Ellipse, Text, > | Image, Widget) > | > +--> Items We Provide which Are Not associated with data > | (Arrows, ???) > | > +--> Items We Provide which are associated with data > | > +--> Sequence(s) [individual residue/graphical] > | > +--> Plasmid Map > | > +--> Linkage Map > | > +--> Gel > | > +--> .... > > +----------+ > | Modifier | Data Conversion/Manipulation > +----------+ > > +----------+ > | Document | Data Storage > +----------+ By George, I think he's got it! :-) Of course these aren't the only locus types/classes. We'll find a few more, like "Translator" should be one. About the nomenclature, loci are named as things that perform an action upon a document: a Modifier modifies a document, a Viewer views a document, etc. > So would you consider a derived item widget and it's wrapper as I've laid > out above to be a "component"? > > BioData >-------\ > \ > >----> ViewerItemWidget --> Display on Canvas > / > Markup Data >---/ > I would consider the ViewerItem to be a subtype of the Viewer. Yes, it probably would be a component, with its own box on the workspace. The scheme I have worked up now is that the major type of a locus (such as a Viewer) is represented by a certain box shape in the workspace. The minor type (such as a ViewerItem) is represented by an icon in the major type's box shape. So, for example, a Document is represented by a rectangular box, and a Sequence is represented by an icon that is a picture of a sequence inside of a rectangular box. The screenshot on the homepage doesn't show different box shapes, but you can see how icons are shown inside of boxes. > Yes! This is exactly what I am thinking: So for a given figure or view, > there is an associated xml data file with all the information the viewer > needs to build the figure. The Viewer parses the file and for each > element it creates the appropriate CanvasItemWrapper passing it the > data contained in the XML element. This is what I'm thinking: Major Class / Minor Class ========================= ---+ XML data | Document / Plasmid Sequence | ---+ | | ViewerTranslator ---+ calls XML parser | to get data objects | | | | | Translator / Plasmid to Restriction | | Map View ViewerTranslator | calls Viewer to | draw a "restriction map" ---+ | | | ---+ Viewer calls RestMapViewerItem | to do its job | | | | | Viewer / Restriction Map | | RestMapViewerItem | draws restriction map | ---+ Do you really want to combine the Translator and the Viewer? In the above example, the translator connects a plasmid to a restriction map view. If the biodata were a bacterial genome (also circular), we could just swap Translators and keep the same Viewer. > This way the general purpose viewer > doesn't need to know anything about the item it is constructing, only the > wrapper does. The wrapper will process the data and create the > appropriate canvas item with the appropriate parameters. I guess what you're calling a "Wrapper" is a combination of my Translator + ViewerItem. > > Viewer Data File (XML): > +---------------------------------------------------+ > | | > | | > | | > +---------+-----------------------------------------+ > | > | > > | > | > V > Viewer > ^\ > / \\_________________________________________ > _________/ \___________________________ \ > / \ | > / \ | > | | | > V | | > | | > | | | > | V | > | | > | | | > | | V > | | > | | | > | | | > V V V > LineItemWrap LabelItemWrap SeqItemWrap > | | | > | | V > V V 1. Get sequence data > Create Gnome Create Loci 2. Create Loci Canvas > Canvas Line Canvas Label Item > Item Item Nice diagram! I think we're just switching a couple things around. When SAX passes the data to the Viewer in this diagram, are the data in a Python object? > > > Will we then be directly mixing graphical markup with biodata markup? > > What do you mean by graphical? Graphical representation of sequences, > pictures, random vector drawings? All of the above I would hope. Yes, any presentation data. Are you mixing presentation and biology in the same XML flat file? I guess you answer that below. > > > How will the graphical feature be marked in the sequence XML? > > > > > > So things are a little complicated in that data for a particular item on > the figure may be spread across several xml files. So inorder to display > a conserved motif on a sequence we have in the Viewer XML Data file an > element for a Sequence Item with information needed by the Sequence Canvas > Item on how to display it. With that information there may be a directive > (an xml element) directing the display of a given feature. Probably a Bad > Example: > > > > > > > ... GnomeCanvasItem Rectangle Info ... > > > > > > So from this example, the Sequence Item would be derived from the gnome > canvas group and it would minimally include a text item to display the > sequence and a rectagle item to draw the box. The General Viewer would > pass all the info between and including the sequence tags when creating > the Sequence Item (through the sequence item wrapper). The Sequence Item > Wrapper would request the sequence information form the given location > and would then create a text item using that info. It would also create > a gnome canvas rectange item passing everything between the rectangle to > the rectangle item wrapper. It would set the location and size of the > rectangle based on the sequence position for the feature from the data in > the sequence file before passing the data on to create the rectangle. Oh, so the example above is contrived by mixing biodata with the Viewer items. So, methinks the workflow diagram may include another Document: Document / Sequence | | | Translator / Seq To Motif | | | Document / Motif | | | Viewer / Motif ViewerItem How do you think this fits in with what you're saying? Again, I think we're just switching thinks around. So, I guess this is the way you see it: Document / Sequence | | | Viewer | | | Translator / Seq To Motif | | | Document / Motif | | | Motif ViewerItem I just think it would be easier to represent loci on the workspace by keeping the ViewerItem as a minor class of Viewers. > The ability to at least imbed loci stuff into the gnome office package > (once it exists) would be awesome, but we would probably be better off to > focus on loci as a stand alone package for now. I agree. Loci should have a consolidated core and act like a single application, rather than be a loose collection of Gnome tools. However, import/export should work in Loci like it would in any other Gnome app. But that's a long way off. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Tue Jun 22 18:52:09 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building References: Message-ID: <37701399.AFE4DDB7@bc.edu> "Alan J. Williams" wrote: > > Does anyone know of any good references on the Gnome Canvas? Do you mean documentation, as in whitepapers? There is a page at the Gnome site with just about all of the information that is available: http://www.gnome.org/devel/canvas/ Cheers. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From Alan at TheWilliamsFamily.org Tue Jun 22 22:48:55 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: <377012ED.D59EE71@bc.edu> Message-ID: On Tue, 22 Jun 1999, J.W. Bizzaro wrote: > posted, and every page had to be changed when a link changed. It might have > made sense to use an HTML preprocessor to give every page the same table on the > right, but I just settled on the old fashioned approach. Anyway, here is the As an aside, I've done a fair amount of server side scripting with PHP. Great for just this thing! > The ones that are not connected to data, are they added by the user? I was > thinking, to keep things simpler, the viewers that are part of the workspace or > workflow diagram don't act as fully functional drawing programs. IOW, the I think this is where I am confused. So your thinking with loci is flow centric where the view I am working with is data centric. So what type of data flows do you envision Loci supporting. One-to-one, one-to-many, many-to-one? > workspace viewers can't get new items attached (like arrows) by the user. I > think that the figure produced in the workspace should be exported to the Figure > Builder (the actual drawing program) for that purpose. Of course the figure > would still maintain its links to the biodata while in the Figure Builder; it's > just that the FB would give the user more space and tools, and would allow > multiple viewer items to be combined. Now I am confused! So, from a loci perspective, there is the figure builder which builds a figures from output. Separately there are viewers which are essentially objects which translate biodata into a 2-d visual presentation. The viewers can appear in the workspace data flow as well as in the figure builder, however the figure builder is NOT part of the workspace data flow? > By George, I think he's got it! :-) Don't be so sure! :oP > This is what I'm thinking: > > Major Class / Minor Class > ========================= > > ---+ > XML data | Document / Plasmid Sequence > | ---+ > | > | > ViewerTranslator ---+ > calls XML parser | > to get data objects | > | | > | | Translator / Plasmid to Restriction > | | Map View > ViewerTranslator | > calls Viewer to | > draw a "restriction map" ---+ > | > | > | ---+ > Viewer calls RestMapViewerItem | > to do its job | > | | > | | Viewer / Restriction Map > | | > RestMapViewerItem | > draws restriction map | > ---+ > > Do you really want to combine the Translator and the Viewer? In the above > example, the translator connects a plasmid to a restriction map view. If the > biodata were a bacterial genome (also circular), we could just swap Translators > and keep the same Viewer. ABSOLUTELY NOT! But you actually are combining them in the above example. The restriction map data would be generated by a program/translator and the viewer would simmply be responsible for presenting the data, not generating data as in the example above. > I guess what you're calling a "Wrapper" is a combination of my Translator + > ViewerItem. Wrapper is a bad term here. Sorry. By wrapper I am just meaning an object wrapper around the appropriate gnome canvas item widgets. > Nice diagram! I think we're just switching a couple things around. When SAX > passes the data to the Viewer in this diagram, are the data in a Python object? Uh, SURE! Well, maybe. The simplest would be for an object to be generated from the xml. The disadvantage of this approach is if we are processing a huge sequence (or better yet a database!). These are better (memory wise) handled as streams to my thinking. So if data objects in loci can take xml input for construction and dump xml, we can just pipe/pass the xml. > Yes, any presentation data. Are you mixing presentation and biology in the same > XML flat file? I guess you answer that below. No. But in some sense the data to generate a figure is "biodata" in and of it's own right from a program perspective with references to other types of "biodata". So there might be a: Sequence Document Restriction Map Document Restriction Map Figure Document > How do you think this fits in with what you're saying? Again, I think we're > just switching thinks around. So, I guess this is the way you see it: > > Document / Sequence > | > Viewer > | > Translator / Seq To Motif > | > Document / Motif > | > Motif ViewerItem > I am not sure I understand where the translator and motif doc come from. What I am thinking is that sequence is not just a simple sequence, but an annotated sequence with features. Alternatively we might have a sequence document and a separate features document generated by a translator (lets say a gene prediction program). Both these documents and the translator have nothing to do with viewing the data and shouldn't be part of the viewer. The viewer simply takes the sequence document and lets say an optional separate features document and displays it. No there are a lot of different ways to display features: draw a box around the actual sequences, draw the sequence as a box with features in different colors, draw filled in boxes under the sequence, etc... So once a user decides what features he wants displayed and how, we need to be able to store that info so we can rebuild the figure. I think your confusing what you are calling the motif document with what I was calling the markup document. The markup document would not have any motif/feature data. It would only contain the information that the figure builder would need to reconstruct the figure. Well, not sure this helps at all. Some other questions: 1) In your workspace, each line is a data pipe. Will loci (the program) be able to store that data and essentially restart the data pipe from that point? 2) Do you envision loci managing other data besides strict biodata (ie data on how to reconstruct the data flows, data on how to build displays/figures, ...) 3) Is there are good post/document/whatever that outlines the philosophy/architecture for loci? ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From Alan at TheWilliamsFamily.org Tue Jun 22 22:50:47 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: <37701399.AFE4DDB7@bc.edu> Message-ID: Yeah, I've already checked those out. I was just wondering if there was any tutorals or annotated docs on the canvas. I haven't found any and it would seem that at this point the only way to get to know how to program the canvas would be to look at other source code. On Tue, 22 Jun 1999, J.W. Bizzaro wrote: > > Does anyone know of any good references on the Gnome Canvas? > > Do you mean documentation, as in whitepapers? There is a page at the Gnome site > with just about all of the information that is available: > > http://www.gnome.org/devel/canvas/ ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From bizzaro at bc.edu Tue Jun 22 23:24:57 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building References: Message-ID: <37705389.7276B397@bc.edu> You'll find the technical reference to the Gnome canvas at this URL: http://www.gnome.org/devel/docs/libgnomeui/gnome-objects.html And attached are the Python bindings to the canvas classes. If you want a Python example, the Loci Workspace I have been working on is probably one of the most complete. I think Mavric may also be a good place to look. :-) Jeff "Alan J. Williams" wrote: > > Yeah, I've already checked those out. I was just wondering if there was > any tutorals or annotated docs on the canvas. I haven't found any and it > would seem that at this point the only way to get to know how to program > the canvas would be to look at other source code. > > On Tue, 22 Jun 1999, J.W. Bizzaro wrote: > > > Does anyone know of any good references on the Gnome Canvas? > > > > Do you mean documentation, as in whitepapers? There is a page at the Gnome site > > with just about all of the information that is available: > > > > http://www.gnome.org/devel/canvas/ > > ************************************************************************ > Alan Williams > ------------------------------------------------------------------------ > University of California, Riverside "Where observation is concerned, > Dept. of Botany and Plant Sciences chance favors the prepared mind." > Alan@TheWilliamsFamily.org -- Louis Pasteur > ************************************************************************ > > _______________________________________________ > pipet-devel maillist - pipet-devel@bioinformatics.org > http://bioinformatics.org/mailman/listinfo/pipet-devel -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- -------------- next part -------------- class GnomeCanvas(_gtk.GtkLayout): get_type = _gnomeui.gnome_canvas_get_type def __init__(self, aa=_gtk.FALSE, _obj=None): if _obj: self._o = _obj; return if aa: self._o = _gnomeui.gnome_canvas_new_aa() else: self._o = _gnomeui.gnome_canvas_new() def root(self): return GnomeCanvasGroup( _obj=_gnomeui.gnome_canvas_root(self._o)) def set_scroll_region(self, x1,y1, x2,y2): _gnomeui.gnome_canvas_set_scroll_region(self._o, x1,y1, x2,y2) def get_scroll_region(self): return _gnomeui.gnome_canvas_get_scroll_region(self._o) def set_pixels_per_unit(self, n): _gnomeui.gnome_canvas_set_pixels_per_unit(self._o, n) def set_size(self, width, height): print "GnomeCanvas.set_size deprecated -- use set_usize" self.set_usize(width, height) def scroll_to(self, cx, cy): _gnomeui.gnome_canvas_scroll_to(self._o, cx, cy) def get_scroll_offsets(self): return _gnomeui.gnome_canvas_get_scroll_offsets(self._o) def update_now(self): _gnomeui.gnome_canvas_update_now(self._o) def get_item_at(self, x, y): ret = _gnomeui.gnome_canvas_get_item_at(self._o, x, y) if (ret): return _obj2inst(ret) return None def request_redraw(self, x1,y1, x2,y2): _gnomeui.gnome_canvas_request_redraw(self._o, x1,y1, x2,y2) def w2c(self, wx, wy): return _gnomeui.gnome_canvas_w2c(self._o, wx, wy) def c2w(self, cx, cy): return _gnomeui.gnome_canvas_c2w(self._o, cx, cy) def get_color(self, spec): return _gnomeui.gnome_canvas_get_color(self._o, spec) def set_stipple_origin(self, gc): _gnomeui.gnome_canvas_set_stipple_origin(self._o, gc) _gtk._name2cls['GnomeCanvas'] = GnomeCanvas class GnomeCanvasItem(_gtk.GtkObject): get_type = _gnomeui.gnome_canvas_item_get_type def __init__(self, _obj=None): if _obj: self._o = _obj; return def set(self, **args): _filtprops(args) _gnomeui.gnome_canvas_item_set(self._o, args) def move(self, dx, dy): _gnomeui.gnome_canvas_item_move(self._o, dx, dy) def affine_relative(self, affine): _gnomeui.gnome_canvas_item_affine_relative(self._o, affine) def affine_absolute(self, affine): _gnomeui.gnome_canvas_item_affine_absolute(self._o, affine) def raise_(self, positions): _gnomeui.gnome_canvas_item_raise(self._o, positions) def lower(self, positions): _gnomeui.gnome_canvas_item_lower(self._o, positions) def raise_to_top(self): _gnomeui.gnome_canvas_item_raise_to_top(self._o) def lower_to_bottom(self): _gnomeui.gnome_canvas_item_lower_to_bottom(self._o) def show(self): _gnomeui.gnome_canvas_item_show(self._o) def hide(self): _gnomeui.gnome_canvas_item_hide(self._o) def grab(self, mask, cursor, event_time): _gnomeui.gnome_canvas_item_grab(self._o, mask, cursor, event_time) def ungrab(self, event_time): _gnomeui.gnome_canvas_item_ungrab(self._o, event_time) def reparent(self, new_group): _gnomeui.gnome_canvas_item_reparent(self._o, new_group._o) def grab_focus(self): _gnomeui.gnome_canvas_item_grab_focus(self._o) def get_bounds(self): return _gnomeui.gnome_canvas_item_get_bounds(self._o) def w2i(self, x, y): return _gnomeui.gnome_canvas_item_w2i(self._o, x, y) def i2w(self, x, y): return _gnomeui.gnome_canvas_item_i2w(self._o, x, y) _gtk._name2cls['GnomeCanvasItem'] = GnomeCanvasItem class GnomeCanvasGroup(GnomeCanvasItem): """has arguments 'x' and 'y'. eg you can go new_item('group', x=4)""" get_type = _gnomeui.gnome_canvas_group_get_type def add(self, tp, **args): types = { 'item': _gnomeui.gnome_canvas_item_get_type, 'group': _gnomeui.gnome_canvas_group_get_type, 'icon_text':_gnomeui.gnome_icon_text_item_get_type, 'image': _gnomeui.gnome_canvas_image_get_type, 'line': _gnomeui.gnome_canvas_line_get_type, 'polygon': _gnomeui.gnome_canvas_polygon_get_type, 're': _gnomeui.gnome_canvas_re_get_type, 'rect': _gnomeui.gnome_canvas_rect_get_type, 'ellipse': _gnomeui.gnome_canvas_ellipse_get_type, 'text': _gnomeui.gnome_canvas_text_get_type, 'widget': _gnomeui.gnome_canvas_widget_get_type } if type(tp) == type('string'): tp = types[tp]() _filtprops(args) return _obj2inst(_gnomeui.gnome_canvas_item_new(self._o, tp, args)) def new_item(self, tp, **args): print "I renamed this func to GnomeCanvasGroup.add" return apply(self.add, (tp,), args) _gtk._name2cls['GnomeCanvasGroup'] = GnomeCanvasGroup class GnomeCanvasImage(GnomeCanvasItem): """has arguments 'image', 'x', 'y', 'width', 'height', 'anchor'""" get_type = _gnomeui.gnome_canvas_image_get_type _gtk._name2cls['GnomeCanvasImage'] = GnomeCanvasItem class GnomeCanvasLine(GnomeCanvasItem): """has arguments 'points', 'fill_color', 'width_pixels', 'width_units', 'cap_style', 'join_style', 'first_arrowhead', 'last_arrowhead', 'smooth', 'spline_steps', 'arrow_shape_a', 'arrow_shape_b', 'arrow_shape_c'""" get_type = _gnomeui.gnome_canvas_line_get_type _gtk._name2cls['GnomeCanvasLine'] = GnomeCanvasLine class GnomeCanvasPolygon(GnomeCanvasItem): """has argyments 'points', 'fill_color', 'fill_color_gdk', 'outlint_color', 'outline_color_gdk', 'fill_stipple', 'outline_stipple', 'width_pixels', 'width_units'""" get_type = _gnomeui.gnome_canvas_polygon_get_type _gtk._name2cls['GnomeCanvasPolygon'] = GnomeCanvasPolygon class GnomeCanvasRE(GnomeCanvasItem): """has arguments 'x1', 'y1', 'x2', 'y2', 'fill_color', 'outline_color', 'width_pixels', 'width_units'""" get_type = _gnomeui.gnome_canvas_re_get_type _gtk._name2cls['GnomeCanvasRE'] = GnomeCanvasRE class GnomeCanvasRect(GnomeCanvasRE): """has arguments of GnomeCanvasRE""" get_type = _gnomeui.gnome_canvas_rect_get_type _gtk._name2cls['GnomeCanvasRect'] = GnomeCanvasRect class GnomeCanvasEllipse(GnomeCanvasRE): """has arguments of GnomeCanvasRE""" get_type = _gnomeui.gnome_canvas_ellipse_get_type _gtk._name2cls['GnomeCanvasEllipse'] = GnomeCanvasEllipse class GnomeCanvasText(GnomeCanvasItem): """has arguments 'text', 'x', 'y', 'font', 'anchor', 'justification', 'fill_color'""" get_type = _gnomeui.gnome_canvas_text_get_type _gtk._name2cls['GnomeCanvasText'] = GnomeCanvasText class GnomeCanvasWidget(GnomeCanvasItem): """has arguments 'widget', 'x', 'y', 'width', 'height', 'anchor', 'size_pixels'""" get_type = _gnomeui.gnome_canvas_widget_get_type _gtk._name2cls['GnomeCanvasWidget'] = GnomeCanvasWidget From bizzaro at bc.edu Wed Jun 23 00:40:00 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building References: Message-ID: <37706520.BB0E61C9@bc.edu> "Alan J. Williams" wrote: > > As an aside, I've done a fair amount of server side scripting with PHP. > Great for just this thing! Maybe after things pick up we can bring back the table of contents. If you'd like to help, that would be great! BTW, I noticed you haven't logged into your account yet. > I think this is where I am confused. So your thinking with loci is flow > centric where the view I am working with is data centric. So what type of > data flows do you envision Loci supporting. One-to-one, one-to-many, > many-to-one? Yes. Yes. Yes. Yes. Yes. What were you thinking? I guess we are on different planes of quantum reality, but we'll hammer things out ;-) > Now I am confused! So, from a loci perspective, there is the figure > builder which builds a figures from output. Separately there are viewers > which are essentially objects which translate biodata into a 2-d visual > presentation. The viewers can appear in the workspace data flow as well > as in the figure builder, however the figure builder is NOT part of the > workspace data flow? That pretty much sums it up. Except, Viewers are widgets+ that handle ViewerItems, which are graphical objects, and Figure Builder is one "big" Viewer running as a separate application. The user can then copy/paste or drag/drop multiple graphical objects (ViewerItems) onto the FB. This allows the user to choose only the Loci-generated figures they want for an illustration. What were you thinking of? Is there anything wrong with this approach? > ABSOLUTELY NOT! But you actually are combining them in the above example. > The restriction map data would be generated by a program/translator and > the viewer would simmply be responsible for presenting the data, not > generating data as in the example above. I agree with you. I'm not sure why you think I combined the Translator and Viewer. Maybe saying "ViewerTranslator" is confusing. I only wanted to show that the Translator is specifically for Viewers. So, ViewerTranslator == Translator. Sorry, I deleted the diagram. > > I guess what you're calling a "Wrapper" is a combination of my Translator + > > ViewerItem. > Wrapper is a bad term here. Sorry. By wrapper I am just meaning an object > wrapper around the appropriate gnome canvas item widgets. You mean something like a composite widget? > > Nice diagram! I think we're just switching a couple things around. When SAX > > passes the data to the Viewer in this diagram, are the data in a Python object? > Uh, SURE! Well, maybe. The simplest would be for an object to be generated > from the xml. The disadvantage of this approach is if we are processing a > huge sequence (or better yet a database!). These are better (memory wise) > handled as streams to my thinking. So if data objects in loci can take > xml input for construction and dump xml, we can just pipe/pass the xml. Yep. This whole "what format is best for object transfer?" problem is what Justin and Humberto have been working on. We'll have to come up with something soon. > > Yes, any presentation data. Are you mixing presentation and biology in the same > > XML flat file? I guess you answer that below. > No. But in some sense the data to generate a figure is "biodata" in and of > it's own right from a program perspective with references to other types > of "biodata". > > So there might be a: Sequence Document > Restriction Map Document > Restriction Map Figure Document I like that. I think we're on the right track. > > Document / Sequence > > | > > Viewer > > | > > Translator / Seq To Motif > > | > > Document / Motif > > | > > Motif ViewerItem > > > I am not sure I understand where the translator and motif doc come from. I just thought this is how you are seeing things. I guess not. > What I am thinking is that sequence is not just a simple sequence, but an > annotated sequence with features. Alternatively we might have a sequence > document and a separate features document generated by a translator (lets > say a gene prediction program). That sounds good. (Technically though a gene prediction program would be a "Modifier" that is linked to a Translator ;-) > Both these documents and the translator > have nothing to do with viewing the data and shouldn't be part of the > viewer. The viewer simply takes the sequence document and lets say an > optional separate features document and displays it. Nothing directly to do with viewing. So you have: Doc / Seq | | | Trans / Seq to GenePredict | | | Mod / GenePredict | | | Trans / GenePredict to AnnotatedSeq | | | Doc / AnnotatedSeq (Translators sandwich everything.) Now this has nothing to do with a Viewer, but if the user wants to add a Viewer, they have to add these loci: Trans / AnnotatedSeq to ASViewerItem | | | Viewer / ASViewerItem > No there are a lot > of different ways to display features: draw a box around the actual > sequences, draw the sequence as a box with features in different colors, > draw filled in boxes under the sequence, etc... So once a user decides > what features he wants displayed and how, we need to be able to store that > info so we can rebuild the figure. I was thinking that the custom ViewerItems (canvas objects) would be composite/derived GnomeCanvasItems. So, you only need to say (a little PyGNOME here): annotated_seq = canvas.group.add('ASViewerItem', ) And the specifies just how the annotated sequence will look. For example, this is a text item I added to a canvas group on the Workspace: locus.group.text = locus.group.add('text', text=locus.label, x=text_x, y=text_y, font=font, fill_color_gdk=self.color_fg, anchor=ANCHOR_NORTH) We can make a whole set of arguments for an annotated sequence or any other graphical bioinformatics object. This information can be stored/passed in a markup document. > I think your confusing what you are > calling the motif document with what I was calling the markup document. > The markup document would not have any motif/feature data. It would only > contain the information that the figure builder would need to reconstruct > the figure. Oh, okay. > Some other questions: > > 1) In your workspace, each line is a data pipe. Will loci (the Yes. > program) be able to store that data and essentially restart the data pipe > from that point? That's a good question. I thought about that while coding the Workspace, and it seems I may need another XML to store an entire WorkFlow Diagram, in case it needs to be restarted. The Workspace can even be something the user can New/Open/Save/Close as if it were a wordprocessing document. I think this should be tied into the whole "Electronic Notebook" idea, since the EN and the Workspace will be managing the same information. Interesting. Any thoughts? > 2) Do you envision loci managing other data besides strict biodata > (ie data on how to reconstruct the data flows, data on how to build > displays/figures, ...) This may be related to the first question, but yes. There will be lots and lots to manage. Perhaps Loci will make the most use of different XML's of all other programs before it. I think everything will be linked, even different XML's, and this would be done by cross-referencing. At the highest level we will have the Workspace, and at the lowest level, maybe the biodata. Loci will be a big challange for some XML "X-perts". > 3) Is there are good post/document/whatever that outlines the > philosophy/architecture for loci? Loci has evolved a great deal. I started with some sketchy concepts about a year ago, which are still present in the current model. But the actual implementation of the concepts was never put in stone. Justin has been with the project for a while and set up the original mailing list. The archive is still up. There's lots of design talk in those old messages. It's probably a good way to get the feel for where Loci is going. http://toaster.sped.ukans.edu/tulip-list/ But again, so much has changed. We'll just have to see how it turns out in the end :-) Everytime I try to write something, it's incomplete because the implementation is unclear, and then everything changes a week later after a few messages have been passed. And often just doing some coding takes us down a different path, as with the Workspace. Some people are very good at planning every move. I hope that leaving a few things up in the air will allow for some innovation and creativity. For example, I think you contributed a great deal to the design already :-) Some of what we've been talking about were not very clear to me before. Cheers. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Wed Jun 23 00:51:29 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] GST, widgets & figure building References: <37706520.BB0E61C9@bc.edu> Message-ID: <377067D1.B707CAF5@bc.edu> "Alan J. Williams" wrote: > > The ones that are not connected to data, are they added by the user? I was > > thinking, to keep things simpler, the viewers that are part of the workspace or > > workflow diagram don't act as fully functional drawing programs. IOW, the > I think this is where I am confused. So your thinking with loci is flow > centric where the view I am working with is data centric. So what type of > data flows do you envision Loci supporting. One-to-one, one-to-many, > many-to-one? I guess I misread this the first time. I wouldn't say my thinking is NOT data centric. Afterall, I'm saying functions are always performed ON a document (data): A Modifier modifies data, a Translator translates data, etc. You could say that the Workspace tends to emphasize flow, having all those arrows. But maybe my view is a little more flow oriented. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Wed Jun 23 01:28:38 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] Data Storage Interfaces References: Message-ID: <37707086.D0F79579@bc.edu> Ah, I finally get to this message. "Alan J. Williams" wrote: > > Jeff wrote: > > (2) GST requires Oracle DB; Loci uses PAOS and a simple DB > > I guess I didn't present this clear enough. My thought for GST was to > define an interface for data retrieval and storage and then provide > several standard data manager plug-ins that implement this interface. That sounds like a good idea to me. > Iinitially it would probably be a local filesystem plug-in and a db > plug-in [either mysql or acedb].) Yeah, I'd go with a free database. There is also MSQL and PostgreSQL (and BerkeleyDB?). I'm not sure about AceDB, but PostgreSQL has a BSD license; MSQL and MySQL are not free for commercial use. I was just thinking that Oracle would be a huge prerequisite for a program. > In many ways, it sounds simmilar to what > is being talked about for Loci. I guess the only difference is that the > GST approach would be limited to local plug-in based access (However, a > plug-in that utilized CORBA could be implemented). The advantage from what > I can tell of using a local plug-in approach as opposed to a PAOS approach > is that you are not tied to a specific transfere technology, you are only > limited in that the endpoint/startpoint from the client side must be a > plug-in that implements the DM interface. Hmmm. I think I get it now. So, DM is almost an abstraction layer for plug-ins, an "interface" or API? > > And the GST DM is something like Loci's server/daemon, "Locid". > Sort of (I am probably not being careful about consistently using the term > DM). So the approach would be: > 1) A well defined DM interface (which opperates on the client side) > 2) DM Plug-in(s) which implement the DM Interface (These also opperate > on the client side) > 3) Depending on the DM Plug-in, various backend or middle end > "servers" may need to be implemented. > So with Loci, it sounds like the middle end and backend "servers" > technologies are fixed, where with a plug-in approach only the plug-in > interface in fixed. Well, I'm thinking of something that any locus can talk to at any time and that can provide a seamless interface across networks. I'm not sure how this would work is the deamon is really deamons, and some might be there and some might not be. It seems like you would still need some central manager to keep track of what is what, which is really what Locid would do. > > In Loci, a single XML document "travels" a workpath, so everything is > > done serially (within one path). The document will collect various XML's > > So do you mean that the original data, and analysis results, etc will all > accumulate in one xml document? This doesn't sound like a good idea, but I > am probably misunderstanding what you are saying. Why might this be a bad > idea (I am thinking as I write, so don't flame me too hard ;o): Flame flame flame. I guess the document "conceptually" travels the workpath, but not really. I've been working with concepts mostly. In reality, there may be better approaches. > 1) Complicates data locking in a multi-user model > 2) Increases server load by forcing parsing of un-needed data (See recent > post from bioperl with comment on server side XML parsing). If > parsing isn't done on the server side, then you have the issue of > having to transfere all that combined data. > It sounds better to me to just implement a robust cross referencing > mechanism, assume each data object to be just one "item" (ie blast > results, or a sequence, or a restriction map). Then let the backend server > store the data as it sees fit (ie as one huge flat file, as individual > database entries, as individual files, ...) I agree. This is probably what we'll do. > > But as far as Loci is concerned, can we make it so that the XML types > > (DTD's) are not hard-coded into Loci? What if each locus were > > responsible for finding its own XML parser/translator? That would > > pretty much make Loci a general purpose command-line wrapper. > > Definately! To take it to a further extreme, Loci should provide the xml > parsing and a SAX like interface to the data. Then each loci doesn't have > to implement an XML parser, only the SAX callbacks to handle the XML data. > Then loci could handle any XML data for which a locus is provided and it > could even handle data in the absence of a DTD. (The advantage of the SAX > interface over passing the whole tree, is that memory isn't wasted on > building parts of the tree that aren't needed.) I am not really familiar > with the SAX interface, but hints could be provided so that only elements > the locus wants are passed through the SAX callbacks. We agree again :-) > > Justin's comments are basically what I was thinking. Rather than passing > a program all the data, just pass it a reference (maybe an XLink/XPointer) > to the input data. Under the plug-in model, plug-ins for networked data > could implement a cache mechanism to speed up access. > It sounds like we could be working on the same project. Enough of Loci's data management model is still unimplemented and up in the air, that we could come up with something by working together on this. So, we'll use DM's, if everyone likes the idea. I'm flexible, and this isn't The Bizzaro Project :-) If we work together, it would just be more likely we'll have something successful. We spoke before about licensing issues, and the problem was that if Loci uses the GPL, it could not be extended with something that has your own license (APL - Alan's Public License :-). But if Loci uses the LGPL (Lesser GPL), which we could if we don't include straight GPL parts, you could extend Loci with your own stuff and use whatever license you want. The LGPL is the same license that the Gtk+ toolkit uses, and you're using that already. Personally, I'd love to see Loci extended and find its way into all sorts of places and uses. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Wed Jun 23 01:50:58 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] Data Storage Interfaces References: Message-ID: <377075C2.44DB49@bc.edu> "Alan J. Williams" wrote: > Iinitially it would probably be a local filesystem plug-in and a db > plug-in [either mysql or acedb].) I'm not really up on databases, but is there a DB that uses XML as a data format? Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From pmr at sanger.ac.uk Wed Jun 23 04:03:27 1999 From: pmr at sanger.ac.uk (Peter Rice) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <37707086.D0F79579@bc.edu> (bizzaro@bc.edu) References: <37707086.D0F79579@bc.edu> Message-ID: <199906230803.IAA13317@unst.sanger.ac.uk> Jeff Bizzaro wrote: >"Alan J. Williams" wrote: >> Iinitially it would probably be a local filesystem plug-in and a db >> plug-in [either mysql or acedb].) >Yeah, I'd go with a free database. There is also MSQL and PostgreSQL (and >BerkeleyDB?). I'm not sure about AceDB AceDB is freely available. See http://www.sanger.ac.uk/Software/Acedb/ -- ---------------------------------------------------------------------- Peter Rice | Informatics Division, The Sanger Centre, E-mail: pmr@sanger.ac.uk | Wellcome Trust Genome Campus, Tel: (44) 1223 494967 | Hinxton, Cambridge, CB10 1SA, England Fax: (44) 1223 494919 | URL: http://www.sanger.ac.uk/Users/pmr/ From justin at ukans.edu Wed Jun 23 12:55:49 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <377075C2.44DB49@bc.edu> Message-ID: > > Iinitially it would probably be a local filesystem plug-in and a db > > plug-in [either mysql or acedb].) > > I'm not really up on databases, but is there a DB that uses XML as a data > format? I've heard of one commercial one, and supposedly Oracle is going to do some XML stuff also. This is one thing I would like to write, actually. It's just a database that can deal with XML structure/relationships. I've seen a few papers talking about it (they call it semi-structured data). Then XQL, XML-Q?, or whatever the w3c comes up with for an XML query language. Also, a DOM and SAX interface would be good, too. I'll see if I can find any references, again. Justin From justin at ukans.edu Wed Jun 23 13:25:40 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:39 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: Message-ID: > I've heard of one commercial one, and supposedly Oracle is going to do > some XML stuff also. > > This is one thing I would like to write, actually. It's just a database > that can deal with XML structure/relationships. I've seen a few papers > talking about it (they call it semi-structured data). Then XQL, XML-Q?, or > whatever the w3c comes up with for an XML query language. Also, a DOM and > SAX interface would be good, too. > > I'll see if I can find any references, again. This looks like a good place to start, if anyone is interested: http://www-db.stanford.edu/lore/research/ Justin From hortiz at neurobio.upr.clu.edu Wed Jun 23 14:30:26 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: Your message of "Mon, 21 Jun 1999 23:20:52 GMT." <376EC8D4.AD62D4AA@bc.edu> Message-ID: <199906231830.OAA08663@chimbo.neurobio.upr.clu.edu> > > I like your point which reinforces the idea that the editor should simply > > be used to define features, not how they are displayed. > > Me too. Good, now just keap repeating after me "the data is separate from the markup, the data is separate from the markup, the data is separate from the markup, ..." > Will we then be directly mixing graphical markup with biodata markup? Not if I can help it. > > Another way (with the rendering widget idea) is that each rendering widget > > can register whether or not it can handle dependent markup. > > The rendering widget will store the characteristics of > > the arrow in the markup object. The rendering widget will also create a > > new feature in the sequence data object using the position and id info in > > addition it will prompt the user for a label as well as a residue to > > anchor the arrow to. Yes, this is right. And the only widget that can do this is the rendering widget. It has to understand the data (a multiseq object) and the markup (a markup object) and be able to combine them. > How will the graphical feature be marked in the sequence XML? It won't, it'll be stored in a separate markup XML (remember: "the data is separate from the markup, the data is separate from the markup, the data is separate from the markup, ...") > > Now when an old Markup Data Object is to be rendered (or a Figure Builder > > Data Object), figure builder will go through the list of markup rendering > > each one. When it gets to an unknown item type, it will create the > > appropriate rendoring widget to handle the display. > > I'm a little foggy on how the viewer ("figure builder") will create a viewer > item ("rendering widget") it knows nothing about. It can ask the locid who registered as being able to render multiseq objects. It actually can be simpler than this though. I've been beating myself up trying to figure out how to get the figure builder to be able to produce a multisequence figure without duplicating the multisequence editor. This is exactly what the koffice and gnome office suites do, for a quick overview, see: http://www.mieterra.com/article/koffice.html http://www.gnome.org/white-papers/Components/Components/ Forget office suites, and think: kpresenter is the figure builder, kspreadsheet is the multisequence editor, kdiagram is a multisequence renderer. So you can think of editing a multisequence in a small app, then dropping the multisequence onto a blank figbuilder canvas. clicking on the multisequence figure in the figbuilder will allow us to use the multisequence editor menus to manipulate the figure. This is how embedding charts and figures into MS-Word works, and is doable with the corba-based tools in KOffice and in the gnome toolset (I'm not suggesting we move to KDE, the same facilities are available in gnome, so it's just a matter of getting the python bindings set up). By the way, pygnome already links to the ORBit libraries, are there bindings available to access any of this? > There is A LOT of new functionality coming to Gnome and even Mozilla (SVG for > example) that is very much like what we need for Loci. A point though that I > would like to make is that these features are so general-purpose, it would be as > difficult using them to embed a component into Loci, as making a new stand-alone > program (a reason for not using Loci). In addition, they are made with C/C++, > which would have to be wrapped in Python. It would be much simpler using only > Python. (The same argument though can be used against CORBA and in favor of > PAOS.) But corba buys us much of the hard parts of the loci model: embeding active figures in a figure builder, and exporting a program's functions to a remote site. We'll get this for free, by just setting up python bindings to the already available tools, plus we get a whole pile of developers working on the ugly issues and lets us get back to what we do best, bioinformatics programming. A good set of python bindings to ORBit would allow us to build a locus by just subclassing from LocusComponent and adding some bindings for our application specific functions. -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From hortiz at neurobio.upr.clu.edu Wed Jun 23 14:57:12 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA Message-ID: <199906231857.OAA08839@chimbo.neurobio.upr.clu.edu> First an apology: I know I haven't really contributed anything to loci except a lot of hot air, but I've been busy, and for a while I thought I had bitten off more than I could chew with this project. I hope to contribute some real programs soon, I'm learning to use python (with Tkinter for now) and am starting to learn to use pygnome to write image viewers with the gtk canvas and imlib. As I said, for a while, I thought I had bitten off more than I could chew with this project. What I wanted to do was develop a working gui shell for the free sequence analysis apps, like GCG, or gde, or SeqPup. gnome-python seemed like a good fit, so I was happy when I found tulip. When I start reading the mailing list, I'm appaled to find out that we need to have network aware apps, and workflow, and Gateways and Hubs, and PAOS, and lion and tigers and bears, oh my! The things we've discussed make for a huge system. Paos and the locid in particular look nasty. This is where I almost turned back and decided to write my own simple shell using unix pipes and simple stuff like that. Now, just so you grok where I'm coming from, I don't much like gnome, it seems big and slow and bloated, even compared to KDE. I've mainly heard bad things about CORBA too, even from the gnome developers, who scrapped MICO in favor of ORBit because it was too big and slow. Having said that, a lot of other biowidget groups are into CORBA in a big way, as are the KDE and gnome developers. I think we should give it serious consideration for loci too, heres why: I'm quoting liberally from the Gnome Component white paper: http://www.gnome.org/white-papers/Components/Components/gnome-corba.html "CORBA is used in GNOME to export the internal engine of an application to the rest of the system. Any CORBA client (GNOME compliant applications, regular Unix applications, remote clients running on a different operating system) can use the services that these application provides." Sounds like loci right? "In addition to exporting the API, GNOME is generating a list of standard interfaces that other applications can be written to. For example, the interface Desktop::Editor defines an interface to an editor. This means that an application (such as a Mail client or an IDE program) that is developed using these interfaces can decouple itself from a specific implemenation of the interface." This solves the problem of how to register a locus, Locus::MultisequenceEditor specifies a generic interface, which GST or any other locus can export, the user can then choose which loci to use to handle a request for a Locus::MultisequenceEditor. "CORBA provides a standard, and easy to use Remote Procedure Call (RPC) system that allows different applications to communicate with each other." Just like we want to in loci. "Bonobo is the GNOME Document model. It lets document-based applications embed themselves in each other. Its design is influenced heavily on OLE2 and Active X by Microsoft and has a similar functionality. It is currently in Alpha state, and should be used only by developers, but has been used successfully to embed a Guppi graph within Gnumeric. For those unfamiliar with OLE2, it provides a framework for applications to embed themselves within the same GUI framework. This facilitates, for example, a graph to be embedded in a word-processing document, or a spell checker to be embedded in a plotting program. Bonobo also provides the ability to wrap GTK-style objects around a component, allowing an application developer to use one in his application. A more technical description is available in the BONOBO cvs repository." Damn, they've done it again. This is just how we want the figure builder to work. We could reinvent the wheel in developing loci, but it's best if we at least know what everyone else's wheels look like first. We've got most of the hard parts of loci done, let's try building loci over the top of the gnome CORBA layer, and if it's too slow then consider (a fixing ORBit, or b rewriting just the parts of CORBA we need). -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From Alan at TheWilliamsFamily.org Wed Jun 23 18:14:51 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <377075C2.44DB49@bc.edu> Message-ID: Having the database able to serve up parsed xml would probably make for a substantial speed improvement. On the other hand, do we really want to lock all loci users into installing a DBM? Probably not, so either way there needs to be an abstraction layer between loci and the data storage implementations. On a related note, I wonder how difficult it would be to map xml data onto an object oriented relational database system such as postgresql? On Wed, 23 Jun 1999, J.W. Bizzaro wrote: > I'm not really up on databases, but is there a DB that uses XML as a data > format? ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ------------------------------------------------------------------------ "But as for me and my house, we will serve the Lord." Joshua 24:15 ************************************************************************ From justin at ukans.edu Wed Jun 23 18:35:29 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <37707086.D0F79579@bc.edu> Message-ID: > > Definately! To take it to a further extreme, Loci should provide the xml > > parsing and a SAX like interface to the data. Then each loci doesn't have > > to implement an XML parser, only the SAX callbacks to handle the XML data. > > Then loci could handle any XML data for which a locus is provided and it > > could even handle data in the absence of a DTD. (The advantage of the SAX > > interface over passing the whole tree, is that memory isn't wasted on > > building parts of the tree that aren't needed.) I am not really familiar > > with the SAX interface, but hints could be provided so that only elements > > the locus wants are passed through the SAX callbacks. > > It sounds like we could be working on the same project. Enough of Loci's data > management model is still unimplemented and up in the air, that we could come up > with something by working together on this. So, we'll use DM's, if everyone > likes the idea. I'm flexible, and this isn't The Bizzaro Project :-) If we > work together, it would just be more likely we'll have something successful. I think we both had the same plans, anyway. However, rather than just a SAX interface, I want DOM, too, and possibly others. I think the DOM level 2 spec is shaping to handle what I want. With SAX, a callback function is registered, and then the parser calls this function with each tag of XML it finds. The callback function does what it likes with the data. Instead, we could pass a certain filter, and only those nodes would be returned, but then we're getting into DOM2. Plus, DOM2 is supposed to eventually describe a query method, which will be even better. DOM2 also describes ranges, which could be used to mark off chunks of the data (data from a specific analysis, for instance). DOM2 even describes notification events Now, even if it's a DOM object, the whole thing does not have to be passed over the network. Remember DOM is really just an interface description. What's actually behind the interface is arbitrary. Now what if we do this -- We describe a protocol to provide access to a DOM object over a network. GNOME is (or may have already) doing this for CORBA. An IDL is provided which describes the DOM interface. It doesn't matter what the object is, how big, or how it is stored on the other side of the CORBA interface, and most importantly, the client does not have to transfer the whole object across the ORB to use it. It just calls the functions (via the ORB). We can do the same thing over TCP/IP, if necessary, too. So, we have some kind of object storage, which has our _virtual_ DOM interface over some kind of back-end (be it a big tree in memory, that Lore XML database, AceDB, Oracle, or some kind of caching, seekable XML parser). These are the central repositories of objects. Other things connect to them and take/put little pieces that they want. A remote object could create it's own object representation of the data, or just rely on the object store to hold it over the network. A really smart one of these things would be obviously integrated with the Workshop components. Whenever some operation was "committed" beyond the specific tool in use, it would update this object. There might have been a notification signal sitting on that specific node, too, which would then cause some other tool to get a DOM notification (the details here are still sketchy, as the DOM2 spec is still an early draft). This would be the central hub of Loci communication. Now, is this what you mean by DM (data manager, right?)? This is kind of like the workflow server (or was it system) I described back in the early days, but after reading the DOM2 ideas, it's starting to take shape. Also note, the DOM2 stuff is just an interface. Consider the concept for now. We could just as easily do the same thing in a different interface we made up (but why?) and really, there's no reason why the virtual interface on each end has to be the same (as long as they speak the low-level network protocol). This could potentially allow for a little more distributed, roaming path for the object. A client to a store could really be another store itself (where a store is simply a program somewhere which presents a virtual interface to some data it has), taking bits of data from other places and presenting a new interface. Justin From justin at ukans.edu Wed Jun 23 19:30:57 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA In-Reply-To: <199906231857.OAA08839@chimbo.neurobio.upr.clu.edu> Message-ID: > When I start reading the mailing list, I'm appaled to find out that we need to > have network aware apps, and workflow, and Gateways and Hubs, and PAOS, and > lion and tigers and bears, oh my! It's really not as bad as it looks. Really. > The things we've discussed make for a huge system. Paos and the locid in > particular look nasty. This is where I almost turned back and decided to write > my own simple shell using unix pipes and simple stuff like that. I'm currently at Genentech in the Biocomputing department, and that's basically where things are now (except with a web front-end and a sprinkle of Java). I'd like Loci to be a next generation kind of project, but really, I think we have more complicated interfaces, but not a whole lot more programming than doing it the old way. > We could reinvent the wheel in developing loci, but it's best if we at least > know what everyone else's wheels look like first. We've got most of the hard > parts of loci done, let's try building loci over the top of the gnome CORBA > layer, and if it's too slow then consider (a fixing ORBit, or b rewriting just > the parts of CORBA we need). As we are making use of GNOME, I would agree that it makes sense to use their technology for these things. It makes all of the tools more flexible in the GNOME environment. However, there has to be some flexibility in the interface (to the data in particular). What if I want to make a tool that uses data in a unique way (which we don't have an interface for)? I guess, fundamentally, I'm interested in Loci from a data storage, management, and interface issue. I'm thinking of a system which can pull data from various sources in a very flexible way and then combine and present that data in an equally flexible way to any arbitrary program. Justin From Alan at TheWilliamsFamily.org Wed Jun 23 20:08:13 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA In-Reply-To: Message-ID: On Wed, 23 Jun 1999, Justin Bradford wrote: > I'm currently at Genentech in the Biocomputing department, and that's > basically where things are now (except with a web front-end and a sprinkle > of Java). I'd like Loci to be a next generation kind of project, but > really, I think we have more complicated interfaces, but not a whole lot > more programming than doing it the old way. I've toyed with using bioperl and writing an apache mod_perl system for doing bioinformatics, but you can only do so much through that medium. Non-the-less, it would be great if the underlying infrastructure of loci could be utilized through a mod_perl or cgi interface should someone decide to implement a web interface down the road. > As we are making use of GNOME, I would agree that it makes sense to use > their technology for these things. It makes all of the tools more > flexible in the GNOME environment. Agreed. > However, there has to be some flexibility in the interface (to the data in > particular). What if I want to make a tool that uses data in a unique way > (which we don't have an interface for)? This is why the data access/storage needs to have good/general abstraction. > I guess, fundamentally, I'm interested in Loci from a data storage, > management, and interface issue. I'm thinking of a system which can pull > data from various sources in a very flexible way and then combine and > present that data in an equally flexible way to any arbitrary program. One of my frustrations as a researcher is all the bits of sequence data, blast reports, restriction maps, structure predictions, etc.. that are strewn over many directories and accross five computers. Furthermore, what sequence did this blast report come from? GDE does a decent job of providing an interface to the many programs (if you go to all the trouble of putting together the gdemenus and writing glue scripts) but it does a horrible job at managing data. If Loci is going to be the next generation bioinformatics package, it is going to have to have good (and flexible) data management. But hey, you already knew that. Just adding my support to Justin's Comments. ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From bizzaro at bc.edu Thu Jun 24 01:55:53 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA References: Message-ID: <3771C869.D99ABB67@bc.edu> "Alan J. Williams" wrote: > > I've toyed with using bioperl and writing an apache mod_perl system for > doing bioinformatics, but you can only do so much through that medium. > Non-the-less, it would be great if the underlying infrastructure of loci > could be utilized through a mod_perl or cgi interface should someone > decide to implement a web interface down the road. You see, we are thinking about the same thing. We discussed doing just this a few months back. Rahul volunteered to take the "Web interface" on. The idea is, if we are going to create a communication network for lots of command-line tools, why not let people tap into parts of it (what can be tapped into) with a Web browser. It may actually answer the portability problem to some extent: If you can't run Unix/Linux, try the Web interface. My thoughts were that a CGI interface could control what the Workspace might control. So everything in the Workspace that would require user input, would instead bring up an HTML form. And everything that would return an figure, would convert the figure to GIF or JPEG and make an HTML page for it. > > As we are making use of GNOME, I would agree that it makes sense to use > > their technology for these things. It makes all of the tools more > > flexible in the GNOME environment. > Agreed. Me too. If we aren't going 100% Python, we need to look at other toolsets, and Gnome puts it all together in a nice little package. > > However, there has to be some flexibility in the interface (to the data in > > particular). What if I want to make a tool that uses data in a unique way > > (which we don't have an interface for)? > This is why the data access/storage needs to have good/general > abstraction. Yeah. The general idea is that we can plug in new components to manage new data types. This will in the long run make Loci a general purpose...whatchamacallit, but maybe more of a thingamajiggy or even a thingamabob ;-) There isn't a name for this sort of thing, is there? Cheers. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Thu Jun 24 02:27:20 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA References: <199906231857.OAA08839@chimbo.neurobio.upr.clu.edu> Message-ID: <3771CFC8.5FE51599@bc.edu> Humberto Ortiz Zuazaga wrote: > > First an apology: I know I haven't really contributed anything to loci except > a lot of hot air, but I've been busy, and for a while I thought I had bitten > off more than I could chew with this project. Same here. I've been contributing hot air for over a year :-) > I hope to contribute some real programs soon, I'm learning to use python (with > Tkinter for now) and am starting to learn to use pygnome to write image > viewers with the gtk canvas and imlib. Cool. I'm obviously getting into "PyG" by working on the Workspace, and I really like it. I have nothing but good things to say about Python and Gnome/Gtk. I think if you try the Gnome canvas, you'll like it too. > When I start reading the mailing list, I'm appaled to find out that we need to > have network aware apps, and workflow, and Gateways and Hubs, and PAOS, and > lion and tigers and bears, oh my! It has evolved quite a bit, and I think this is because we stumbled across a whole new software frontier...or at least something that hasn't been done before :-) > Now, just so you grok where I'm coming from, I don't much like gnome, it seems > big and slow and bloated, even compared to KDE. I've mainly heard bad things > about CORBA too, even from the gnome developers, who scrapped MICO in favor of > ORBit because it was too big and slow. That has to be the most common knee-jerk criticism that I hear about software people just don't like for some reason. When was the last time someone said they like such-and-such because it is small and fast, and just how much is such-and-such really capable of doing? > Having said that, a lot of other > biowidget groups are into CORBA in a big way, as are the KDE and gnome > developers. Yeah, I have seen that, along with Java and Perl. > "CORBA is used in GNOME to export the internal engine of an application to the > rest of the system. Any CORBA client (GNOME compliant applications, regular > Unix applications, remote clients running on a different operating system) can > use the services that these application provides." > > Sounds like loci right? A simpler approach would be to just say, "Hey this is Python so live with it." And we could just wrap some command-lines in Python and tada! But how successful can Loci really be if we ignore the masses of Perl, Java and C++ developers? So, let's look into CORBA. ... > "CORBA provides a standard, and easy to use Remote Procedure Call (RPC) system > that allows different applications to communicate with each other." > > Just like we want to in loci. I agree. > "Bonobo is the GNOME Document model. It lets document-based applications embed > themselves in each other. Its design is influenced heavily on OLE2 and Active > X by Microsoft and has a similar functionality. It is currently in Alpha > state, and should be used only by developers, but has been used successfully > to embed a Guppi graph within Gnumeric. This is the screenshot I mentioned the other day. > For those unfamiliar with OLE2, it provides a framework for applications to > embed themselves within the same GUI framework. This facilitates, for example, > a graph to be embedded in a word-processing document, or a spell checker to be > embedded in a plotting program. Bonobo also provides the ability to wrap > GTK-style objects around a component, allowing an application developer to use > one in his application. A more technical description is available in the > BONOBO cvs repository." > > Damn, they've done it again. This is just how we want the figure builder to > work. You're in a sarcastic mood :-) Pretty much. Of course having it so that a Loci figure can be embedded in a word processing doc would be swell. > We could reinvent the wheel in developing loci, but it's best if we at least > know what everyone else's wheels look like first. We've got most of the hard > parts of loci done, let's try building loci over the top of the gnome CORBA > layer, and if it's too slow then consider (a fixing ORBit, or b rewriting just > the parts of CORBA we need). Great. Let's have at it. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Thu Jun 24 02:52:26 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] Xterminal Message-ID: <3771D5AA.1203F836@bc.edu> This might be of some interest: http://www.linuxsupportline.com/~Xterminal/manual/index.html Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From hortiz at neurobio.upr.clu.edu Thu Jun 24 10:42:05 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA In-Reply-To: Your message of "Thu, 24 Jun 1999 06:27:20 GMT." <3771CFC8.5FE51599@bc.edu> Message-ID: <199906241442.KAA18518@chimbo.neurobio.upr.clu.edu> > > Now, just so you grok where I'm coming from, I don't much like gnome, it seems > > big and slow and bloated, even compared to KDE. I've mainly heard bad things > > about CORBA too, even from the gnome developers, who scrapped MICO in favor of > > ORBit because it was too big and slow. > > That has to be the most common knee-jerk criticism that I hear about software > people just don't like for some reason. When was the last time someone said > they like such-and-such because it is small and fast, and just how much is > such-and-such really capable of doing? Well, I really like the flwm window manager because it is small and light. It even manages windows. It's my window manager of choice at home and for my VNC sessions. I like rxvt too, so there. > > "CORBA is used in GNOME to export the internal engine of an application to the > > rest of the system. Any CORBA client (GNOME compliant applications, regular > > Unix applications, remote clients running on a different operating system) can > > use the services that these application provides." > > > > Sounds like loci right? > > A simpler approach would be to just say, "Hey this is Python so live with it." > And we could just wrap some command-lines in Python and tada! But how > successful can Loci really be if we ignore the masses of Perl, Java and C++ > developers? So, let's look into CORBA. Well, that's exactly my point. If we were going to just write a simple shell over command line apps, we could just use python. If we want to do all the other neat things we either have to: 1) write our own 100% python custom loci RPC, registry, xml, ... tools 2) write python bindings to standard, in use, well maintained tools 3) scrap python and write C/C++/java apps I think #2 is our best bet, heck you wouldn't catch me dead writing bioinformatics apps in a low level language, and #2 is simpler than #1, don't you think? -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From hortiz at neurobio.upr.clu.edu Thu Jun 24 10:52:05 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: Your message of "Wed, 23 Jun 1999 17:35:29 EST." Message-ID: <199906241452.KAA18589@chimbo.neurobio.upr.clu.edu> justin@ukans.edu said: > However, rather than just a SAX interface, I want DOM, too, and > possibly others. I think the DOM level 2 spec is shaping to handle > what I want. Guys, another white paper on gnome: http://www.gnome.org/white-papers/DOM/domination.html This one talk about "sliding windows" and DOM, as a way of not keeping a whole DOM tree in memory, and ways the gnome project uses DOM with a different (more efficient) internal representation. -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From justin at ukans.edu Thu Jun 24 13:23:32 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <199906241452.KAA18589@chimbo.neurobio.upr.clu.edu> Message-ID: > Guys, another white paper on gnome: > > http://www.gnome.org/white-papers/DOM/domination.html > > This one talk about "sliding windows" and DOM, as a way of not keeping a whole > DOM tree in memory, and ways the gnome project uses DOM with a different (more > efficient) internal representation. >From my reading, it sounds like the sliding part is mostly concerned with reducing the number of objects (and their corresponding mappings) generated due to the CORBA interface. This does help of course, but I still think the whole tree is in memory. I had intended to use the gdome spec for the CORBA interface to DOM, anyway, but I'd like to be able to swap out the actual back-end, as well. I had envisioned a more lightweight method, as well. I'll read more about it, and email the author of the white paper. I think an additional client-side optimization could provide the same effect as their sliding windows, and still keep the standard DOM interface. Also, I expect an additional need to map DOM over TCP/IP, but that won't prevent us from using gdome. Justin From Alan at TheWilliamsFamily.org Thu Jun 24 13:23:24 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: <37706520.BB0E61C9@bc.edu> Message-ID: On Wed, 23 Jun 1999, J.W. Bizzaro wrote: > Yes. Yes. Yes. Yes. Yes. What were you thinking? I guess we are on different > planes of quantum reality, but we'll hammer things out ;-) I was just thinking of the data as the objects and the programs/modifiers/translaters/etc... as the lines. > That pretty much sums it up. Except, Viewers are widgets+ that handle > ViewerItems, which are graphical objects, and Figure Builder is one "big" Viewer > running as a separate application. The user can then copy/paste or drag/drop > multiple graphical objects (ViewerItems) onto the FB. This allows the user to > choose only the Loci-generated figures they want for an illustration. What were > you thinking of? Is there anything wrong with this approach? That's what I was thinking as well. I guess the issue is how do we go about implementing this. > I agree with you. I'm not sure why you think I combined the Translator and > Viewer. Maybe saying "ViewerTranslator" is confusing. I only wanted to show > that the Translator is specifically for Viewers. So, ViewerTranslator == > Translator. Sorry, I deleted the diagram. I think I finally see what you mean by translators and their role in Loci! See below... > You mean something like a composite widget? Basically what I was calling the ViewerItemWrapper is the combination of data objects and widget objects into one self-contained object that could be embeded into the Main Viewer. I think this is what you are think as well. > Yep. This whole "what format is best for object transfer?" problem is what > Justin and Humberto have been working on. We'll have to come up with something > soon. Somebody in one of the other recent posts mentioned that DOM2 allows for filtering of the elements on the server side. It sounds like a combination of SAX/DOM2 might be the way to go (although I know next to nothing about DOM). The next question is whether we provide a single interface to the XML data or several? > That sounds good. (Technically though a gene prediction program would be a > "Modifier" that is linked to a Translator ;-) ... > (Translators sandwich everything.) Now this has nothing to do with a Viewer, > but if the user wants to add a Viewer, they have to add these loci: Oh, so that is what translators are! But it would seem to me that we don't need translators to sandwich everything. For instance an viewer for sequences should understand sequence documents directly. > We can make a whole set of arguments for an annotated sequence or any other > graphical bioinformatics object. This information can be stored/passed in a > markup document. Just to be sure we are on the same track, by "information" you mean the "how to display feature X information" which goes in the markup document and the "where/what is feature X" goes in the annotated sequence document. > That's a good question. I thought about that while coding the Workspace, and it > seems I may need another XML to store an entire WorkFlow Diagram, in case it > needs to be restarted. The Workspace can even be something the user can > New/Open/Save/Close as if it were a wordprocessing document. I think this > should be tied into the whole "Electronic Notebook" idea, since the EN and the > Workspace will be managing the same information. Interesting. Any thoughts? So we have the workspace which is used for: 1) Setting up a new analysis flow 2) Track currently running analysis flow Now for the notebook we want to track what analysis were run and what documents were generated. So we have a xml fragment for the analysis that was run. This allows us to redo the analysis, restart the analysis, find what docs the analysis produced, modify the analysis. Furthermore, when setting up the flow, we should be able to insert a document type to indicate that we want a permanent record of that data (even if it is in the middle of an analysis flow). So for instance: Seq Doc --> Digest Mod --> Digest Map Viewer would show a digest map to the user, where Seq Doc --> Digest Mod --> Digest Doc --> Digest Viewer would not only show the map, but store a copy of the results. In a perfect world you wouldn't need to store the itermediate, you could just regenerate it from the flow info in the notebook, however lets say there was (God forbid) a bug in the Digest Mod that caused a restriction site to not be recognized. So I run the analysis, figure out what enzymes to use, do my subcloning, get the new clone sequence, and and find out that I have the wrong section of sequence. I go back to show my PI the initial analysis I did but because the bug was fixed, my digest now shows the extra site! This is why we need to be able to document not just how to generate the data, but we need to be able to store and track the data that is generated. > This may be related to the first question, but yes. There will be lots and lots > to manage. Perhaps Loci will make the most use of different XML's of all other > programs before it. I think everything will be linked, even different XML's, > and this would be done by cross-referencing. At the highest level we will have > the Workspace, and at the lowest level, maybe the biodata. Loci will be a big > challange for some XML "X-perts". Sounds good. > Justin has been with the project for a while and set up the original mailing > list. The archive is still up. There's lots of design talk in those old > messages. It's probably a good way to get the feel for where Loci is going. > > http://toaster.sped.ukans.edu/tulip-list/ Thanks. > But again, so much has changed. We'll just have to see how it turns out in the > end :-) Everytime I try to write something, it's incomplete because the > implementation is unclear, and then everything changes a week later after a few > messages have been passed. And often just doing some coding takes us down a > different path, as with the Workspace. > > Some people are very good at planning every move. I hope that leaving a few > things up in the air will allow for some innovation and creativity. For > example, I think you contributed a great deal to the design already :-) Some of > what we've been talking about were not very clear to me before. I agree. Planning is essential, but there is a lot that changes when I start to actually code. Inevitably I will get a project half coded and then decide that I did it total wrong. Then I start coding all over. ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From Alan at TheWilliamsFamily.org Thu Jun 24 13:38:13 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: <37707086.D0F79579@bc.edu> Message-ID: On Wed, 23 Jun 1999, J.W. Bizzaro wrote: > Yeah, I'd go with a free database. There is also MSQL and PostgreSQL (and > BerkeleyDB?). I'm not sure about AceDB, but PostgreSQL has a BSD license; MSQL > and MySQL are not free for commercial use. I was just thinking that Oracle > would be a huge prerequisite for a program. Yeah. For the single user or small lab user any relational database may be too much to setup and maintain. With loci I think we want to allow for data access and storage to/from robust relational databases, but it isn't the job of loci to setup/maintain/administer those databases. > Hmmm. I think I get it now. So, DM is almost an abstraction layer for > plug-ins, an "interface" or API? Exactly! A data access abstraction layer! Of course this abstraction layer should be designed with the data passing through the workflow in mind. Data exchange through the workflow could be accomplished through the same API as documents are stored and retrieved by. > Well, I'm thinking of something that any locus can talk to at any time and that > can provide a seamless interface across networks. I'm not sure how this would > work is the deamon is really deamons, and some might be there and some might not > be. It seems like you would still need some central manager to keep track of > what is what, which is really what Locid would do. This is one of the parts I haven't thought out yet. How do we keep track of and advertise data sources. CORBA has this functionality, but do we want to force all data sources to use CORBA? > It sounds like we could be working on the same project. Enough of Loci's data > management model is still unimplemented and up in the air, that we could come up > with something by working together on this. So, we'll use DM's, if everyone > likes the idea. I'm flexible, and this isn't The Bizzaro Project :-) If we > work together, it would just be more likely we'll have something successful. I am exited enought about the goals of Loci that I will focus my efforts on Loci and not GST. Having said that, I am in my last year and have a huge dissertation to write up! Ug! Needless to say, I am not sure how much I can commit to. (Although coding for Loci would be infinitely more exciting than writing a dissertation!) > We spoke before about licensing issues, and the problem was that if Loci uses > the GPL, it could not be extended with something that has your own license (APL > - Alan's Public License :-). But if Loci uses the LGPL (Lesser GPL), which we > could if we don't include straight GPL parts, you could extend Loci with your > own stuff and use whatever license you want. The LGPL is the same license that > the Gtk+ toolkit uses, and you're using that already. When we talked about license before, I wasn't really thinking about this angle, but I think your right. We may need to look at licensing the core components of loci under the LGPL and the different modifiers, viewers, etc, under the GPL. I am sure that the GNOME and KDE developers have already had to deal with this. I think the GNOMEish way is to use BONOBO to connect modules. > Personally, I'd love to see Loci extended and find its way into all sorts of > places and uses. Heck, I just want a great bioinformatics package that newbies can jump right into and old-timers can twist around. Oh yeah, anything that will make the windows and mac users green with envy wouldn't be too bad either. ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From Alan at TheWilliamsFamily.org Thu Jun 24 13:45:03 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: <199906231830.OAA08663@chimbo.neurobio.upr.clu.edu> Message-ID: On Wed, 23 Jun 1999, Humberto Ortiz Zuazaga wrote: > Good, now just keap repeating after me "the data is separate from the markup, > the data is separate from the markup, the data is separate from the markup, > ..." for(i=1;i!=0;i++) printf("the data is separate from the markup\n"); > Not if I can help it. Ditto! > Yes, this is right. And the only widget that can do this is the rendering > widget. It has to understand the data (a multiseq object) and the markup (a > markup object) and be able to combine them. Absolutely! It is also the only object to understand what actions the user can perform on it and what things can be altered in regards to it's presentation. > > How will the graphical feature be marked in the sequence XML? > > It won't, it'll be stored in a separate markup XML (remember: "the data is > separate from the markup, the data is separate from the markup, the data is > separate from the markup, ...") Separate Markup Data File or Figure Data File > It can ask the locid who registered as being able to render multiseq objects. > It actually can be simpler than this though. I've been beating myself up > trying to figure out how to get the figure builder to be able to produce a > multisequence figure without duplicating the multisequence editor. Sounds good. > http://www.mieterra.com/article/koffice.html > http://www.gnome.org/white-papers/Components/Components/ I'll have to check these out. Sounds like I need to bone up on Bonobo, DOM2, and CORBA. If only I had a couple extra hours each day. ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ------------------------------------------------------------------------ "But as for me and my house, we will serve the Lord." Joshua 24:15 ************************************************************************ From Alan at TheWilliamsFamily.org Thu Jun 24 13:56:17 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] Data Storage Interfaces In-Reply-To: Message-ID: On Wed, 23 Jun 1999, Justin Bradford wrote: > However, rather than just a SAX interface, I want DOM, too, and possibly > others. I think the DOM level 2 spec is shaping to handle what I want. A well laid out interface to the data is important, but do we really need to provide multiple interfaces. For instance, we could standardize on the SAX interface, but provide function calls and data structures so that somebody developeing a loci could easily build his/her own tree. I don't know anything about DOM level 2. Guess I will have to head over to the OMG web site. > With SAX, a callback function is registered, and then the parser calls > this function with each tag of XML it finds. The callback function does > what it likes with the data. Instead, we could pass a certain filter, and > only those nodes would be returned, but then we're getting into DOM2. This is exactly what I was thinking (but had no clue that DOM2 addressed it). It is essential that we be able to provide filters to speed up data access across the network. > Plus, DOM2 is supposed to eventually describe a query method, which will > be even better. DOM2 also describes ranges, which could be used to mark > off chunks of the data (data from a specific analysis, for instance). > DOM2 even describes notification events Ok, now I am drooling. (There must be a catch.) > We describe a protocol to provide access to a DOM object over a network. > GNOME is (or may have already) doing this for CORBA. An IDL is provided > which describes the DOM interface. It doesn't matter what the object is, > how big, or how it is stored on the other side of the CORBA interface, > and most importantly, the client does not have to transfer the whole > object across the ORB to use it. It just calls the functions (via the > ORB). We can do the same thing over TCP/IP, if necessary, too. So could we use the GNOME DOM2 Interface IDL as a starting point for developing a data access abstraction layer or API for loci. > So, we have some kind of object storage, which has our _virtual_ DOM > interface over some kind of back-end (be it a big tree in memory, > that Lore XML database, AceDB, Oracle, or some kind of caching, > seekable XML parser). These are the central repositories of objects. > Other things connect to them and take/put little pieces that > they want. A remote object could create it's own object representation > of the data, or just rely on the object store to hold it over the > network. Sounds just like what I was thinking. > A really smart one of these things would be obviously integrated > with the Workshop components. Whenever some operation was "committed" > beyond the specific tool in use, it would update this object. There > might have been a notification signal sitting on that specific node, > too, which would then cause some other tool to get a DOM notification > (the details here are still sketchy, as the DOM2 spec is still an > early draft). This would be the central hub of Loci communication. > > Now, is this what you mean by DM (data manager, right?)? Yep. > This is kind of like the workflow server (or was it system) I > described back in the early days, but after reading the DOM2 ideas, > it's starting to take shape. Also note, the DOM2 stuff is just an > interface. Consider the concept for now. We could just as easily > do the same thing in a different interface we made up (but why?) > and really, there's no reason why the virtual interface on each > end has to be the same (as long as they speak the low-level > network protocol). > > This could potentially allow for a little more distributed, roaming > path for the object. A client to a store could really be another > store itself (where a store is simply a program somewhere which > presents a virtual interface to some data it has), taking bits of > data from other places and presenting a new interface. ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From Alan at TheWilliamsFamily.org Thu Jun 24 13:59:23 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA In-Reply-To: <199906241442.KAA18518@chimbo.neurobio.upr.clu.edu> Message-ID: On Thu, 24 Jun 1999, Humberto Ortiz Zuazaga wrote: > Well, that's exactly my point. If we were going to just write a simple shell > over command line apps, we could just use python. If we want to do all the > other neat things we either have to: > > 1) write our own 100% python custom loci RPC, registry, xml, ... tools > > 2) write python bindings to standard, in use, well maintained tools > > 3) scrap python and write C/C++/java apps My vote is for #2 or #3. ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From Alan at TheWilliamsFamily.org Thu Jun 24 14:05:09 1999 From: Alan at TheWilliamsFamily.org (Alan J. Williams) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA In-Reply-To: <3771C869.D99ABB67@bc.edu> Message-ID: On Thu, 24 Jun 1999, J.W. Bizzaro wrote: > You see, we are thinking about the same thing. We discussed doing just this a > few months back. Rahul volunteered to take the "Web interface" on. The idea > is, if we are going to create a communication network for lots of command-line > tools, why not let people tap into parts of it (what can be tapped into) with a > Web browser. It may actually answer the portability problem to some extent: If > you can't run Unix/Linux, try the Web interface. I came across the thread and am in agreement. > My thoughts were that a CGI interface could control what the Workspace might > control. So everything in the Workspace that would require user input, would > instead bring up an HTML form. And everything that would return an figure, > would convert the figure to GIF or JPEG and make an HTML page for it. So it is important that viewers (and possibly the figure builder) be able to not only present to the screen, but present to paper (postscript?), and present to graphics (PNG or JPEG). Of course ghostscript will convert postscript to PNG,JPEG,... > > This is why the data access/storage needs to have good/general > > abstraction. And it is also why the workflow, notebook, etc ... all need to have an xml representation. > Yeah. The general idea is that we can plug in new components to manage new data > types. This will in the long run make Loci a general purpose...whatchamacallit, > but maybe more of a thingamajiggy or even a thingamabob ;-) There isn't a name > for this sort of thing, is there? Yep. An xml document workspace!? ************************************************************************ Alan Williams ------------------------------------------------------------------------ University of California, Riverside "Where observation is concerned, Dept. of Botany and Plant Sciences chance favors the prepared mind." Alan@TheWilliamsFamily.org -- Louis Pasteur ************************************************************************ From bizzaro at bc.edu Thu Jun 24 23:31:50 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:40 2006 Subject: [Pipet Devel] Gary Van Domselaar joins! Message-ID: <3772F826.4DB9F04B@bc.edu> Guys, I want to welcome Gary Van Domselaar to the team! He's coming to us from the BioTools project, and would like to help with Loci coordination and documentation. Well, he wrote an introduction below, and you've got to check out the logos and GIF animations he's been working on. Really neat stuff. Jeff. ------------- Locians, My name is Gary Van Domselaar (http://redpoll.pharmacy.ualberta.ca/~gvd). I'm interested in joining the Open Lab to work on the Loci project. I am currently doing Ph.D. studies in pharmaceutical biotechnology and drug delivery at the University of Alberta in Edmonton, Alberta, Canada. I have been involved in bioinformatics for about four years now. I wrote a good deal of the documentation for PepTool (proteomics) and GeneTool (genomics) for BioTools (http://www.biotools.com). I think I may be able to contribute to the Loci project as assistant coordinator [ie Jeff's flunkie :-)], documentation author, web page development, and hopefully some gui hacking, once I get up to speed on python and gtk. For my first task I think I should probably flesh out the web site a bit. What would you like to see? I personally like the GNOME web page design (http://www.gnome.org), which is quite a bit like the loci pages are now. For starters I propose the following structure: Main Page - Logo, blurb, and News. User Info Introduction to Loci Tour of Loci Download Documentation Developer Info Introduction (Joining, Background Info & Development Tools) Loci Architecture & Systems CVS instructions (CVS server?) Mailing List This structure is based loosely on the gnome web pages. I will be attending the American Peptide Symposium from June 26 - July 1, so I can get started when I get back. I made a couple Loci logos last week when i was playing with the GIMP and POVRAY. You can check them out at http://redpoll.pharmacy.ualberta.ca/~gvd/loci/ Give me your input and I will GIMP up an 'official' logo when I get back >from Minneapolis. Gary Van Domselaar -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Gary Van Domselaar gvd@redpoll.pharmacy.ualberta.ca Faculty of Pharmacy Phone: (780) 492-4493 University of Alberta FAX: (780) 492-5305 Edmonton, Alberta, Canada http://repdoll.pharmacy.ualberta.ca/~gvd/ From hortiz at neurobio.upr.clu.edu Thu Jun 24 23:48:37 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: Your message of "Thu, 24 Jun 1999 10:23:24 MST." Message-ID: <199906250348.XAA23201@chimbo.neurobio.upr.clu.edu> > > That pretty much sums it up. Except, Viewers are widgets+ that handle > > ViewerItems, which are graphical objects, and Figure Builder is one "big" Viewer > > running as a separate application. The user can then copy/paste or drag/drop > > multiple graphical objects (ViewerItems) onto the FB. This allows the user to > > choose only the Loci-generated figures they want for an illustration. What were > > you thinking of? Is there anything wrong with this approach? > > That's what I was thinking as well. I guess the issue is how do we go > about implementing this. Ah, but now I think the implementation is actually doable. Look at this qute from the World DOMination white paper: http://www.gnome.org/white-papers/DOM/domination.html Individual renderers take a fragment of the document tree as an argument, and return a DOMination Rendering object, which contains size info and a Gnome Canvas item. The renderer also installs event listeners at this time, so that when the fragment changes, the Canvas item and size info automatically get updated. Does this sound like a viewer locus to you? The only trick left is to think of the Workplace and the figbuilder as two different views into the same object, let's call it an experiment DOM tree. Figbuilder asks for all final figure fragments and calls viewers to produce figure canvas objects for display, and the workspace calls all loci and asks them to render their control boxes and editor widgets. The trick is we can do all this (and more) by just slightly enhancing gdome and adding python wrappers. From hortiz at neurobio.upr.clu.edu Thu Jun 24 23:55:53 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] Notebook and storing workflow. In-Reply-To: Your message of "Thu, 24 Jun 1999 10:23:24 MST." Message-ID: <199906250355.XAA23251@chimbo.neurobio.upr.clu.edu> > > That's a good question. I thought about that while coding the Workspace, and it > > seems I may need another XML to store an entire WorkFlow Diagram, in case it > > needs to be restarted. The Workspace can even be something the user can > > New/Open/Save/Close as if it were a wordprocessing document. I think this > > should be tied into the whole "Electronic Notebook" idea, since the EN and the > > Workspace will be managing the same information. Interesting. Any thoughts? I think this is right on the money. > Furthermore, when setting up the flow, we should be able to insert a > document type to indicate that we want a permanent record of that data > (even if it is in the middle of an analysis flow). So for instance: > > Seq Doc --> Digest Mod --> Digest Map Viewer > > would show a digest map to the user, where > > Seq Doc --> Digest Mod --> Digest Doc --> Digest Viewer > > would not only show the map, but store a copy of the results. > > In a perfect world you wouldn't need to store the itermediate, you could > just regenerate it from the flow info in the notebook, however lets say ... > the extra site! This is why we need to be able to document not just how to > generate the data, but we need to be able to store and track the data that > is generated. I think we may have to store _all_ intermediates by default, with the option of pruning intermediates (but never final results), and may well want to always keep everything, (like a real lab notebook). From bizzaro at bc.edu Fri Jun 25 00:17:30 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA References: Message-ID: <377302DA.F13302C5@bc.edu> "Alan J. Williams" wrote: > > On Thu, 24 Jun 1999, Humberto Ortiz Zuazaga wrote: > > Well, that's exactly my point. If we were going to just write a simple shell > > over command line apps, we could just use python. If we want to do all the > > other neat things we either have to: > > > > 1) write our own 100% python custom loci RPC, registry, xml, ... tools > > > > 2) write python bindings to standard, in use, well maintained tools > > > > 3) scrap python and write C/C++/java apps > > My vote is for #2 or #3. #3?! You've got to give Python a chance, Alan :-) I was originally hoping for something like #1, but #2 really makes more sense. Cheers. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From hortiz at neurobio.upr.clu.edu Fri Jun 25 00:30:14 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] An apology, Loci, gnome, and CORBA In-Reply-To: Your message of "Thu, 24 Jun 1999 11:05:09 MST." Message-ID: <199906250430.AAA23450@chimbo.neurobio.upr.clu.edu> > > My thoughts were that a CGI interface could control what the Workspace might > > control. So everything in the Workspace that would require user input, would > > instead bring up an HTML form. And everything that would return an figure, > > would convert the figure to GIF or JPEG and make an HTML page for it. > So it is important that viewers (and possibly the figure builder) be able > to not only present to the screen, but present to paper (postscript?), and > present to graphics (PNG or JPEG). Of course ghostscript will convert > postscript to PNG,JPEG,... If all viewers render to a gnome canvas, can't we just dump that to a file? I think the gnome printing api will even allow rendering to memory without opening an X window (I sure hope so). From bizzaro at bc.edu Fri Jun 25 01:17:30 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] graphical API's (was: An apology...) References: <199906250430.AAA23450@chimbo.neurobio.upr.clu.edu> Message-ID: <377310EA.62B919F4@bc.edu> > > > My thoughts were that a CGI interface could control what the Workspace might > > > control. So everything in the Workspace that would require user input, would > > > instead bring up an HTML form. And everything that would return an figure, > > > would convert the figure to GIF or JPEG and make an HTML page for it. > > So it is important that viewers (and possibly the figure builder) be able > > to not only present to the screen, but present to paper (postscript?), and > > present to graphics (PNG or JPEG). Of course ghostscript will convert > > postscript to PNG,JPEG,... The Gnome canvas is supposed to eventually have printing facilities. I may have seen somewhere, but I'm not sure about this, that this will be handled via p/ghostscript. Humberto Ortiz Zuazaga wrote: > > If all viewers render to a gnome canvas, can't we just dump that to a > file? I think the gnome printing api will even allow rendering to > memory without opening an X window (I sure hope so). Yeah. Hopfully we can just tap into whatever printing API the canvas gets, and convert it to JPG or GIF. One other thing though, that we haven't mentioned recently. I don't mean to stir things up any more, but we should keep in mind one other drawing toolset for Loci: OpenGL. I was orginally thinking about using OpenGL alone, but there are no bindings yet to the GtkGLArea widget. Also, having worked with the Gnome canvas for the Workspace, I realized that it gives us a lot of high-level 2D capabilities that would be hard to do with OpenGL. So we'll start with the Gnome canvas, but I'd like to have a 2-part plan: (1) Use Gnome canvas for all 2D schematics. (2) Use OpenGL for all 3D models, where a 3D effect is needed. Duh. Pretty obvious, huh? But let's please keep OpenGL on the backburner. We have 3 guys who actually wanted to work with OpenGL for Loci: Rob Harrison, SooHaeng Yoo, and Greg Waltz. :-) Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From hortiz at neurobio.upr.clu.edu Fri Jun 25 12:18:32 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] XML-RPC Message-ID: <199906251618.MAA10213@chimbo.neurobio.upr.clu.edu> Info on a new release of XML-RPC is at: http://lwn.net/1999/0624/a/xml-rpc.html This is an alternative way of calling remote programs using xml data, and is implemented in python. In case we do decide CORBA is too big for our needs. -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From justin at ukans.edu Fri Jun 25 13:00:07 1999 From: justin at ukans.edu (Justin Bradford) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] GST, widgets & figure building In-Reply-To: <199906250348.XAA23201@chimbo.neurobio.upr.clu.edu> Message-ID: > The only trick left is to think of the Workplace and the figbuilder as > two different views into the same object, let's call it an experiment > DOM tree. Figbuilder asks for all final figure fragments and calls > viewers to produce figure canvas objects for display, and the > workspace calls all loci and asks them to render their control boxes > and editor widgets. > > The trick is we can do all this (and more) by just slightly enhancing > gdome and adding python wrappers. I've joined the gdome mailing list and sent email to author of the white papers. After my DSL line is installed today, I'll be able to grab whatever gdome source is available. gdome looks pretty cool, and I think we can make it perfect for Loci. We'll need Python bindings for ORBit, so I'll look into that also. Also, I'm going to implement a gdome interface over tcp/ip, so not all tools will need to be CORBA apps (which will make the remote tool development easier [just some minor scripting]). Justin From hortiz at neurobio.upr.clu.edu Sat Jun 26 22:39:56 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] I am pretty, oh so pretty... In-Reply-To: Your message of "Sat, 19 Jun 1999 08:47:52 GMT." <376B5938.7E43CDFA@bc.edu> Message-ID: <199906270239.WAA24288@chimbo.neurobio.upr.clu.edu> > I merged the best of the GUI paradigms I've been toying with the past month or > so. Now each locus has a box with a drop shadow, an icon, and a label. Box > shape, color, icon and label can all be specified. > > Attached is a screenshot. You can see the popup menu (from a right mouse button > click) over one of the loci, which is also black because it is selected. Below > another locus is a windowlet, which you can show/hide by double-clicking in the > box. I checked out loci-core and loci-widgets from CVS, but can't get anything like the screenshot. Can you give a step by step? From bizzaro at bc.edu Sun Jun 27 09:53:19 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] I am pretty, oh so pretty... References: <199906270239.WAA24288@chimbo.neurobio.upr.clu.edu> Message-ID: <37762CCF.2777B279@bc.edu> Humberto Ortiz Zuazaga wrote: > > > I merged the best of the GUI paradigms I've been toying with the past month or > > so. Now each locus has a box with a drop shadow, an icon, and a label. Box > > shape, color, icon and label can all be specified. > > > > Attached is a screenshot. You can see the popup menu (from a right mouse button > > click) over one of the loci, which is also black because it is selected. Below > > another locus is a windowlet, which you can show/hide by double-clicking in the > > box. > > I checked out loci-core and loci-widgets from CVS, but can't get > anything like the screenshot. Can you give a step by step? You only need loci-core from CVS. I just found out that I forgot to add an icon. It should work now. loci-core requires the following packages: gnome-libs 1.0.10 gnome-python 1.0.3 If you don't have these, the workspace will crash from time to time or might not work at all. loci-widgets is not needed yet. Just checkout loci-core, and run "loci.py". It should be an executable. Pressing the right mouse button will bring up a popup menu for adding new loci. Cheers. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From hortiz at neurobio.upr.clu.edu Sun Jun 27 14:54:01 1999 From: hortiz at neurobio.upr.clu.edu (Humberto Ortiz Zuazaga) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] Updating gnome-libs and gnome-python to run loci In-Reply-To: Your message of "Sun, 27 Jun 1999 13:53:19 GMT." <37762CCF.2777B279@bc.edu> Message-ID: <199906271854.OAA28478@chimbo.neurobio.upr.clu.edu> > > > Attached is a screenshot. > > I checked out loci-core and loci-widgets from CVS, but can't get > > anything like the screenshot. Can you give a step by step? > > You only need loci-core from CVS. I just found out that I forgot to add an > icon. It should work now. > > loci-core requires the following packages: > > gnome-libs 1.0.10 > gnome-python 1.0.3 Got it, I had the rpm packages for gnome, that only have gnome-libs 1.0.8, and gnome-python 1.0.1 (1.0.3 won't compile on the 1.0.8 libs). Updating these is simple, just download, configure, make, then rpm -e gnome-libs --nodeps, make install, and ldconfig. Loci now works, well, at least I can make pretty pictures too! -- Humberto Ortiz Zuazaga Bioinformatics Specialist Institute of Neurobiology hortiz@neurobio.upr.clu.edu From bizzaro at bc.edu Wed Jun 30 10:55:50 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] Help: ORBit, Python and PyGNOME References: Message-ID: <377A2FF5.AD03AA69@bc.edu> > From: David Arnold > alternatively, both ILU and Orbit (and Mico?) have Python bindings as > well, so you could consider them if Fnorb doesn't work out. Can you tell us where we might find Python bindings to ORBit? I haven't heard that they even exist. Also, James, it appears that you started to link PyGNOME to the ORBit libraries. Are you planning on doing anything more with PyGNOME and the GNOME object models: ORBit, GDOME, XML-libs? PyGNOME would be even better if it included Python bindings to these things, as you may already have planned. Can you fill us in on what you want to do? This is very important to a project I coordinate. Cheers. Jeff -- J.W. Bizzaro mailto:bizzaro@bc.edu Boston College Chemistry http://www.uml.edu/Dept/Chem/Bizzaro/ -- From bizzaro at bc.edu Wed Jun 30 12:17:14 1999 From: bizzaro at bc.edu (J.W. Bizzaro) Date: Fri Feb 10 19:18:41 2006 Subject: [Pipet Devel] [Fwd: [pygtk] Re: ORBit stuff] Message-ID: <377A430A.A6DD3842@bc.edu> This is James Henstridge's reply to my questions about Python bindings to all the Gnome object stuff. Apparently nothing exists. But on the bright side, this is all rather new, and Python support will probably be one of the first to come along...Jeff -------------- next part -------------- An embedded message was scrubbed... From: James Henstridge Subject: [pygtk] Re: ORBit stuff Date: Wed, 30 Jun 1999 23:28:51 +0800 (WST) Size: 3355 Url: http://bioinformatics.org/pipermail/pipet-devel/attachments/19990630/b4b5c1ee/attachment.mht From david at coconut-palm-software.com Sat Jun 19 09:50:24 1999 From: david at coconut-palm-software.com (David Orme) Date: Fri Feb 10 19:18:43 2006 Subject: Meta-Application Interface: a proposal References: <199906171842.OAA19301@egret.cbl.umces.edu> Message-ID: <376BA020.EE778964@coconut-palm-software.com> Ferdinando Villa wrote: > <> > This leads easily to the idea of a 'meta-application > interface' as a core desktop functionality, where users can 'patch' > applications together graphically (connecting their icons with the > mouse), save their meta-application to the desktop, and use it as a > single app. At the very simplest level this allows to use the UNIX > pipe mechanism graphically: e.g. to print text files as postscript, I > create a new meta-app (a box with input and optionally output ports), > drop the ascii-to-ps translator and the ps-printer icons into it, > connect the meta-app input to the translator and the translator to the > printer, assign an icon and save it. Now I can drop my ascii file on > the icon and have it translated and printed in one step. It's not much > of a deal but you get the idea - try and find something like that on > Windows or MacOS. One of the most powerful features of Unix comes to > the friendly desktop. Myself, I would love to have it. > > Going up with the complexity, you can carry this to a very high > and very new level of power. As a first stab at it: through an API, > applications can notify their 'output ports' and make them available > as outputs in a meta-application context. This translates in users > being able to select graphically the ports from the application's icon > in the meta-application editor. The 'semantics' of the info that > passes through there can be described in XML and the connections, of > course, can be based on CORBA, which makes it possible to use remote > apps ('services') as well. An application in the meta-application can > play a 'lead role' by showing its own interface when running the > meta-app, and is notified of the connected apps through the API, > adjusting its menus and functions (possibly directed from similar info > set in the connected apps) according to what services are available to > it in the meta-application context. Specific 'GUI' apps can offer > specific services like e.g. interactive output port selection: you > drop the Ascii file on the meta-printing app and you get the pop-up > asking which printer/viewer you want it to print on (all printers are > connected to the outputs of the switcher). This is a good idea. It overlaps somewhat with existing work, although I am not aware of existing work that uses XML as the data modelling language. The Bonobo component architecture will let you hook software components together using CORBA--the components are able to read or write using a specified CORBA interface, and you can hook up compatible components in a pipelike way. Another PhD project I'm aware of that was similar to this was Cube, done at Univ. of Illinois. It is much lower level than what you are trying to do but many of the concepts are similar and the bibliography should be useful to you. Here's the URL for the project: http://www.research.digital.com/SRC/personal/Marc_Najork/cube.html Also, the Gnome-filer project (soon to be renamed) aims to provide a programming language-neutral IDE, specifically designed to let graphical languages like what you describe "plug in". Basically, our idea is to provide a graphical project and build manager plus a pluggable API for developing graphical editors for parts of programs. These editors won't be limited to just editing data entry forms, but could graphically edit an object specification, database schema, or maybe graphically "glue" components together using pipes or any other relevant abstraction to edit applications. It sounds like your idea might plug nicely into our framework. Also, for text editing, Gnome-filer will use the Gnome::Editor Bonobo interface so when Emacs, Vim, Joe, etc., are upgraded to support this interface, you will be able to work comfortably in your favorite editor *and* have graphical app editing tools available to you. I'm the maintainer of the Gnome-filer project and if you're interested in working with us, let me know. Currently there is no project home page (until we get ready for an initial developer's release), but our source code is stored in the Gnome-filer module in the Gnome CVS repository. For more information about how things work, our goals, etc., consult the TODO file in the repository and maybe compile the current code. (The current code compiles against the Gnome libs included in Red Hat 6.) > In any case, thanks for reading, and best to you all Wish you the best! Regards, Dave Orme -- David Orme LINUX... david@coconut-palm-software.com ...it already goes there http://www.coconut-palm-software.com :-) ----- End of forwarded message from David Orme ----- --------------6C85F3E51361B397F2E8C308--