[Pipet Devel] excerpt from CHANGES

J.W. Bizzaro bizzaro at geoserve.net
Mon Jan 31 10:54:58 EST 2000

Brad Chapman wrote:
> Thanks for writing, Jeff and Gary--I got lots o' questions/comments on
> this:

I just sent a reply to Gary's message.  Maybe that will answer some of your
questions here.  But I'll continue...

> I don't know about you, but if I look at my /usr/local/bin directory
> there are hundreds of programs there, all ordered in a non-useful
> alphabetical fashion. Jumbled in with the useful programs are a whole
> crapload of randomprogram-config programs which I never need to run,
> etc. I personally do not want to look through a list of all these
> programs just to run a program.

Well, I was using /usr/bin as an example.  As I mentioned in that message, the
default (root level) directories (and thus containers) are...

    * loci/back/public/container/
    * loci/back/private/container/

Of course, for security we can't have a container for /usr/bin.  Using our
restricted sandbox directories is a lot like what some other apps are using. 
Sendmail, for example, now has its own bin directory AND its own shell.

> > Well, if containers are directories, then the user can sort loci
> > however they want.  They just DnD a locus into a different
> > container.  EXACTLY like files and folders in a Mac hard drive.
> I agree with this for files, but I don't see how this would work for
> programs, if we arejust finding them all in the /usr/bin directory. I
> would not expect users to add sub-directories to their /usr/bin, sort
> the programs by hand, and then adjust their PATH to move into all
> these directories. Or, would moving a program loci to a different
> container only create a link to that program in the filesystem, and it
> would look to loci like the program is actually in the new folder?

Again, we're not going to use /usr/bin.  Sorry about that confusion.  But how
a user will 'import' a program from /usr/bin (or wherever) into our sandbox
directories, is a good question.  I think it would be best for the sysadmin to
_manually_ create symlinks from /usr/bin/progname to the sandboxes.  Maybe
later we can work on a more automatic 'import' method.

> > 'ls', for example, is a 'processor' type locus, but it doesn't
> > _contain_ anything.  It _represents_ ls, and that's all.
> Okay, gotcha, only containers contain other loci! But then, why would
> you ever want to add a processor directly to the workspace, as opposed
> to dragging it out of a container?

Just to 'work with it' more easily.  Hence the name 'workspace' :-)  But,
yeah, in the Loci model, the loci (wrapper and all) stay put...in their
original 'location'.  Only the _representation_ (XML!) is dragged to the
workspace.  Get it?  :-)

> Should the default gui for a
> processor by a selection widget to choose the program it represents?

Should the default gui for a processor _have_ a selection widget to choose the
program it represents?  We're talking about a 'blank processor' (program name
never selected), right?  Ummm.  I was thinking about just having a edit box
where the user/developer can type in the program name.  Remember, for
command-line construction, we're just adding strings:

    command_name = 'ls'
    flag_name = '-l'
    file_name = '*'
    command = executable_name + flag_name + file_name
    print command

>>> ls -l *


> > So, now you're probably asking, 'What about the wrappers for
> > seqManip?  Where do they go?'  Well, my thoughts are that Loci
> > will (1) identify the location of the file, and (2) then check
> > the 'wrappers' directory (back.private.wrappers?) for a wrapper.
> > (3) If no wrapper is found, then (4) Loci uses a default wrapper
> > based on file type (executables are processors, etc.) or (5)
> > tries to make its own (as you were doing with man file parsing
> > in middle.library.modules.wrap).
> Okay, so for 4, what would a default wrapper look like? Would it have
> flag options?

No.  A 'flag' type locus is added by the user, to the workspace, and then the
lines are connected.  Tada!  You now have processor + flag.

> Would it specify input and output files?

Regarding I/O, that should be done via connecting lines and keeping everything
on the workspace.  Now that you mention it, the 'filename' type that I
mentioned IS the 'document' type :-P  I forgot about that.

***Okay, nix what I said in the last message about typing in filename. 
Connecting a document to the command-line that is under construction, is how
filenames should be determined.  Loci will have to supply the command-line
with the actual location of the file, if it can't be piped in.


    grep 'hello' myfile.txt

should be constructed from this WFD:

 +------------+      +-----------+      +-------+
 |  document  | ---> | processor | ---> | regex |
 +------------+      +-----------+      +-------+
 | myfile.txt |      |   grep    |      | hello |
 +------------+      +-----------+      +-------+

which should be interpreted by Loci as...

    cat myfile.txt | grep 'hello'

But let's say grep can't take pipes or work from stdin.  Then Loci would have
to interpret the same WFD as...

    grep 'hello' myfile.txt

and IF myfile.txt were not in our sandbox directories, Loci would have to get
it from its locale, save it in a temporary file and do...

    grep 'hello' /home/jdoe/loci/middle/temp/tempfile372873

> I really think
> that the chances of being able to develop a "standard" wrapper that
> will work for many programs is very slim.

Oh no, it's quite simple.  All I'm thinking about for an 'unknown executable'
is a processor that spits out the _name_ of the executable.  You've got a
symlink to povray in your sandbox directory; Loci doesn't know povray from
jack; Loci says, 'Eh, just spit the name povray out on the command-line'. 
That's all.  The user/developer can then work on defining it better later on.

> For 5, please notice that I'm not automatically parsing man files, I
> made the one for ls by hand. I just don't want you to expect too much.

Yeah, I realize that was a suggestion I made based on an assumption that it
was easy to do.

> Also, man files seem to be getting outdated, as more and more info on
> programs move to the web--it seems like this strategy will miss a lot
> of programs.

Hmmm.  True.

> What I am proposing is not to try and have a "catch all" mechanism to
> try and get every program, but instead to require an xml file for
> every program you want to run. If we are using Loci as an analogy to a
> web browser, I think of this xml files as analagous to an html page
> for the info you want to display. I don't really think that a gui
> interface to a program will work well at all without some information
> about a program. Otherwise, the gui interface is just as unintuitive
> as using the command line, and then, why use the gui interface?

Yes, but the experienced user/developer can _construct_ WFD's and _composite_
them, making something simple for the inexperienced user/nondeveloper.  I want
Loci to be able to accommodate _both_ types of users.

When I started designing Loci, I thought we'd have two main programs: (1) the
workspace and (2) an interactive development environment (IDE).  But then I
realized that most IDE's and most programming languages organized information
into 'connectivity trees' (WFD's), which is what the workspace was supposed to
be good at doing.  So, SHAZAAM, the workspace took on the responsibility of
also being an IDE and a programming language.  And the concept of
'compositing' loci/WFD's made this possible by hiding the complexity of the
low-level stuff.

>     This is the way things work for appLab with their meta-data files.

XML files, sure!  But they start out simple, are then built up via WFD
construction and composting, and then end up large and descriptive.

> However, I think a big problem with AppLab is that there is no way to
> get one of these meta-data files for programs that have already been
> wrapped. To remedy this situtation, I think it would be relatively
> easy to have a repository of xml files written for programs available
> along with Loci,

Yeah, but the idea behind Loci is that this repository can be anywhere on the

> and some script that automatically installs them for
> the user.

Simply open a container pointing to the remote filesystem, and get what you

> Then the user can download all of the xml files they want
> and automatically install them.
>     Am I making any sense?

Yes, automagic retrieval of XML representations is a major, fundamental design
feature of Loci.

(I'll continue this in another e-mail.)

                      |           J.W. Bizzaro           |
                      |                                  |
                      | http://bioinformatics.org/~jeff/ |
                      |                                  |
                      |           THE OPEN LAB           |
                      |    Open Source Bioinformatics    |
                      |                                  |
                      |    http://bioinformatics.org/    |

More information about the Pipet-Devel mailing list