[Pipet Users] Re: making more CLI programs Piper-compliant

J.W. Bizzaro jeff at bioinformatics.org
Tue May 22 11:07:49 EDT 2001

(late reply)

Roland Walker wrote:
> There is absolutely a better way.  Don't parse the help output.  This
> is harder than doing things the nominally hard way.
>    1 First, you must wrap some CLI elements by hand.  This is needed
>      to get started, yes? and you are doing it already, yes?  Wrapping
>      basically means standardizing the handling of STDIN, STDOUT,
>      and STDERR, trying to tame buffering, and providing some
>      formalization of the options.  This is not too hard; it's my own
>      stock in trade.

Actually, we've come up with a way to do the wrapping using connect-the-dots:


Sure, programmers will find it simpler to write one by hand, but perhaps
non-programmers will find it usable.  In any case, it's been an academic
excercise to see how many things can be done using connect-the-dots.

>    2 Provide an _easy_ way for CLI authors to export options, making
>      makes their programs drop-in elements for you.  This means not
>      writing yet another standard, which authors will just ignore, but
>      giving authors the needed code.  This is not as hard as it seems.

Is there ANY standard out there?  Can you provide us with an example?

>      You won't have to conquer SEALS, as I'll just give it to you.


> I assume you have written an XML spec to define the options?  That's
> what I would have done if XML was around when I started my thing.

Sort-of.  It's a long story.  Basically, we already *CAN* define options using
XML, but that is due to the dataflow nature of Piper.  We should probably come
up with clear specs though.

> Give me your spec and a give me a couple of hours, and I can give you
> a Getopt::Piper that is drop-in compatible with Getopt::Long.


> I'm not claiming this is perfect, just useful as heck.  Cram new
> programs in to your system as hard as you can.  If you reach a certain
> critical mass, all kinds of stuff will start to come your way.

That's our motto :-)

> There are dozens of scripts here at NCBI that use the same library for
> argument processing.  For some reason argument processing and CLI
> usability is an obsessive interest of mine.  Anyway, I build a data
> structure that holds more than anyone could want to know about
> processing the options for a CLI script.

You're the man to talk to about this then!

> Then I like to do more with
> that than merely process the options.  For instance, each script can
> generate programmable completions for tcsh based on its expected
> arguments:

For Piper, grepping what the program needs for parameters is more important
than grepping what the parameters do.  Piper is "dumb" that way: it doesn't
care what data passes through it; it's the responsibility of the node/program
to deal with it (although we'd like to help the user make the right

> So to you Piperians, I say, show me your spec, I'll throw it into
> the main SEALS library so that all scripts can be queried, say,
> like this
>    gi2fasta -piper_options
> to return your spec, giving you a sizable wad of new CLI elements
> for your system.

Again, I'd be interested in seeing what others have come up with and if there
may be some sort of a standard approach.  We'd be interested in promoting a
standard (say for GNU programs).

> There is certainly some duplication, but duplication
> can even be good.  I am not into the idea of a single perfect
> system, or a single best way, but rather in opening up new ways
> which are new paths for users to wander.

The old standards vs. freedom to innovate argument :-)

> Furthermore, you may have our module if you find it useful.  I'd say
> the module is wonderful, though idiosyncratic.  Fixing it for general
> use means mostly indenting and deleting.


> In addition, the research side at the NCBI is a powerfully active
> testbed.  If you could get folks using Piper here, you would get
> a _lot_ of feedback.
> Think critical mass.

>From the bioinformatics side of Piper development, I can say that it would be
a Great Thing to have NCBI involved :-)

> So I think that the only hitch between us and a world of excitement
> is the lamentable lack of a packaging system that can tame the SEALS
> monster.  We have multiple revisions we need to reconcile, and
> tangles of code that we rather urgently need to clean up and push
> out the door in some usable form.

Are you thinking along the line of RPM, Deb, or CPAN?

Thanks for the message.  It was very helpful.

J.W. Bizzaro                                jeff at bioinformatics.org
Director, Bioinformatics.org        http://bioinformatics.org/~jeff
"As we enjoy great advantages from the inventions of others, we
should be glad of an opportunity to serve others by any invention
of ours; and this we should do freely and generously."
                   -- Benjamin Franklin

More information about the Pipet-Users mailing list