[Pipet Devel] Understanding Overflow + Getting programs to work under piper

Brad Chapman chapmanb at arches.uga.edu
Fri May 26 19:06:27 EDT 2000

Hey everyone;
	I've been on a mission to try and understand how Overflow works so
that we can start to get it integrated into Piper. Towards this goal,
Jean-Marc was kind enough to help me work through some of the areas where I
was stuck. We thought the IRC conversation might also be useful for other
people who are interested, so here goes. Enjoy!


     brad-: Started <- Sure. Did what I was talking about in my e-mail make
any sense?
        jm: Well, I'm having some problems with your terminology...
     brad-: What terminology?
        jm: But I understand you don't want to link the DL with Overflow
     brad-: Linking Dl and Overflow <- Well, I think most of the linking
will be occuring at the level of the bl, right?
     brad-: If there are parts of Overflow you think would be useful in the
dl, I'm definately more than happy to use them!
        jm: I don't understand what you're refering to by "A piper user
should be able to wrap any program"
     brad-: Okay, what I'm trying to talk about is having a user be able to
convert a program (or library) to be able to run under piper.
        jm: Well, all Overfow nodes need to be classes... using a program
has to be done by wrapping it in a node.
     brad-: I've been trying to go through Overflow to see how the
libraries you are using get loaded to work with the program, but I get kind
of lost. I can't see the link...
     brad-: When you say wrapping in a node, how does this work. For
instance, let's just take the HMM library. Where in the code does it get
integrated into Overflow?
        jm: Well when you do a dlopen() on the librairy, the code for the
node gets loaded and there's an initializer that sets some data about how
to create each node in the lib, as well as what inputs/outputs/parameters
are required
     brad-: Okay, this was the general picture I had. What specifically is
the initializer that sets the data and figures out the
        jm: BTW, all those libraries contain classes that all derive from
the base "Node" class
        jm: If you look at .cc files that define nodes, all have the
NODE_INFO(...) macro at the start.
        jm: The macro arguments are the node name, category, inputs,
outputs, parameters
     brad-: Oh, macros, okay. Where is the macro defined?
        jm: It's in data-flow/include/Node.h
     brad-: Okay. Gotcha on that. So If I wanted to get a new library
working under Overflow, what steps would I need to take?
        jm: Basically, it uses a C++ initializer to call addFactory(...)
with all the data.
     brad-: Yeah, I'm just looking at it. Man, I was having the hardest
time trying to figure out how NodeFactories got added :-)
        jm: You create a class that derives from Node and understands the
getOutput(...) method. Then you include the NODE_INFO macro and link with it
     brad-: Okay, makes sense, I gotcha. What are your thoughts on ways to
move this system into piper?
        jm: The idea is that in C++, you can define a global variable as
int dummy = some_funct(); and some_funct() will get called before the main
starts (or when the library is linked)
        jm: Two ways: 1) get the data by linking with the library (which
you don't like), or 2) get the data somewhere else.
        jm: One way would be to create an executable that loads all those
libs and outputs the content of the NodeDictionary in an XML file.
     brad-: Get the data by linking with the library <- It is not that I
don't like this :-). I just was trying to think of a way to generalize it
so people wouldn't have to know c++ to get a program working under piper.
        jm: What do you mean by "get programs working under piper"?
     brad-: loading libs to XML <- This sounds very nice :-). That sounds
like a good way to go. The Overflow could link in the libraries and get
their info, and I could link in the libraries in the dl and output the info
as an XML file. This way the info would then be available to send to the ui.
        jm: There is already a node (C++ class) that takes a program name
as input and outputs the stdout of the program
     brad-: "get programs working under piper" <- Oh sorry, my bad
terminology again :-). What I'm just trying to say: integrate a program
into piper so that it can be used as a piper node.
     brad-: program name node <- Where is this defined? Also, how can we
get at the available flags, etc to send to the program?
        jm: There will be a set of nodes that can "talk" to other
programs... But no need to code a new node for each program you want to use
in piper.
        jm: It's called ExecStream
     brad-: ExecStrem <- Gotcha. I'll talk a look at this later and
understand what is going on :-)
     brad-: Okay, maybe I should stop chatting now and take some time to
look things over again with all of the new info :-)
        jm: I'm going to have dinner anyway... ttyl
     brad-: I'll have better questions to ask you once I manage to
understand things more.
        jm: BTW, we should save the log... it might help other people...
     brad-: Have a good dinner. Thanks for talking me through this, I
really appreciate it! ttyl

More information about the Pipet-Devel mailing list