[Pipet Devel] more batch processing

[J.W. Bizzaro]

Harry Mangalam wrote:
>     Re: the question of queueing, load-sharing/leveling, and being able to
> track a pid persistently, isn't this something that should be addressed at
> the system level?  ie - isn't this something that should be taken up in
> concert with the kernel folks or maybe the gnome folks so that the tulip
> plan doesn't go off in a direction ideal for us but turns out to be the one
> NOT chosen by others?
> 
As you mentioned below, if we expect the kernel (Linux?) or GNOME developers to
solve the problem, we (1) have to wait for these guys to do it, if they even
want to, and (2) we end up with something that is platform (in this case Linux)
dependent.

> 
> In the interim, if we need something to get this off the ground, a little
> hack could be writ to take the pid of the process and track it thru a cgi
> call to look at the the appropriate /proc entry.  This approach would, of
> course, require a different shim for evey OS (Irix is different than linux
> is different than Solaris, etc), but it would allow progress without
> committing to a possibly nonsensical path.

If there is some other way to do it, that won't require a different version of
Loci for each flavor of UNIX (some on the team think it is bad enough we are
ignoring Windows), that's fine with me.

I think "all we need" is a binding from Python to GNQS (Generic NQS).  We can
get the source code, but I don't think writing a binding will require that we
recompile it.  It shouldn't be all that bad.

Regarding compatibility, GNQS has been ported to nearly all flavors of
UNIX...just like Python and GTK and GNOME.  I don't know if we can call it
nonsensical.  From what I read, it was one of the first of all UNIX batch
systems, derived from the very first one used by NASA.  Is it out of date?  I
don't know.

> 
> Or we just ignore it for the present and write a dummy call
> HereBePersistantIds() that allows us to sidestep it.  If it's gonna be done,
> it should be done right, but waiting for it to be done right doesn't have to
> lock other efforts.

We absolutely need to have the Paos and XML framework set up before we can even
test something like a batch system.  So I agree with you.  Let's just pretend
for now that we will have some system set up for doing this.  Exactly what it is
I think Justin and Carlos need to think about very carefully...I'll put the
burden on someone else ;-)


Jeff
-- 
J.W. Bizzaro                  Phone: 617-552-3905
Boston College                mailto:bizzaro at bc.edu
Department of Chemistry       http://www.uml.edu/Dept/Chem/Bizzaro/
--