[Pipet Devel] batch processing

[Jean-Marc Valin]

> > While designing Loci, my first thought was that these "jobs" could be handled
> > as they are on a supercomputer: with batch processing.  This allows some
> > information to be returned about the progression of the job, cpu time used,
> > etc.  It also allows jobs to be scheduled to run at a certain time, with a set
> > priority, etc.
> 
> I think the sequence or hiarchy of the nets are something of high importance to a
> batch schedular: job that should not start untill other(s) are finished. Or job

Wouldn't all those dependencies be solved by "pulling" on the data-flow? or am I
missing something?

> termintion once certain situations occure.  I've worked a lot with JCL, job control
> language, with is the scripting language of IBM mainfraimes. A very typical aspect
> of jcl is that it's laking iterations and jumps.
> So no for\next until\while stuff and no goto's.  Does Loci depends upon those or

Loops are supported by Overflow through the "Iterator" (which derives from the
subnet). I'm also using some kind of "checkpointing" in Overflow for very long
jobs (a couple days of processing), such as neural network training. Now, about
goto's, I'm strongly against them in a data-flow type of processing. 

> suggest we'll ALWAYS use timeouts on jobs and use triggering or stalling as the
> only scheduling primals.
> I'm only talking about SUBNET batching, inside those at PL level nothing will
> change.

I think most (though not all) of this is likely to be simpler if implemented in
the PL.

	J-M

P.S. Brad and I have ran into some problems with the antialiased gnome canvas as
used in Overflow (But everything now *compiles* on FreeBSD). Has anyone compiled
Overflow on something else than Mandrake 7.0 and FreeBSD?

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
valj01 at gel.usherb.ca