[Pipet Devel] batch processing

Jarl van Katwijk jarl at casema.net
Wed May 24 08:15:03 EDT 2000

> > I think the sequence or hiarchy of the nets are something of high importance to a
> > batch schedular: job that should not start untill other(s) are finished. Or job
> Wouldn't all those dependencies be solved by "pulling" on the data-flow? or am I
> missing something?

consider this structure:

           /       /
>--O3     /

Where the O's are the nodes. Once could execute on dataflow basis, so O1,2 & 3 will run,
and before O5 O4 has to run, so O5 will stall untill O4 is ready.
As good batching will run O4 before O5 is stalled. But I must admit the dataflow is 90%
of the batch logic.

> > termintion once certain situations occure.  I've worked a lot with JCL, job control
> > language, with is the scripting language of IBM mainfraimes. A very typical aspect
> > of jcl is that it's laking iterations and jumps.
> > So no for\next until\while stuff and no goto's.  Does Loci depends upon those or
> Loops are supported by Overflow through the "Iterator" (which derives from the
> subnet). I'm also using some kind of "checkpointing" in Overflow for very long
> jobs (a couple days of processing), such as neural network training. Now, about
> goto's, I'm strongly against them in a data-flow type of processing.
> > suggest we'll ALWAYS use timeouts on jobs and use triggering or stalling as the
> > only scheduling primals.
> > I'm only talking about SUBNET batching, inside those at PL level nothing will
> > change.
> I think most (though not all) of this is likely to be simpler if implemented in
> the PL.

You did take cpu load, ram\HD space usage etc into account already?

> P.S. Brad and I have ran into some problems with the antialiased gnome canvas as
> used in Overflow (But everything now *compiles* on FreeBSD). Has anyone compiled
> Overflow on something else than Mandrake 7.0 and FreeBSD?

I had it compiling on my slackware 7.0 with gcc 2.95.x manually installed. But I
reinstalled slack :(

More information about the Pipet-Devel mailing list