[Pipet Devel] Paos article

[J.W. Bizzaro]

Locians,

Attached is the Linux Magazin article on Paos, translated to English.  I did
this with the help of Babelfish, but it still required/s some cleanup.

I made this effort because there is much confusion about just what Paos will do
for Loci.  I hope it helps.

Enjoy!


Jeff
-- 
J.W. Bizzaro                  Phone: 617-552-3905
Boston College                mailto:bizzaro at bc.edu
Department of Chemistry       http://www.uml.edu/Dept/Chem/Bizzaro/
--
-------------- next part --------------
Persistent Objects and Workflow-Management with Paos
by Carlos Maltzahn

----------------------------------------------------------------------------

Paos is a system for the remote-network and consistent administration of Python objects. Carlos leads us in today's issue of Python Tools in providing background to this interesting tool. As the larger application developed besides the Workflow Managment system, Chautauqua thereby is introduced and occupies the possibilities by Python in real applications. 

----------------------------------------------------------------------------

The Python module shelve supports a storing of Python objects into a file. After opening shelve files can be entered any objects with a name: 

>>> import shelve
>>> db = shelve.open('database')
>>> db['first object'] = [1, 2, 3]
>>> db['second object'] = ('hallo', [])
>>> db['first object']
[1, 2, 3]
>>> db.close()

If now this interpreter session is terminated, this way the entered objects are preserved. With the next call of Python these objects can be loaded by means of shelve.open('database ') again, i.e. these objects are persistent. The implementation of shelve can be configured at the Compile time of the Python of interpreter. Different data base c-libraries are available, e.g. dbm, gdbm and bsddb. These libraries implement data structures, which enable fast access to the stored data. In two important points shelve however offers no support: It does not implement a parallel access supervision, i.e. if several processes access persistent objects of the same shelve file, the file can become inconsistent. Additionally no inquiry language makes shelve available. 

Paos (Python Active Object server) structures on shelve and implements a Client/Server architecture with parallel access supervision and a simple inquiry language. 

    http://www.cs.colorado.edu/~carlosm/paos1_arch.gif

In addition Paos makes a notification service available, by which Python can use the server to be able to be informed about certain conditions. These conditions are defined by applications in the form of inquiries and registered with notifications the service in the server. Every time an application stores something with the server, the server applies the registered inquiries again to the stored objects. If a response on a request is not empty, the server transmits the response to the application, which registered the inquiry. 

An example and somewhat more (or too much?) Detail 

The following example illustrated how an application with the server constructs a connection, sets an inquiry and accesses to attributes of a loaded, persistent object. We assume a Paos server on the machine runs cheesy.cs.colorado.edu and waits for inquiries on the port 5000. The example produces some objects of the class person, stores it and executes an inquiry. 

import Client
import ExampleSchema

# builds connection with the Paos server on
conn = Client.Connection('cheesy.cs.colorado.edu', 5000, 'example')

# produces objects
john = ExampleSchema.Person()
john.name = 'John'
sue = ExampleSchema.Person()
sue.name = 'Sue'
john.loves = sue
bill = ExampleSchema.Person()
bill.name = 'Bill'
sue.loves = bill
bill.loves = sue

# registers objects with the server
conn.register_objs([john, sue, bill])

# stores objects off
conn.commit([john, sue, bill])

# gets all instances of ' person ', those who Sue falls in love with
answer = conn.get('r', 'Person', [('loves', '==', sue)])

# for each object in the response prints out the names of the loved.
for obj in answer:
  if obj.hasattr('sibling'):
    print obj.name, obj.sibling.name

First we import the module Client, in order to be able to structure a connection with the Paos server. Afterwards we import the module ExampleSchema, which the class person defined (see further below). Finally we structure a connection, by instantiating the class of the connections. We indicate the host names and the port, on which the Paos server runs. In the third argument any name for application can be entered. This name emerges then in the appropriate log entries Servers. 

We produce then three person - for instances and assign them attribute values. Before we can store these objects, they must be registered only with the server. The registration assigns a unique data base number to the new objects. This benefits us in the following inquiry, which follows storing: "give me all objects from the class person, those the object sue holds dear." If sue had not been registered, the server could not compare this object with the stored objects. 

The first argument 'r' in the inquiry means that the objects are only read in the response by application. If we liked to modify objects, then we must indicate either in the inquiry instead of 'r' the argument 'rw', or acquire the write rights for the objects which can be manipulated subsequently as the method conn.lock. We can acquire the write rights only if no different one possesses the write rights. If we possess the write rights, no different one can modify the corresponding objects. With everyone conn.commit and with program abort we lose all acquired write rights.

The inquiry supplies a list with two new objects, which are equivalent to John and bill to us. In the loop following on it we print the name of the respective loving out (both times ' Sue '). This harmless looking loop has it however in itself: The Client module guarantees that sue, john.loves and bill.loves to the same object point. This is enabled by the registration of sue and a resolution process, which is built into the attribute access of john.loves and bill.loves. This resolution process is permitted to be implemented over the inserted Python method __getattr__, those the redefinition of attribute accesses. Additionally these extended attribute access provides for dynamic loading of objects, which do not exist yet in Client application (the implementation of the attribute access is defined in Schema.py in the class DBobject. The method register_objs the class Connection in Client.py installs this attribute access for each Object in the argument list).

It is important to understand that this resolution process can provide only for the referential consistency of registered objects among themselves. In the above example the variables John and bill point to objects, which are not contained in the response. It is situated here in the responsibility of the programmer to detect when variables point to outdated objects. With a simple trick, variables can become "refurbished": John = conn.cache[john.db_id ]. In addition it is to be known necessarily that the Connection object administers a Cache for loaded objects and this Cache accesses over the data base numbers of the registered objects. Each registered object possesses the attribute "db_id with unique data base number. The Cache contains the version of all loaded objects, last-loaded in each case. 

Paos's most interesting characteristic is however the notification service. The following example shows how this service is used: 

import Client
import ExampleSchema
import Utilities
import os
import pickle

# defines a Pipe for notifications
(read_pipe_fd, write_pipe_fd) = os.pipe()

# builds connection with the server on
conn = Client.Connection('cheesy.cs.colorado.edu', 5000,
                         'example', (read_pipe_fd, write_pipe_fd))

# registers inquiry with notifications the service
request_id = conn.register('Person', [('name', '==', 'Sue')])

while 1:

  # control room on a notification and reads it
  data = Utilities.READ(read_pipe_fd, 10000)

  # packet notification out
  (req_id, obj_list, other_client) = pickle.loads(data)

  # packet identification from other client
  (other_host, other_pid, other_uid, other_name) = other_client

  # makes something with 

Compared with the first example three additional modules must be loaded: Utilities is a module with auxiliary procedures, which are used in all Paos modules. os and pickle are inserted modules of Python, the operating system functions and make available functions for the transformation of objects into a string (serialization).

First we define a Pipe, which will serve us later than recipients for notifications. We structure then a connection to the Paos server. The call of the Connection function has the Pipe as the fourth argument, so that the Pipe can be associated with the connecting object. Then we register an inquiry, which ensures that the server sends us all new person objects with the name 'Sue', as soon as these objects are again entered into the data base. Conn.register(...) Call returns the delivery a registration number.

We receive the notification over the Pipe. We use for it an auxiliary procedure, which guarantees that the full length of the notification of the Pipe is read. The notification is sent as string over the network and must be converted into a Python object again with the receiver side. This occurs with the assistance of the call pickle.loads(data). A notification consists of a Tripel, which those 

    * Registration number of the inquiry,
    * the response of the inquiry in form of a nonblank object list and
    * the identification of application, stored those the objects and with it the notification released 

includes. This identification again consists of 

    * the computer name
    * the number of the user process,
    * the number of the user user
    * and the third argument of the Client. Connection function call in application. 

Chautauqua: A larger application with Paos

Paos is a " spin off " product of a Workflow research project. One of the results of this project is the experimental Workflow system Chautauqua. Paos makes notifications available with the service the communication infrastructure for the different Chautauqua system components. Chautauqua users interact with web browser and with the system over a graph wordprocessor. The web browser displays dynamically generated "to-cDo" lists for each coworker, and is used for filling out forms. The graph wordprocessor displays the structure of an office process and the status of differently jobs. E.g. if an office coworker stores contents of a form in Paos, Paos notifies the Chautauqua Workflow manager, who delegates the form to the next office coworker. This process can track each user on their graph wordprocessor, since each wordprocessor receives and converts the appropriate notifications immediately into graphic representations.

The specialty of Chautauqua is that it enables to the users to change the structure of the office processes during running jobs. In the following snapshot we see the structure of an office process:

    http://www.cs.colorado.edu/~carlosm/paos2_ICN.gif

Coworkers are explained by asterisks, office roles by squares, and activities by sets and triangles. Small points on the right above activities represent "tokens", which represent the status of the work, and which move with the work progress by the graph. With the wordprocessor it is now possible to change any part of the graph. If activities are deleted, tokens can lose their location. Chautauqua offers mechanisms to gather and assign these lost tokens new locations in the changed graph.

Information 

Paos and Chautauqua are completely programmed in Python and freely available at:

    ftp://ftp.cs.colorado.edu/users/carlosm/paos-1.4.tar.gz 

    ftp://ftp.cs.colorado.edu/users/carlosm/chautauqua-1.4.tar.gz
    (Chautauqua contains Paos)

More detailed documentation for Paos and Chautauqua is at present in preparation and is announced in the new group comp.lang.python.

----------------------------------------------------------------------------

Carlos Maltzahn is at present a computer science student in the Ph.D. program of the University of Colorado in Boulder. His interests in research concentrate at the moment on Internet Caches and distribution indicating. In his spare time he roams either somewhere in the fantastically beautiful Rocky Mountains or spends his time building mobile robots from Fischer technique. To reach him use carlosm at cs.colorado.edu

----------------------------------------------------------------------------

Copyright ? Linux Magazin