Metannogen: Data storage

Each dataset is stored in one short text file on the server Demo_datasets/. The file names are the dataset ID. The central data store is managed by a PERL script metannogen.pl. The Perl script is called in two different ways:
  1. If called without parameters, it delivers all datasets as one single text.
  2. If called with the variable "datasets=...text...", the given datasets are written to the server.
The address of the Perl script of the demo database is metannogenDemo.pl This simple approach is extremely fast and robust. Transfer of a database of 3000 biochemical reactions from the data store to the client takes about a second.

See the comprehensive benchmark tests for SQLite, Postgres and MySQL.

In a multiuser environment it must be prevented that changes by one user are overridden by another user. Therefore the server response ends with a line "#CURRENTTIME=number" where number is an integer value representing the universal time on the server in seconds or milliseconds. Further it is important, that the program Metannogen is accessing the server directly and not through a because only direct contacts return the correct "last-modified" time for in the http-header.

Metannogen is not using a relational DB

In earlier days days the RAM was not yet sufficient to hold entire networks in the main memory.

Therefore it had been considered to store the data in a relational database like Postgres. However, current SQL database back-ends are still too slow for the functionality and high responsiveness of Metannogen. Another disadvantage of a relational database approach is, that due to the complexity it is much more liable to data loss as a consequence of programming errors.

Because RAM is cheap these days, we use the object oriented programming paradigm and decided against a relational data model and database back-end.