view TODO.org @ 265:b75b45d76e45

add ioi-from-bar method to amuse-mtp

* AMuSE
** amuse-database-admin + CHARM db persistence
   - Generalize mtp-admin?
   - How general should amuse-database-tools be?
   - General import/export API?
*** TODO Use database user accounts
*** TODO Move prototype versioning code from CHARM into amuse-database-admin

Provide the logic to update and retrieve successive versions of
constituents. Includes providing a way of deleting (updating, marking
something as deleted) and destroying. The latter should require
adequate permissions and also test that no analyses depend on them.

***** TODO Define and implement versioning logic

Revision control. There needs to be some logic to work out the union,
intersection, relative complement between successive versions of
constituents. Surely best to do this server side? Use temporary
tables?

***** TODO Simplify current code

Column names could be in a variable: implementation-id parent-id
ext-properties int-properties owner version creation-timestamp
deletion-timestamp.

Some macros would also be helpful here, particularly if it would be
possible to abstract out the table creation and versioning stuff. That
way, all database tables created within AMuSE (in backends, or
specialized applications like SIA) could be created with a macro that
automatically generates the necessary revision control columns,
triggers and functions.

***** TODO Decide on the long-term aim of amuse-database-admin

More adventurously, the database admin stuff here could form the basis
of a general mechanism for creating tables and getting data into and
out of the database. Each implementation could be stored in tables
generated by the template CHARM reversion controlled table, with
backend-specific columns. SIA is not a backend as such, it can take
any other backend and generate data from that. Anyway, the database
interaction is very similar.

This could tie up with better overall integration of CHARM
constituents within each implementation. We could also ensure that all
compositions added to the database, and all their events have unique
ids! I still think we need an amuse_implementation table though, along
with consistent constructor methods specialised on packages -
otherwise, how else do we know to create a geerdes-constituent or an
mtp-constituent. Or maybe these should be properties of constituents?
But we'd still need the methods specialized on packages, or at least
we'd need to look up consistently named functions within individual
packages.

How possible or beneficial would this be? Might simplify things in the
long run and shouldn't have to compromise the flexibility of
individual implementations. This would be more about general work flow
and database interaction. I was going to try and do something along
these lines with the midi-db backend.

Or perhaps it's better to stick with specialized implementation
specific database functions with some code duplication?

*** TODO Check all backends use *amuse-database*

Sort out the passing around of the database variable. Basically, there
should only be one database connection accessible to every backend, as
presumably it would be very difficult to maintain consistency across
multiple databases (e.g. for the results of general analysis tools or
the CHARM constituents mechanism).

The database connection could be passed in as an optional argument
(defaulting to *amuse-database*). But that would require changing the
existing API (e.g. get-composition). The way I've done this in midi-db
is to make sure all the database functions explicitly use
*amuse-database* for any database call, rather than defaulting to
clsql:*default-database*. This allows other databases to be used for
testing purposes by shadowing *amuse-database* within the backend
package.

** Geerdes
*** TODO Sort out general weirdness in the data
    - Geerdes compositions timepoint = 0 instead of timepoint of first event.
    - g-id-file-id 1004 is in 6/4, which tactus-duration treats as compound time!
    - g-id-file-id 3822 is in 4/64!
    - g-id-file-id 8261 has a 63/32 time signature!

*** TODO Import new files
** Constituents
*** DONE Are the floating constituent abstractions sensible?
    - Or should the base constituent class not inherit from anchored period?

*** TODO CHARM
**** TODO Sort out old CHARM mess in my darcs Geerdes branch
**** Questions
     
     1. 'Parent' slot: the constituent the particles derived from
        (current implementation), or always the composition?

     2. What to do about constituent properties? They are currently
        'charm-property-list' objects, which are a subclass of
        sequence, but do not actually contain any additional slots!
        Should we define more structure for these objects? Should
        there be a differentiation between intrinsic and extrinsic
        lists? Should they be keyword-value lists instead of lists of
        strings? Or make better use of the class hierarchy?

     3. Related to property-lists, how should we determine identity
        between constituents? Can this be done purely from the
        headers - in particular, the property lists, or is it
        necessary to also test for identity between all of the
        particles?

     4. What to do about time? Should they be slots in the header?
        Time represented in database format (integer) or backend data
        type?

     5. The need for a consistant API. If we are going to ask for
        specific constituents from the database, and these might be
        from any backend, there needs to be consistantly named
        functions like 'make-composition-identifier' (as opposed to
        g-id-file-id in geerdes) in each package. Examples of generic
        functions I've added are 'identifier' and 'id', which will
        return the identifier (implementation specific object) or
        database-id-number (integer) respectively for any
        'amuse-database-object' given to them. (The class
        'amuse-database-objects' currently does not exists, but it
        probably should, providing the slots that are shared between
        all AMuSE objects stored in a database.)

        Is this a bad thing to do? The semi-general API I work with is
        most similar to Geerdes, just because I was most familiar with
        that backend when I started. Marcus seems to have his own set
        of conventions, but the two are probably not incompatible.

	If we don't have a consistant API, we have to make a CHARM
        implementation within each backend?

** Future development
*** Viewpoints, projections, properties and features
    - What is the difference between a viewpoint (MTP), a projection
      (SIA), and features (SIA, CONCEPTUAL-SPACES, and general
      constituent features and properties).
    - Retain original amuse context (like sia-datasets).
    - Define a general design for developing general tools?