I had already seen the wonderful job done by Turfs and Yale university and have found it a valid description for the requirements of a trustworthy recordkeeping system. It intersects at some point with our work and we will be using several of their conclusions on the final system.

I’ll try to introduce the project scope as briefly as I can because it will be a very complex and long project.

The first thing is that it is a public institution funding project. With funding enough for 5 to 7 senior programmers over a period of 11 months. The final goal of the project is to obtain a trustworthy record keeping system to be used in production by all the public institutions in a Spanish region named Catalonia.

As such the main users will be archiving people at public institutions in general such as city halls, councils, historical records institutions and so on.

Briefly the main aspects of the project are:

-Fedora as the core digital object repository
-Global Object Identifier through URI
-Digital Object Versioning Support
-Support for Digital Objects containing several datastreams
-Full text search support through abstraction API first using Google appliance
-Package Ingestion support for METS, XFDU, MPEG-21 DIDL, FOXML
-Package Dissemination support for METS, XFDU, MPEG-21 DIDL, FOXML
-Full support for SPARQL queries
-Package Archiving support for FOXML, XFDU
-Support for Representation information using PREMIS [Object/Characteristics, Object/Environment], MIX , AES-X098B
-Support for preservation using PREMIS [Event, Agent], Context preservation through RELS-EXT, RELS-INT
-Object reference through DCMI
-Security support using PREMIS [Object/Fixity] and XML Timestamps
-Rights management through XACML
-Automatic non-RDF ontology support through automatic crosswalks [ej. MARCS, MODS,MARCXML, EAD,…]
-Identity federation through liberty alliance tickets
-Ingestion interfaces using WEBDAV/DELTAV, SOAP, SFTP
-Automatic validation of Digital Signatures, SIPS, metadata format and Antivirus support
-Package dissemination support through WEBDAV, SOAP, HTTP REST
-Metadata federation support for several archives (OAI-PMH)
-Storage decoupling so that the AIP might be stored in HDD, WORM, etc.
-Digital Signature Preservation through automatic evidence recollection at ingestion (CRL, timestamp, etc), including re-signing policies
-Integrity support through incremental signature policies
-Complete representation knowledge base including algorithms, software, operating systems and available emulation software
-Support for automatic format migration among several formats, word, pdf, odf, ..
-Support for record expiring policies
-Support for offline data
-Auditing support
-Full HTML administration and preservation interfaces

And many more that either are not important enough or I might have missed. Sorry for not tagging accordingly all the references but there are simply too many.

As you can see many of these features are already supported by fedora. But there are many areas that are either out of fedora scope or simply not implemented.

Our intention is to release all the generated code under a compatible license such as the one fedora has. And to revert back to fedora community all the code they wish to have either in the core or as tools.

As such any collaboration is always welcomed.

I was not aware of the XFDU format until you mentioned it — that looks pretty interesting! Have you written up any documentation or whitepapers about the intersection of FEDORA with XFDU? Also, I presume you are aware of the work of Tufts University and Yale University on the preservation of university records using FEDORA. I don’t know how much that intersects with your work.

[quote]
In this way we try to totally eliminate the techonological preservation risk involved in using only fedora tools for the archiving.
[/quote]

Agreed!

[quote]Another point we are trying to cope with is the kowari triplestore dependency. Kowari is a good enough triplestore but the project is no longer under development. To do so we are also going to implement compatibility for both Sesame and Jena.[/quote]

There has already been some work by Case Western Reserve University to use Oracle for the triplestore. At the moment, I can’t remember it Case’s work has been integrated into the FEDORA core developer’s trunk, but at the very least the could be a level of abstraction that you could leverage to use other triplestores. I’ve worked with the folks up at Case Western — if you’d like, I could make an introduction.

Nevertheless, we are currently working in a records preservation project in Spain using fedora and one of the features that we will try to implement is that the final AIP written to the HDD must be in XFDU format (user selectable).

In this way we try to totally eliminate the techonological preservation risk involved in using only fedora tools for the archiving.

Althought it is true that you can rebuild the kowari triplestore index using fedora tools we also find it a high technological risk that seriously affects preservation policies.

Another point we are trying to cope with is the kowari triplestore dependency. Kowari is a good enough triplestore but the project is no longer under development. To do so we are also going to implement compatibility for both Sesame and Jena.