There was a time when I was moving in both the worlds of the Sakai Collaborative Learning Environment and the Fedora Commons digital content repository. It seemed like a good idea to bring these two worlds together — Fedora as a content repository for Sakai learning objects. Back in 2006, I logged a ticket in Sakai’s tracker to see if anyone was interested. This morning I got notification that they are thinking of closing the ticket.

I’ve since moved away from both communities into other areas of interest, but felt one final duty of stewardship over this idea before it drifts out to sea. Perhaps integration between Sakai and Fedora Commons is already happening and this ticket is anachronistic. Perhaps this wasn’t such a good idea to begin with and it should die on the vine. Perhaps someone else will think this is a good idea and become the champion for it. At this point, though, my professional interests are elsewhere and I can’t carry this one forward.

Here is the text of the message:

Issue (View Online) Key: SAK-7833 Issue Type: Feature Request Status: Open Priority: Major Assignee: Unassigned Reporter: Peter Murray Operations  View all  View comments  View history

Fedora as Sakai’s Content Repository  Updated: 23-Feb-2010 06:32   Created: 27-Apr-2006 08:20   The following issue has been updated. Updater: David Horwitz Date: 23-Feb-2010 06:32 Comment: MAINT TEAM REVIEW: This feature request is currently unassigned and will be reviewed. In line with stated Jira practice [http://confluence.sakaiproject.org/display/MGT/Sakai+Jira+Guidelines] Feature requests that are not going to be implemented will be closed with a status of "wont fix". If you intend implementing this issue please ensure that its up to date and assigned correctly Project: Sakai Components: Content service (Pre-K1/2.6), Resources Affects Versions: 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1  Description    Provide an alternative to the existing ContentHostingService interface that stores content in database tables, directories, and files with one that stores and retrieves content in a Fedora Digital Object Repository.

This could easily go in the “Disruption in Libraries” category of DLTJ, but it is a disruption of a different sort. Are you making contingency plans to continue library services in the event a Bird Flu pandemic (or an event of similar sort) happens? A recent posting on the Sakai developer’s mailing list prompted the thought. Sakai is an open source collaboration and learning environment that is typically used for electronic courses. John Leasia of the University of Michigan wrote:

We here at UMich are doing some contingency planning in the event we are hit with a Bird Flu epidemic. We have heard some institutions are planning on just shutting down. Some like UM intend to stay open and move from our predominately in class instruction to all online. We are currently planning what needs to be done now (tutorials, tool enhancements, new tools, adoption of existing tools not yet in production here, infrastructure scaling, etc.) [to enhance the local Sakai installation] in order to be ready should an epidemic strike.

Are there others out there in the same situation? How about some of us holding a joint Discussion session at Amsterdam [the location of the next Sakai users conference]? We could each provide a brief overview of our planning thus far and then open it up to see what everyone is doing, raise issues/questions we haven’t thought of, perhaps start coordination of development/training/doc that might be necessary?

Are there other institutions out there thinking about the same thing? Has the library been asked to participate in the contingency planning? Are you undergoing your own planning?

This is a summary of a presentation by James L. Hilton, Vice President and CIO of University of Virginia, at the opening keynote session of Open Repositories 2007. I tried to capture the esessence of his presentation, and omissions, contradictions, and inaccuracies in this summary are likely mine and not that of the presenter.

Setting the stage

This is a moment in which institutions may be willing to invest in open source development in a systematic way (as opposed to what could currently be characterized as an ad hoc fashion) driven by these factors:

• Fear. Prior to Oracle’s hostile take-over of PeopleSoft, the conventional wisdom of universities was that they needed to buy their core enterprise applications rather than build them. In doing so, they sought the comfort of buying the security of a leading platform. Oracle’s actions diminished that comfort level. Blackboard acquisition of WebCT and lawsuit against a competitor does not help either.
• Disillusionment and ERP fatigue. What was largely thought to be an outsourced project was found to be an endless upgrade cycle. Organizations need to build entire support units to handle the upgrades for large ERP systems rather than supporting the needs of the users.
• Incredulity — we’re supposed to do what? The application of technology typically has a disruptive impact (cannot predict the end), the stakes are incredibly high (higher education and/or research could be lost in a decade), it tends to be expensive, and the most common survival strategy is to seed many expensive experiments in the hopes that one will be in the right place at the time the transition needs to happen. The massive investment anticipated for technology to support academic computing (libraries, high-performance clusters, etc) will pale in comparison to the investment in administrative computing.
• Rising tide of collaboration. This is a realization that the only way to succeed is through collaboration. To paraphrase Hilton, “In the new order it will be picking the right collaborative partners where the new competitive advantage will come from.”

Distinctions

Hilton offered these definitions and contrasts as a way to frame the rest of his discussion. First was Open or “free” software. Free as in beer, or free as in “adopt a puppy.” The software comes with the ability to do with as you want with the code, not just the ability to use the code. They he defined the term License as a contract — what ever you agree to you are bound to; you cannot use copyright law to protect you. The rules and conditions that are applied to the software do matter.

Lastly, he talked about Copyleft or “viral” licensing. There are different interpretations of “open” in open source. “Copyleft” has come to mean that code should be freely available to be used and modified, and it should never by locked up. GPL is an example. This is often called “viral” because if you include software with this license in any other work that is released, the additional software must be released under the same license. This is seen by some as valuable because it prevents open source from being encircled by proprietary code. Copyleft is contrasted with an “open/open” license — you can do whatever you want to do with a code under this license. An “open/open” license places no restrictions on what users do with code in derivative software packages.

Case Study — Michigan’s Sakai Sojourn

Hilton briefly described why UMich went down the Sakai path in 2001-2002:

• Legacy system with no positive trajectory forward. It could never be released into open source; all of the development would have to be carried on UMich’s shoulders forever.
• Saw market consolidation in CMS. This was mostly evident in the commercial sector with Blackboard and WebCT being the dominant choices. They had concerns about the cost of licenses in this environment down the road.
• Saw the potential of tapping the institution’s core competencies and starting a virtuous cycle of development, teaching and research. Or, put another way, they didn’t want core competencies in teaching and research held hostage to a commercial development cycle.
• Strategic desire to blur the distinction between the laboratory/classroom and between knowledge creation/digestion. They realized that the functions of a research support tool and a course support tool were pretty much the same under different skins, and they sought to blur that distinction even more.
• NRC report and the need for collaboration. UMich was willing to fund the project two years internally but knew after that need to find collaborative partners by the fifth year in order to be declared a success.
• A moment of time opportunity that synchronized the development process of several partners with funding provided by the Mellon Foundation.

There were also specific goals for the Sakai project. The new system had to replicate the functionality of existing course and research collaboration environments. They also wanted experience in finding partners willing to collaborate. Hilton said, “Sakai was/is at least as interest from a collaboration perspective as it is from the technology perspective.” Bringing together disparate organizations with different beliefs on how things should be done is a challenge. Additionally, they wanted to get better as an institution at discerning open source winners; it shouldn’t be like a lottery. Lastly, they wanted to implement software parts that were not built at UMich. Each partner institutions committed to implementing the same thing even if wasn’t built at that institution. This is tough to do, but they knew they needed to do it for their own good in the long run.

What happened? Not only did the original partners show up, but the community came, too. Even more interesting was that the community was formed with dues-paying members — even in a world where the software is free. It became a vibrant community, too, with a conference every six months. Sakai was released under an open-open license model, and corporate partners showed up as well (selling support services, or hosting services, or hardware for the software). The software did grow up and left its home; a separate foundation now holds the intellectual property of the code (originally partners assigned copyright to UMich). They also positioned Sakai to be a creditable threat to the commercial entities in order to force them to the standards table.

Takeaway lessons that generalize to open source development

First, the benefits of open source development.

• destiny control (but only when you really need to drive). having the control is not always a good thing. Is it worth the effort? Is the project core to the institution’s mission? (Does it directly support scholarship and teaching?)
• builds community and camaraderie (in the case of Sakai, both locally at UMich and internationally)
• unbundles software ownership and its support. inspires more competition in the implementation and support space.
• community source provides institutions an opportunity to leverage links between open source, open access and culture of the academy/wider world (a.k.a. put up or shut up)

Then, the challenges of open source development.

• Guaranteeing clean code (IP) is hard (read as “impossible”). A certain amount of faith about the code they get and there needs to be consideration for mitigating risks.
• Figuring out who is authorized to license institutionally-owned code is challenging and then you have to convince them to give it away. No one in the institution typically has been appointed or given the authority to release code. One of the things that the sakai licensing discussions highlighted was institutional differences in requirements and aesthetics.
• Patent quagmire always looming. How do you know your software is not infringing? How do you make sure you don’t inadvertently give away all institution patents? Be careful when looking at licenses from an institutional perspective versus an individual perspective.
• There is also the inevitable lawsuit risk. Or, as your counsel might say to you, “Let me get this straight, we can get sued but there’s no one we can sue.”

Then, some discoveries that they made along the way.

• An open source project not a silver bullet. The commitment to build rather than buy must align with institutional priorities and competencies; it is not right for every project/application.
• Licensing does matter; it is a contract: whatever you stick in its rules is what sticks. There are probably have too many open source license options and some sort of standardization is needed. Also keep in mind that if you release something under an open/open license, you can’t include any copyleft components.
• Communities don’t just happen, they require: specific shared purpose (when visions vary, or when they change, collaborations struggle); and governance (e.g., separate board with dedicated developers sitting between institutions). Cooperation (“I won’t hurt you if you don’t hurt me”) is not collaboration.
• Open (community) source requires real project discipline. “It is as spontaneous as a shuttle launch.” Along the way one needs to learn to balance pragmatics and ideals. One also needs to learn to trust your partners. “It really requires learning to let go.” Letting go, and having the community make the decisions, may be the quickest path to efficiency.

Reflection on open/community source for repositories

Repositories are at the center of everything at the institution. It connects with the library, with the presses/scholarly publishing operation, with classroom teaching, with the laboratory, and with the world. It is a core piece of of infrastructure for the university of the 21st century. As institutions, we need to make sustaining investments in our repositories.

Hilton sees three different approaches to “community” in the existing projects:

• dspace: community of user/developers. The come together to talk about what they want to do, write code, and support each other. Clearly there are enthusiastic users as developers.
• eprints: appears as like a vendor talking with customers wanting the community help shape the direction.
• fedora: in transition from a combination of the previous two models moving towards a Sakia-like model. it will require institutions to make commitments to it.

In the end, Hilton asked some thought-provoking questions. Is now the time for institutional investment in open/community source? Will a coherent community (or communities) emerge in ways that are sustainable? — is there a shared vision?

The text was modified to update a link from http://www.virginia.edu/vpcio/bio.html to http://www.virginia.edu/vpcio/biography.html on January 19th, 2011.

Earlier this year, I was on a quest to hook a FEDORA content repository into the Sakai collaboration and learning environment. What looked at first to be a fairly easy integration turned out to be rather complicated and I set the project aside for another time. Today brings word from Ian Boston of a JSR-170 implementation in Sakai:

During the Summer of 2006, I did a JSR-170 Implementation of ContentHostingService as a prototype against the then Trunk 2.2 ContentHostingService. The implementation took the ContentHostingService API and re-implemented it using JSR-170 under the covers. It was done in in such a way as to allow JSR-170 clients (eg WebDAV implementations) to use the JSR-170 API directly and still obey the Sakai AuthZ implementation.

JSR-170 – Project: Resources – Sakai Confluence

JSR 170, as you might recall, is the “specification for a Java platform API for accessing content repositories in a uniform manner.” ¹ What makes this implementation most interesting, I think, is Ian’s last sentence — using a WebDAV implementation to speak directly to the JSR-170 content repository while taking into account the AuthZ settings in Sakai.

Now we need a JSR-170 implementation on top of FEDORA to complete the pairing. We’d want Sakai’s AuthZ settings to be reflected in FEDORA’s XACML rules, of course, the the mind boggles a bit about how to get this done, but hopefully we can get back to it again soon and see if we can make it work.

The text was modified to remove a link to http://issues.sakaiproject.org/confluence/x/ikE on January 13th, 2011.

The text was modified to update a link from http://www.tfd.co.uk/blogs/sakaiblog/ to http://blog.tfd.co.uk/ on January 19th, 2011.

Footnotes

1. “JSR-170,” Wikipedia, The Free Encyclopedia, http://en.wikipedia.org/wiki/Content_repository_API_for_Java?oldid=84044565 (accessed November 8, 2006)

Edited by Peter Murray, OhioLINK. This document represents a summary of comments on the Sakai Developers mailing list on April 26-28, 2006 to a question posted by this document’s editor regarding possible integration points between Fedora and Sakai. The resulting threads were:

• Seeking status of ‘Rep API’ and ‘JSR-170′ shim from "Sakai Repository API Design"
• Repository integration with Sakai
• Fedora in Content Hosting – An Alternative

The editor is indebted to the public and private responses of: Ben Brophy, Ian Dolphin, Glenn Golden, Adam Hochman, Jeffrey Kahn, Beth Kirschner, Peter Knoop, Anoop Kumar, Jim Martino, Mark Norton, and Charles Severance. Information from these individuals is supplemented by additional research by the editor, and the whole should not be viewed as a coherent or even advisable strategy. Direct quotes in this summary are not attributed to individuals, and the editor alone is responsible for omissions and quotes-out-of-context errors.

The editor’s question related to the integration of Sakai and Fedora. OhioLINK’s desire with the Digital Resource Commons is to build a unified content repository by meeting Sakai (and other tools such as OJS/DpubS for electronic journals, etc.) right where they store the data. In doing so, it is our expectation that allowing data to flow across applications will be much easier because there is one underlying repository.

Sakai Infrastructure

Description of ContentHostingService interface

ContentHostingService is one of the “backbones” of Sakai. To make Fedora the content repository (i.e. the attachment helper and resources tool, presentation tool, and any other tool using “content” in Sakai) then one must implement the ContentHostingService API. The API is found in https://source.sakaiproject.org/svn/content/ as org.sakaiproject.content.api.ContentHostingService (JavaDoc) and the baseline implementation can be found as BaseContentService.

The ContentHostingService API supports WebDav, the AccessServlet, the WebServlet, Resources Tool, Presentation Tool, interoperates closely with Sakai’s Authorization model, generates well-formed Sakai notifications, and real-time events, and is cluster aware and on and on. This is a key API used heavily by Sakai; an alternative implementation probably needs to implement it completely to be of any use.

Plugin Pattern

Most current open-source repository systems cannot handle the on-line performance demands of a large-scale Sakai installation. As an example, a Sakai installation supporting 500 users could easily store all of its resources in a separate repository. But a system supporting 100,000 users may not be able to “outsource” all of its storage.

What is needed is a sophisticated set of plug-ins that allow great flexibility for different Sakai installations. Instead of a complete re-write of the ContentHostingService interface, a plugin pattern that allows ContentHosting to precisely outsource things like blob storage and metadata storage with simpler APIs than ContentHosting and a very clean and simple API contract. Then one can write plugins for things like Fedora, JSR-170, DSpace, iTunes, etc.

Extend ContentCollection

Suppose one were to extend the existing ContentHostingService, specifically the ContentCollection class (JavaDoc) such that it serves as a gateway to a repository. It would need to represent the protocol to be used (Fedora, SRA, JSTOR, whatever) and likely include at least some support for digital rights management.

A simple idea with a lot of consequences, both easy and hard. On the plus side, repository-based collections could either be tied to a search pattern on the repository,or tied to a specific remote collection (including the root). Done right, it would integrate seamlessly into existing Sakai tools, such as the ResourceTool, etc. On the minus side, ContentResource might also be affected, since the tools would list content that is no longer directly accesible via the current implementation. WebDAV integration might also be challenging.

In the intial stages, this would have to be an optional feature that could be enabled or disabled since complete integration may not come quickly. Still, one could have an instance of the ResourceTool tied to a specific repository (not mixed) that could be billed as not supporting WebDAV or other regular content hosting features.

ContentHostingService refactoring

Ian Boston of Cambridge University reports development of a plugin into ContentHostingService that enables an implementing class, once it has registered itself with the ContentHostingService to be consulted if it ‘owns’ a node in the content hosting tree. Once it ‘owns’ the node, it takes responsibility for providing ContentResources for that node and all child nodes.

Cambridge has used this to implement an IMS Global Learning Consortium Content Packaging specification (IMS-CP) plugin, that ‘plays’ IMS-CP files but one could just as easily use it to ‘mount’ a repository inside the ContentHostingService at an arbitrary position.

If the resource is read-only, or can’t be accessed via DAV, then it is the responsibility of the plugin to act accordingly (ie, not return the content).
Cambridge is also looking to write a plugin that ‘interprets’ an XML file loaded into resources to invoke a UI. With Fedora, that XML file might be the connection settings to the repository.

Since the plugin patch is small (<50 lines), and the plugins can be implemented inside webapps (ie reloadable), Ian is hoping that it will be useful and can be included in the BaseContentService (JavaDoc) for 2.2. If it isn’t, Cambridge will maintain it as a patch as it as they are going to be running it in production in July. See SAK-4457.

JSR-170 interface

JSR 170 is the Content Repository API for Java. It specifies a standard API to access content repositories in Java independent of implementation. (See also its follow-on: JSR 283 Content Repository API version 2.0). Neither Fedora nor Sakai have JSR-170 interfaces, although if they did it could be a point of integration. Implementation of JSR-170/283 would be a new implementation of the Sakai ContentHostingService API.

AFS interface

UC Davis people have home directories in AFS space, and their course management system also utilizes this space. UC Davis has a problem statement for integrating the user home directory AFS space into Sakai.

Alternate Integration Points

OKI Repository OSID as a Sakai Tool/Service

Sakai offers Repository integration through the O.K.I. Repository OSID. The Repository OSID is used in a small (but perhaps growing) number of places in Sakai as a secondary interface to external repositories — it does not go through the main ContentHostingService interface. If this bridge were wide enough to include all the functionality Sakai needed, these calls could go through OKI. (Additional information is on confluence, implementation is tracked as SAK-2327, and an example application is Twin Peaks.)

One implements a set of interface methods that talk to your repository and then applications (one of which is Sakai) can search, retrieve, and upload content to your repository, subject to permissions. Since the interface is standard (maintained by the IMS Global Learning Consortium), the calling application can use the same approach for all repositories. The caller is also insulated from connection, protocol, and some data format specifics.

The Fedora OKI Bridge under development by Anoop Kumar of Tufts for VUE release 1.5 (expected June 1, 2006) will support following:

• It will work with Fedora 2.0
• It will provide read-only support.
• It provides ability to perform simple and advanced search.
• It will support three types of RecordStructure(OSID).
• Dissemination RecordStructure holds all disseminations of object as its parts.
• Image Record Structure will provide thumbnail, medium and high res images.
• VUE Default Record Structure will provide a default view for digital object and dublin core metadata.

Sakai may be able to use the bridge as is or we can add additional record structures to support Sakai’s needs.

Sakai’s Repository OSID is being considered as an integration point with the DSpace Institutional Repository application.

UMichigan Fedora Tool

Beth Kirschner at the University of Michigan is developing a Fedora repository interface for XSLT-driven CRUD operations against Fedora objects. The current implementation is a prototype, supporting browse, search and editing capabilities. Authorization is not yet integrated with Sakai. For more information, see: source.sakaiproject.org—README.txt.

The text was modified to update a link from http://bugs.sakaiproject.org/confluence/x/ikE to http://confluence.sakaiproject.org//x/bYD9.

The text was modified to remove a link to https://source.sakaiproject.org/svn/content/trunk/content-api/api/src/java/org/sakaiproject/content/api/ContentHostingService.java.

The text was modified to update a link from http://webby.ucdavis.edu:8080/confluence/pages/viewpage.action?pageId=4549 to https://confluence.ucdavis.edu/confluence/pages/viewpage.action?pageId=4549.

The text was modified to remove a link to https://twinpeaks.dev.java.net/ on June 9th, 2011.