Richard Akerman’s recent post highlighting SOA resources at Educause reminded me about the aggregation point on Educause Connect for SOA resources. I’m assuming significant number of those interested in applying SOA to library systems are at an institution of higher education or in some related organization, so I’m adding the RSS feed for that aggregation to Planet LibrarySOA. This will undoubtedly result in a large spike of “new” postings to the planet aggregator, but should settle down after that.

If you are blogging about the application of SOA to libraries and want your postings to see a wider audience, let me know and I’ll add you to the aggregator.

Back in June, Eric Schnell posted a five part introduction to applying Service-Oriented Architecture (SOA) concepts to library applications. Along with his comparison of the predominant ILS architecture with Henry Ford’s application of assembly line manufacturing this is a great non-techie introduction to SOA form a library application perspective. I had reason to run across these again earlier this month and remembered that I had not posted a summary and pointers here.

Part 1 is an introduction with a brief overview of why SOA is interesting. Part 2 is a description of the silos of content that exist today which in part 3 Eric describes how the silos can be collapsed and the structure of services that surround them. Part 4 describes the challenges of doing this from the vendor lock-in that makes is financially not viable for SOA-based solutions to break into the space. In part 5 Eric offers some final comments on the introduction of SOA into library applications.

Well worth reading and bookmarking for future reference. Nice job, Eric!

cforms contact form by delicious:days

Resources tagged with ‘librarysoa‘ in del.icio.us will also appear in the aggregator.

Mechanics

This month I’ve come across one great article and one great report on Service Oriented Architectures. The first came from Sally Rogers at Ohio State University in the form of an article from CIO magazine last year:

Koch, Christopher. 2006. The Truth About SOA. CIO Magazine, June 15. http://www.cio.com/archive/061506/soa.html (accessed March 27, 2007).

This article does a great job at laying the groundwork for the broad “what” and “why” (as well as the “why not”) of SOA, and I agree with Sally that it makes a better introduction to the topic than most of the white paper that I presented at the meeting. The two best paragraphs out of the article come towards the very end:

In the ’90s, your integration strategy was simple: Buy as many preintegrated applications from a single vendor as possible. That worked for you, and it worked extremely well for the vendor; integrated application suites fetched a high price and required long-term maintenance and support contracts that promised a steady, predictable stream of revenue from customers.
[...]
But the rise of service-oriented architecture has produced a shift in integration strategy. SOA makes the radical assertion that the enterprise application infrastructure is irrelevant. Technology is constructed according to services specified by the business, not by processes contained within an enterprise application vendor’s software box. In this scenario, packaged software is a piece of the service, just another component in a larger business process — such as an insurance claims process that links a jumble of functions and data inside [Enterprise Resource Planning], [Customer Relationship Management] and old mainframe legacy systems. The application’s vendor doesn’t matter anymore; the linkages between the applications is the important thing.

Does that sound familiar to anyone? I’d like to couple this with a paragraph from the report by the National Library of Australia on a “Single Business Approach” (discovered via a blog posting by Lorcan Dempsey):

IT Architecture Project Report. 2007. N.d. National Library of Australia. http://www.nla.gov.au/dsp/documents/itag.pdf (accessed March 28, 2007).

The paper defines Single Business Approach in a library context this way:

Even with a service-oriented approach, the Library’s capacity to meet its directions will continue to be eroded as new applications are brought online. As budgets continue to tighten and the Library needs to do more with less, there will come a time when a large proportion of development effort will be spent just maintaining existing applications.

To address this issue, and as part of implementing the service-oriented architecture, it is proposed that the Library regard its digital library services as a single business with a single data corpus that can be deployed in a range of contexts. Rather than developing separate applications to meet a new requirement, each requirement would be viewed as an enhancement to the business that could be deployed across all relevant business contexts.

This is a significant change to the way the Library currently works. As well as resulting in further significant efficiencies for IT staff, it has the potential to bring library staff together in unprecedented ways to work on problems and ideas and to prototype solutions that enhance the user experience regardless of the point of access.

The Australia report focuses on this Single Business Approach along with SOA and the adoption of open source solutions to come up with a vision to support the management, discovery and delivery of the National Library of Australia’s collections. There is a lot here that could be adopted to other similar situations, such as OhioLINK’s.

The text was modified to update a link from http://www.cio.com/archive/061506/soa.html to http://www.cio.com/article/21975/The_Truth_About_SOA on June 9th, 2011.

This is part three of a continuing series on the application of the Service Oriented Architecture (SOA) design pattern to library systems. In the first part, the SOA concept was compared to a transportation network and the basic foundation for defining SOA was set down. The second part described what a “service” in SOA could be and proposed an example using OCLC’s WorldCat interface with item status information being pulled from a library catalog system. That part also left off with a teaser about the juxtaposition of “inventory control system” with “local catalog system” — a foreshadowing of the topic of this post: what to do about the Monolithic (er… “Integrated”) Library System.

How did we get where we are today?

Before the advent of computers, libraries frequently used a card catalog to index its holdings. Computers were used to automate the card catalog, thus the term automation system. Since the late 1980s, multi-tasking modules allowed business functions to be integrated. Instead of having to open up separate applications, library staff could now use a single application with multiple functional modules. ¹

As this article points out, libraries have a grand tradition of applying technology to its business problems. These “integrated library systems” (ILS) — so named for their unified functions of public search, circulation (check-in/check-out), acquisitions, and cataloging (description) — propelled library staff productivity to magnitudes not seen before. “Those systems,” notes Marshall Breeding, “came quite close to delivering comprehensive automation for libraries.”²

So why change? Marshall goes on to say, “the ILS remains shackled to the antiquated print-based paradigm for library automation.” Today’s integrated library system is ill-equipped to handle the desired integration of formats (audio, video, journal article) and delivery mechanisms (online versus on-site) of our users. It is also not a nimble platform; as a single, all-encompassing application, bug fixes and new functionality across the entire must be wrapped in a software distribution to be released to users.

This post is not intended to be an exhaustive examination of the problems with the software design model of current ILS applications; there is a growing body of analysis and commentary on that point. Rather, the rest of this post seeks to point a way forward — a path that involves the dis-integration (“un-integration”? or, as Marshall put it, “re-integration”?) of the integrated library system.

What is the Integrated library System?

Discovery Tool / Indexing Tool

Speaking of Roy’s mantra, I may be showing a bit of a bias here towards one particular solution by pairing the Discovery Tool with an Indexing Tool. The underlying presumption here is that there will be One Common Index for all library content. Given the issues surrounding metasearch (particularly the speed at which a metasearch query can be executed), getting a zippy response for a SOA interaction really requires that an index already exist that is used to answer the query. This bias is fueled, admittedly, by OhioLINK’s fortunate position of storing and serving much of the content under license from publishers/producers from our own servers. When you actually have access to the content and underlying metadata, you can think about creating One Common Index for all of that content. So use of the Discovery Tool is inexorably linked to the Indexing Tool.

Inventory Control Tool

A library’s inventory control challenge is modestly similar to that of a video rental store: we need to track what is within our confines and what has been “rented.” That leads us right into the next tool.

Point-of-Sale (a la video rental) Tool / User Directory Tool

If that analogy holds, then their customer database is like out patron database. In any case, we will need a User Directory Tool where demographic information is kept as well as information about users’ “rentals.” The User Directory Tool can also be a source of authentication and authorization as well as other functions such as storing recommendations and annotations. (More on this topic to come at a later time.)

Ordering and Receiving Tool

Description Tool

Unlike the integrated library systems in use today, however, our next generation Description Tool must be multi-lingual: MARC ⁶, while very expressive to the point of obsessive, cannot capture all of the information about every object. If metadata is going to truly encapsulate the descriptive fidelity of the object, we need to look at standards other than MARC. For example, OhioLINK’s current media content repository has object that are digitized bird calls; the description of those audio files includes elements such as genus and species, the type of microphone used to record the bird call, and a notation of whether the bird was seen or not seen. Odds are those elements will never find their way into MARC, so the Description Tool must be able to pull together schema such as the Nature Services Species Schema, some other schema that describes recording devices, and something that we’ll likely have to make up to represent whether the bird was observed. (See, I told you we were obsessive about description.) In addition, a really good Description Tool would pull in authority files and ontologies of various sorts, beyond the Name Authority File and Library of Congress Subject Headings that enable similar topics to be grouped together. Above all, this tool needs to be easy to use to promote a quick workflow and response SOA Service requests.

What’s Next

Thanks go out to Richard Wallis at Talis and Lorcan Dempsey at OCLC for their comments so far. I’m on the home stretch towards a plateau in this series and which point I’ll take a step back, absorb your observations, and reflect them back out in subsequent postings. Everyone else, please feel free to join in on the conversation as well, whether it be through comments here or commentary elsewhere.

[20060921T0820 Missed the "p" in Lorcan's last name -- my humble apologies, Lorcan.]

The text was modified to update a link from http://en.wikipedia.org/wiki/Integrated_library_system&oldid=76840266#History to http://en.wikipedia.org/wiki/Integrated_library_system?oldid=76840266#History on January 13th, 2011.

Footnotes

1. Adapted from “Integrated library system.” Wikipedia, The Free Encyclopedia. 20 Sep 2006, 19:29 UTC. Wikimedia Foundation, Inc. 20 Sep 2006 http://en.wikipedia.org/wiki/Integrated_library_system&oldid=76840266#History.
2. Breeding, Marshall. “Re-Integrating the integrated library system.” Computers in Libraries. 25(25): Jan 2005. 20-Sep-2006 http://www.librarytechnology.org/ltg-displaytext.pl?RC=11340.
3. A note out to those of you outside the library community who still seem to be reading this series, if the blog statistics are to be any guide: you can help us out with your ideas here — many of us have had our heads buried in ILS systems for so long (my entire career, for instance) that it is difficult to think outside that box. Let me know in the comments if I’ve missed anything.
4. “A Conversation with Werner Vogels.” ACM Queue. 4(4): May 2006. 19-Sep-2006 http://www.acmqueue.com/modules.php?name=Content&pa=showpage&pid=388
5. For a definition and description of Point-of-Sale systems, I recommend this Wikipedia article.
6. Machine Readable Cataloging

This post is the second in a series about the application of the Service Oriented Architecture (SOA) system design pattern to library services. The first post in this series focused on defining “Service Oriented Architecture” using the analogy of a transportation network. This post goes into some detail about what makes a “service” in this architecture and offers an example using a hypothetical use case: a union library catalog (Open WorldCat) making a statement about the availability of a book.

Why are we even looking at SOA?

SOA is a means of organizing solutions that promotes reuse, growth and interoperability. It is not itself a solution to domain problems but rather an organizing and delivery paradigm that enables one to get more value from use both of capabilities which are locally “owned” and those under the control of others. It also enables one to express solutions in a way that makes it easier to modify or evolve the identified solution or to try alternate solutions. ¹

The first sentence is a statement that reflects SOA position as the latest step on an evolutionary path from monolithic to modular computing. In the really old days of electronic computing, every sequence of processing instructions was defined from scratch — you programmed the computer to do everything. Very quickly one realized that the programming code being written contained a lot of redundancy, so “code libraries” were formed that held all of the repeated instructions; no longer did everyone have to write and debug their own code routines to get the current date or write out a character…you used the supplied code libraries.

Gradually, as systems became more and more complex, the computing profession came up with more code libraries and developed higher levels of abstraction to deal with the complexity. “Objects” as programming entities were introduced in the 1960s via Simula 67, a programming language designed for making simulations, and Smalltalk, a programming language developed at Xerox PARC in the 1970s, introduced the term Object-oriented Programming (OOP) to represent the pervasive use of objects and messages as the basis for computation. OOP became the dominant programming methodology during the mid-1980s largely due to the influence of C++, an extension of the C programming language, but for the most part the applications developed using the OOP methodology remained as applications running on a solitary computer system. ²

OOP and SOA are similar in that each seeks to encapsulate activities to an abstract level at which point the abstractions can be reused as building blocks to larger applications. Where they differ, as explained by the OASIS Reference Model for Service-Oriented Architecture ³, is at the level of abstraction: in the OOP paradigm the focus is on packaging data and operations whereas in the SOA paradigm the focus is on “the task or business function &mdash getting something done.” Or, if you will, providing a service.

Definition of “Service”

In the SOA sense of the word, a service is a set of capabilities offered by a service provider through a service definition that makes the capabilities discoverable by a service consumer. The implementation of the service is “opaque” to the consumer, and the invocation of a service itself may seek out the SOA-defined capabilities of another service provider. The real-world consequences of a service invocation can be: 1) information transferred from a provider to a consumer (e.g. “read”); 2) a change in the state of information at the provider (e.g. “write”); or 3) a combination of both.

The amount of information received and/or changed per service invocation can be rather small, and the transaction itself can be rather light-weight. For instance, in an ACM QUEUE interview Werner Vogels, Amazon’s Chief Technical Officer says that when “you hit the Amazon.com gateway page, the application calls more than 100 services to collect data and construct the page for you.” ⁴ If it took seconds rather than miliseconds to respond to a SOA query, Amazon’s architecture would clearly be unsustainable.

Back up to the quotation near the top of this post, the second sentence ends by speaking of “capabilities which are locally ‘owned’ and those under the control of others.” SOA, by defining this abstract notion of service and focusing on the semantics surrounding a service, promotes the exchange of services across organizational boundaries. This is the subject of the following hypothetical use case.

Hypothetical Use Case: Reflection of Local Library Holdings in Open WorldCat

You are all ready to proceed with this plan, ⁵ but there is one sticking point: your local public catalog displays the status of the item (“available”, “checked out”, “lost”, etc.). And WorldCat, although it contains the combined holdings of thousands of libraries, it doesn’t know whether any particular item is available or not — the only thing stored in the WorldCat database is whether an item is owned, not whether it is checked out. The status of the item is only known to the local catalog system.

Enter a Service Oriented Architecture. The local catalog system makes known a service whereby an service consumer can ask for the status of an item. What is returned by the service provider can be as simple as a string of text that is a spelled-out version of the item status or it can be as complex as an XML schema with attributes for “checked out” or “lost” and parsable date fields and so forth. Remember, though — these SOA exchanges need to be processed quickly. As WorldCat is generating the screen of the search results for the user, it calls back to the library catalog system to find the status of the item and then displays it for the user. The result might be something like this:

Okay, but is That What the User Really Wanted?

Available Online

If we see in the description of the item that it is on the web somewhere. In this case, the word “Online” can link right to the item.

Available Onsite; Delivered in 1-2 days

This item is physically held in the library and the inventory control aspect of the library system says that it is available. So the choices to the user are “onsite”, which could link to a page that describes which library holds the item and the approximate location of the item within the library, or “delivered in 1-2 days” for a user service that will pull an item from the shelves and deliver it to an office or home.

Delivery on or about…

This item is checked out. A hold has been placed on it, and when the item is returned (hopefully no later than the original due date) it will be turned around to the next user.

Delivered in 3-4 weeks

This item was declared lost, so the 3-4 week delivery statement represents the estimated time to get the item from another library through interlibrary loan. Ultimately, does the user really care if the item was declared lost? Probably not — they just want to get to the item!

Layers of Services

Delivered in 2-3 days

The item was not available at the local site (so we don’t see an “on-site” link) but it is available from an OhioLINK member institution. The average turn-around time for an OhioLINK physical delivery request is about 2 to 3 days. But again, the user doesn’t care at this point where the item is coming from — their local library or one somewhere across the state; the only thing the user likely cares about is how long it would take to get the item if he or she asked for it to be delivered to their home or office.

But What of the “Integrated Library System”?

The text was modified to update a link from http://en.wikipedia.org/wiki/Object-oriented_programming&oldid=76627897#History to http://en.wikipedia.org/wiki/Object-oriented_programming?oldid=76627897#History on January 13th, 2011.

Footnotes

1. OASIS Reference Model for Service-Oriented Architecture, Committee Specification 1, dated 2 Aug 2006, lines 175-179. The document was retrieved from http://www.oasis-open.org/committees/download.php/19679/soa-rm-cs.pdf on 18-Sep-2006.
2. Summarized version of “Object-oriented programming.” Wikipedia, The Free Encyclopedia. 19 Sep 2006, 18:27 UTC. Wikimedia Foundation, Inc. 19 Sep 2006 http://en.wikipedia.org/wiki/Object-oriented_programming&oldid=76627897#History.
3. OASIS Reference Model for Service-Oriented Architecture, Committee Specification 1, dated 2 Aug 2006, lines 243-259.
4. “A Conversation with Werner Vogels.” ACM Queue. 4(4): May 2006. 19-Sep-2006 http://www.acmqueue.com/modules.php?name=Content&pa=showpage&pid=388
5. For the record, please note the word “hypothetical” sprinkled liberally throughout this example. WorldCat cannot now do the things that are described in this post, the screenshots are unauthorized adaptations of WorldCat web pages, and, to the best of my knowledge, OCLC is not planning on doing anything like what is described here. Although it would be really neat if they could do it….
6. yeah, yeah — I can almost hear the sharp-witted speculations about our dinner-time conversations from those outside the library community.

This post is the first in a series over the next few days that in total will attempt to lay the groundwork for a discussion of applying the Service Oriented Architecture (or SOA) software design pattern to systems and solutions in the library space. It starts with comparing the SOA concept with a multi-modal transportation network. Subsequent posts will outline use cases and describe how SOA can be applied to libraries.

An Analogy: Transportation

Each of these modes of transportation — your car, the shuttle bus, the airplane, and the taxi — came together as you needed them. Or, said another way, when you set out from your home in Cleveland, you had not specified the entire transportation network from exactly which shuttle bus you would take to exactly which taxi you would use. In addition, at the intersection of each of these transportation modes — where one mode stopped and you were not yet at your goal — there is likely more than one option you can use that will take you a step closer to your destination. For example, you could have started from your house with a walk to a light rail station that took you directly into the airport terminal, thus eliminating the car ride and the shuttle bus.

In this analogy, one could call the transportation modes “services.” A service can be defined as “the capability to perform work for another, the specification of the work offered for another, and the offer to perform work for another.” ¹ The car, airplane, bus, taxi, and light rail are all services used to reach the end goal.

Definition

In the same way that the airplane, bus, taxi, and/or light rail came together to form a transportation conglomerate, we can combine units of software applications in new ways to reach goals. This pattern of combining pieces to reach a whole is called “Service Oriented Architecture.” For a formal definition, one can turn to the OASIS Reference Model for Service-Oriented Architecture which says

Service Oriented Architecture (SOA) is a paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains. ²

But what of another alternative to our transportation analogy — you could have driven your car the entire way from Cleveland to Denver, right? That is a single service that would certainly allow you to reach your end goal. Is it the most efficient way, though? The cheapest, fastest, safest? To continue the analogy into SOA, reaching your goal in this way is like “hard coding” an application to do only one thing. By making this choice to structure the solution as a single path, options that may be cheaper and more efficient over some portion of the solution are eliminated.

There are several key aspects to pull out of the OASIS ³ definition that distinguish a multi-modal system from a single tool solution. First is “distributed capabilities” — the concept that the solution’s components are not collocated. In the transportation analogy, the car, bus, airplane, and taxi are spread across the continent. In a system based on the SOA concepts, services are spread across the network. A corollary to the “distributed capabilities” concept is that the components are found and brought together on an ad hoc basis. Just as the how the shuttle bus, airplane, and taxi were brought together in the right sequence of time and space, components of of an SOA system can be discovered and linked at the time of the request of service. In a static system, as with the car-only analogy, the components are pre-defined to run in a single pattern within a closed system.

The second distinguishing characteristic is arguably more important than the first: the distributed components of the SOA system “may be under the control of different ownership domains.” Back to the analogy — there isn’t one entity that owns the car, the bus, the airplane and the taxi. Yet these components, in spite of different business rules and schedules, come together in a solution to transport you from here to there. The same is true in SOA systems; in an e-commerce transaction that results in the shipping of a Bose QuietComfort 2 Noise-Canceling Headset from Amazon.com to your home, you can bet that as part of the execution of that request there are communication between Amazon’s order fulfillment services and UPS’s shipping manifest services.

Next up…

Footnotes

1. OASIS Reference Model for Service-Oriented Architecture, Committee Specification 1, dated 2 Aug 2006, lines 169-171. The document was retrieved from http://www.oasis-open.org/committees/download.php/19679/soa-rm-cs.pdf on 18-Sep-2006.
2. A product of the OASIS Service Oriented Architecture Reference Model Technical Committee (abbreviated as OASIS SOA-RM TC), Committee Specification 1, dated 2 Aug 2006, lines 128-129. The document was retrieved from http://www.oasis-open.org/committees/download.php/19679/soa-rm-cs.pdf on 18-Sep-2006.
3. OASIS is the Organization for the Advancement of Structured Information Standards — a not-for-profit, international consortium that sponsors the development and adoption of standards for security, e-business, and for public sector and application-specific markets.

I’ve been tasked to write a whitepaper envisioning a Service Oriented Architecture for OhioLINK’s services and operations. I’ve found a bit of information through my own networking and searching, but in putting out this list and asking for additions I want to be sure that I’m not already missing the holy grail of documents that I could just rebrand as an OhioLINK document. (With appropriate permission, of course.) I’m looking for strategic/explanatory documents over technical documents, although the latter will undoubtedly be useful in later iterations and derivatives of the whitepaper.

CDL Common Framework

The CDL Common Framework is an open, services-oriented technical architecture that provides an integrating framework for services related to digital libraries. As a layered architecture it aims to separate front-end tools from back-end services from underlying data storage so that different components can be reused in multiple applications, reducing the time and money it takes to develop and maintain code. The Common Framework also supports easy integration of local and third party tools and services through a plug-in approach. It is available through both machine (i.e. web services) and human interfaces.

A ‘service oriented’ view of the JISC Information Environment

e-Framework for Education and Research

DLF Service Framework for Digital Libraries

A progress report for the DLF Steering Committee from 17 May 2005. Has headings of Context, Why is a Digital Library Service Framework needed?, Some benefits of a Service Framework for DLF and research libraries, What has the DLF Service Framework Group done so far?, Some open issues, and Recommendations moving forward.

ALA TechSource’s Web Services and the Service-Oriented Architecture

“In an era of computing dominated by the World Wide Web, technology referred to as Web services stands as a key one for allowing computers to communicate machine to machine, program to program….Web services make it easy to connect all types of computer applications to each other. As you will see throughout this report, Web services deliver a foundation of interoperability greatly needed in a world where computer services and digital information exist in many different forms and flavors,” says Breeding in his “Introduction” in the latest issue of Library Technology Reports, “Web Services and the Service-Oriented Architecture.”

The text was modified to remove a link to http://www.cdlib.org/inside/projects/common_framework/ on November 17th, 2010.

The text was modified to remove a link to http://www.cdlib.org/inside/projects/common_framework/ on November 17th, 2010.

The text was modified to update a link from http://www.e-framework.org/resources/ to http://www.e-framework.org/Resources/tabid/609/Default.aspx on November 17th, 2010.

The text was modified to update a link from http://www.e-framework.org/resources/ to http://www.e-framework.org/Resources/tabid/609/Default.aspx on November 17th, 2010.

The text was modified to update a link from http://www.e-framework.org/resources/eframeworkrV1.pdf to http://www.e-framework.org/Portals/9/docs/papers/Briefing060802.pdf on November 17th, 2010.

The text was modified to update a link from http://www.e-framework.org/resources/eframeworkrV1.pdf to http://www.e-framework.org/Portals/9/docs/papers/Briefing060802.pdf on November 17th, 2010.

The text was modified to update a link from http://www.e-framework.org/resources/SOAandWorkflow2.pdf to http://www.e-framework.org/Portals/9/Resources/SOAandWorkflow2.pdf on November 17th, 2010.

The text was modified to update a link from http://www.e-framework.org/resources/SOAandWorkflow2.pdf to http://www.e-framework.org/Portals/9/Resources/SOAandWorkflow2.pdf on November 17th, 2010.

The text was modified to update a link from http://www.techsource.ala.org/ltr/web-services-and-the-service-oriented-architecture.html to http://www.alatechsource.org/ltr/web-services-and-the-service-oriented-architecture on November 17th, 2010.

The text was modified to update a link from http://www.techsource.ala.org/ltr/web-services-and-the-service-oriented-architecture.html to http://www.alatechsource.org/ltr/web-services-and-the-service-oriented-architecture on November 17th, 2010.

FEDORA Workflow Working Group Meeting

18-Jun-2006, University of Virginia

Attending: Grace Agnew, Rutgers U.; Chris Awre, U. of Hull; Dan Davis, Harris Corp.; Richard Green, U. of Hull; Peter Murray, OhioLINK; Matthias Razum, FIZ Karlsruhe; Bill Parod, Northwestern U; Adam Soroka, U. of Virginia; Thorny Staples, U. of Virginia; Ross Wayland, U. of Virginia

Review of Minutes from 6 Dec 2005

A revised assumption was offered: “FEDORA is typically an application where reads typically exceed writes.”

It was also noted that the working group charter and the assumptions make reference to “BPEL” as the only workflow orchestration engine, excluding other tools such as JBPM. This is not intentional, and does not necessarily reflect a decision to use only BPEL-based engines. The participants noted, however, that no one envisions writing a new workflow engine from scratch.

Participant Desires and Status Review of Workflow-related Activities

FIRE/Workflow 0.1 client update

The December meeting envisioned work on the NSDL NCCS/FIRE client as the starting point for embedding workflow capabilities into FEDORA. The FEDORA core development team looked at both ActiveBPEL and JBPM for integration into the FEDORA-SF and determined that within their available time that the core developers needed to hardwire the workflow into the client with the desire to apply a workflow orchestration engine in a later release.

FEDORA Repository update

Max Plank Society: eSciDoc

Overview

Max Planck Society is made up of 82 institutes, each with many working groups. The ESciDoc_Project is building one repository for all of these institutes and working groups in disciplines ranging from humanities to sciences. In interviews with users, the eSciDoc development team has determined that each user group will likely want a near infinite diversity of combination of workflow steps. eSciDoc developers have built a framework layer around FEDORA and will be putting a workflow engine for human-in-the-loop and process-to-process steps.

Update

Harris Corporation

Update

U. of Hull: RepoMMan

Desires

The RepoMMan project seeks to give researchers some ‘extra services’ almost without them being aware of it: managing work in progress; access for themselves and small collaborative group; safety (backup) of work — and be as easy to use a network drive on their desktop plus interface for repository-related activities. The RepoMMan project is funded to bring a BPEL workflow to a FEDORA system to fulfill a part of this vision.

The development team has surveyed and interviewed researchers about how they did research to understand what repository processes and features might help them. In the autumn, the development team plans to conduct the same survey and interview techniques to get the teaching/learning needs and administrator needs. They are managing the evolving expectations for a general document management system from university’s administration.

Technology

Discussion

Overlap with FEDORA Preservation Services Working Group

It was noted and discussed that the Preservation Services Working Group has begun defining the ‘events’ associated with objects in the repository and was looking for an ‘event management engine’ of some sort. It is clear that there is some overlap in these working group activities and some coordination would be beneficial to both groups. The Preservation Services Working Group refers to this activity as “Event Management” which seems to be equivalent to our “Orchestration Management”.

Areas where the Workflow Working Group can make a meaningful contribution

It was observed that collectively we’re reaching for the final plateau of the desired service, but there are many layers from where we are now to there that needs to be built (e.g. message-oriented middleware). In the course of the afternoon discussion, two work areas were identified.

First, defining and coding the building blocks of repository activities as discrete, WSDL-addressable web services. This work needs to be done in conjunction with the FEDORA Preservation Services Working Group — building on their existing work and aid in specifying these events as discrete web services. These ‘event’ building blocks are likely to be generic to many workflow orchestration engines or other mechanisms to build workflow steps into an application.

Second, building a reference implementation of a workflow stack into the FEDORA-SF using open source components. It is envisioned that this stack will consist of a “workflow engine” layer (BPEL-based, JBPM-based, other, or a combination of these); a “workflow management system” that provides the human interface to instantiate and track workflow scripts in flight (interfaces could be JSR-168 portlets, dedicated web servlet, a desktop Swing-based app, etc.); and a “workflow editor system” for creating new workflow scripts.

Next steps

1. Create a narrative of these two work areas
2. Participants review Preservation WG workflows
   1. Peter will get with Ron Jantz to begin the dialog
   2. WG members will review the Preservation WG “events” as the basis of creating the orchestration “building block” activities

The text was modified to remove a link to http://www.fedora.info/wiki/index.php/Working_Group:_Workflow:Minutes-20060618.

The text was modified to remove a link to http://www.fedora.info/wiki/index.php/Talk:Working_Group:_Workflow:Minutes-20060618.

The text was modified to remove a link to http://www.fedora.info/wiki/index.php/User:Richard_green.

The text was modified to remove a link to http://www.fedora.info/wiki/index.php/User:PeterAtOhioLINK.

The text was modified to remove a link to /wiki/index.php/User:Matthias.

The text was modified to remove a link to http://www.fedora.info/wiki/index.php/User:Rwayland.

The text was modified to remove a link to http://www.fedora.info/wiki/index.php/ESciDoc_Project.

The text was modified to remove a link to http://www.fedora.info/wiki/index.php/University_of_Hull.

The text was modified to remove a link to http://www.fedora.info/wiki/index.php/Working_Group:_Preservation.

While in UNC-CH for JCDL I’ve had occasion to rant with/at some people about the state of the integrated library system marketplace — including, of course, how we got into the spot we’re in and how we might get out of it (and those people were kind enough to engage in the rant). Along comes a series of posts from Casey Bisson and Nicole Engard ultimately pointing back to John Blyberg’s “ILS Customer Bill-of-Rights” that is singing the same tune. There still seems to be a desire for a solution from an existing vendor, and in fact that was part of counter-points brought up by some on the receiving end of the ILS-must-go rant. (Paraphrased: ‘No one can satisfy the need of a library like a library automation vendor’ and ‘As libraries we’re not strong enough to take on the task of building the next ILS ourselves.’) Yet there does seem to be this mounting pressure to get control again over our data and how we present it to patrons.

What’s In A Name?

That started a great discussion. There was this comment from Peter Schlumpf:

The catalog is going to be with us in one form or another. One thing that never ceases to amaze me is how the library field is sooo quick to throw overboard useful tools just for the sake of something different. The ILS in its present form has LOTS of room for improvement but it doesn’t mean we have to hide it behind other labels or have to turn it into some nebulous concept that doesn’t mean anything….The catalog is one of the the main interfaces to the library that all patrons use. How can we make that experience most productive? We need to pay lot of attention to that.

I would challenge the notion that the OPAC is a “useful tool” — if it was, our patrons would still be using it. As it is, anecdotal evidence suggest that the OPAC is the last thing they would choose to use. And this reply by Alexander Johannesen:

I don’t think the OPAC will go away, nor that it absolutely must, but the very idea of an OPAC is based on knowing what our patrons want; books that we’ve cataloged. But all too often we have no idea what they want; all we’ve got are assumptions. I think we’ve come a long way, but the time to look anew to what purpose the OPAC serves certainly is ripe.

I’ll agree with Alexander, and I hope it is for the same reason. If you want to call the ILS/OPAC an inventory of our physical collections, then that’s okay. By definition that means that it is not an inventory of our digital holdings. We can have a debate about whether AACR is a good way to describe a physical item. (MARC, on the other hand, as a container of a descriptive record has got to go.) On the other hand, you’ve got to be prepared to add to that debate what is the proper way to describe other assets — and it ain’t AACR/MARC.

Then there was this exchange between Eric Hellman (first) and Teri Sierra (second):

Is it just me, or does anyone else feel that the very idea of having a catalog as an important component of a library smacks of retrograde thinking? To my mind, in a clean-slate NG Library architecture, the library catalog should only exist as a facade that recognizes of the vanity of libraries and the people who fund them.

I can think of no technical justification for library catalogs as we look forward. If not the next generation, then the next-next generation of libraries. The functions that exist today in library catalogs need to be pushed in two directions- toward the user on one hand, and towards global registries on the other.

Along with Teri, I’ll disagree with Eric on the extreme perspective of a (presumably single) global registry. One thing that I suspect I’ve learned with the OhioLINK DRC project is a phrase I can’t claim to have coined called “institutional ego.” It is important that an institution maintain its identity. But I also disagree with Teri about the solution:

I agree that we need to be thinking about the way libraries will look in the future. But to say that the library catalog is serving only the purposes of the people who fund them and feed on their vanity, is pretty strong and misguided. Maybe you ought to sit with a reference librarian and ask why and how the catalog and OPAC are used.

Listening to ourselves (librarians in general) is what is getting us into this situation in the first place. We keep focusing on increasingly small improvements with a relatively low return on investment while users of a whole new modality of communication (call it “Web 2.0″, if you will) look over their shoulder (if we’re lucky) and wonder why we’re not keeping up.

The Disaggregated Library System

1. An inventory control system. We’ve got to know what we’ve got and where it is.
2. A “point-of-sale” system. Yeah, we’re not selling our books — it’s more like a rental. So we’re up to an inventory control component plus a rental tracking component.
3. An acquisitions/accounts-payable system. We’ve got to buy stuff. But I think we can buy record numbers — record numbers that point back into the inventory control system for things that we have, if you will, “backordered.”
4. A description system. Let’s face it — the obsession with which a library describes an item is second to none. Of all of the pieces that could be found off-the-shelf, this one is the least likely to be found. If after all of the discussion of the pieces we come back to needing a really good description system, then let’s focus our energies on that rather than the rest.

With a foundation based on these components — a “Library Service Oriented Architecture” if you will — we’ll be in a much better position to meet the needs and desires of the users today and change to meet their new needs and desires in the future. It’s not everything we need (I hear there is the rumblings of a debate on the cost effectiveness of serials control systems — do we have to have one of those), but it is a place to start.