Earlier this week, Aaron Swartz of the Internet Archive announced the demonstration website of the Open Library project, a new kind of book catalog that brings together traditional publisher and library bibliographic data in an interface with the user-contributed paradigm of Wikipedia. Okay, I’ll pause for a moment while you parse that last sentence. Think you got it? Read — and watch — further.

Open Library has been mentioned a bit in the blogs this week, but not to the extent I thought was worthy of the magnitude of the project. So I recorded a screencast introduction (in Flash Video format below followed by a rough transcript) that looks at not only the browsing side of the system but also the record editing and record creation aspects of Open Library. As I say at the end of the recording, Open Library is one of those mind-bending, assumption-shattering projects that, at least for me, is challenging my thoughts about what library service could be and should be. Congratulations to the team at the Internet Archive, and I’m looking forward to future enhancements and directions for the project.

Rough transcript of the screen cast is below.

Introduction

Hello, and welcome to this screencast overview of the Open Library project. Open Library is an effort by the Internet Archive to create a comprehensive catalog of every book. As the project’s “about” page says, “Not every book on sale, or every important book, or even every book in English, but simply every book.” The about page goes on to describe the characteristics of Open Library project — that it is a project enabled by Internet technology because no physical space could hold it and that it aims to pull together records from publishers and libraries. It is also a project in the same vein as Wikipedia, meaning that any user can create and edit the records in the system.

In this overview, I’ll lead you through searching and browsing the Open Library’s demonstration website from the perspective of any modern library catalog interface. Then I’ll show you where it deviates from traditional library catalogs by exposing the underlying wiki nature of the database; we’ll examine the changes that users have made and we’ll even make a change ourselves. And finally I’ll show the process of creating entirely new records in the system. So let’s get started.

Searching

We’re looking at the home page of the Open Library project demonstration site. In the middle is a search box with a suggested search — “tom sawyer adventure”. That is a good suggestion so we’ll click on Go. Open Library returns a classic, relevance ranked list of matching records with some book covers along the left side and a faceted list of refinements along the right. So right away you can see that there are some authority control problems here in the author names — Twain comma Mark, Mark comma Twain, and Twain comma Mark with birth and death dates — and here in the language field. But I have high hopes that the developer team will find some intriguing ways to address these problems.

Back over here in the results area we have the various editions of Samuel Clemen’s “The Adventures of Tom Sawyer” — let’s pick the 1876 edition to see the full record display — there. We have the publisher, publication date and place, language, and a summary or review of sorts at the bottom. We also see signs of the availability of full text — over here in the options box there is a download from the Internet Archive link, a “Scan Sponsor” field here and a “View this book” graphic. This is one of the items scanned by the Open Content Alliance and made available by the Internet Archive through the Open Library project. A very nice interface for paging through the book. So one could imagine that the Open Library could become the primary vehicle by which Open Content Alliance materials are made available to the public.

So let’s go back here to the metadata page. Remember in the introduction that I said that the data was malleable in a wiki-like fashion. The Open Library developers created a system that allows for user-contributed updates (a la Wikipedia) to fielded data (like your classic bibliographic record). The two hints that the record is modifiable are this big edit button in the middle of the metadata and this more subtile “[history]” link near the top of the page. Let’s start with the history link to see what has been done to this record.

This page should look familiar to those who have worked with wikis before. It shows a listing of edits that were made to this record from most recent to the very first edit, who made the change (identified by IP addresses in this case because the people making the changes were not logged to an account at the time), an editor-supplied comment about what was done, and when the change was made. We can go back in time and see the page at a particular version through the links under the “When” column, or we can use the compare function to see the difference between two version. In the case of the changes between version 2 and version 3, we see that the editor added “Canada” as the place of publication. On this page you start to see the fielded nature of this wiki structure, but the best place to see it is look at the record edit screen itself.

These are all full-text fields on this page with no controlled vocabulary. You’ll note the absence of any MARC field names here, but as you scroll through you’ll see the evidence of MARC and AACR2 in the field labels. Down at the bottom is an edit summary to describe the changes made to the record, then save, preview and delete version buttons — all classic wiki functions.

Editing

Now, I’d like to show the full record editing process, but since I don’t have this Mark Twain book in hand, I’m going to bring up another record that I created yesterday — “Eric Meyer on CSS“. Before showing the editing process, let’s linger here a moment at the “options” box along the right side. Since this is a more modern book (as opposed to the Tom Sawyer book we saw first), there are additional options here for purchasing the book through these various vendors or borrowing the book through a very nice link into Open Worldcat and two web-based book trading sites.

But back to the metadata. There is one error and one omission in this record — perhaps this is a subtile demonstration of problems that creep in with user-generated content. First, the error, is that there is an extra digit in the ISBN-10 field, which is a big problem because the links in the options box use the ISBN as a linking field and at the time of this recording they don’t work. They will work in a moment, though. The second problem is that I forgot to put in the publication date. But hey, no problem, all I need to do is “Edit” this record.

So we are back to the edit screen, and I’m going to scroll down and fix the ISBN-10 field like so, then scroll down a little further and add the publication date. Then I’ll scroll all the way to the bottom and type in an edit summary — “Fixed the ISBN and added a publication date” — and hit save. We’re now back at the metadata display screen and the link to Open Worldcat now works. So, as an aside, one wonders what the folks in Dublin, Ohio, think about this. It is competition on the one hand since Worldcat is also aiming to be the most comprehensive catalog of books in the world. On the other hand, perhaps there is room for cooperation by somewhat getting vetted changes to Open Library records into the OCLC union catalog. Who knows?

Creating a New Record

Alright, back to current reality. Let’s add a record to Open Library, and in this case I’m going to use an ARL SPEC Kit that I wrote a number of years ago called “Library Patron Privacy”. First let’s run a search in Open Library to see if it is there, and no, it isn’t. The only way I’ve figured out how to enter a new item is to go to the URL where the page would be located and get the classic wiki “This page does not exist. Create it?” message.

One of the quirks I found in the system is that I have to create author wiki pages before book wiki pages — otherwise I’ll get a Python error message on the screen. I’ve reported this to the Open Library developers, but in the meantime just know authors need to be created before their books. Which is to say that authors have wiki pages in Open Library in addition to books. The structure of URLs to Open Library author pages is the letter “a” followed by a slash followed by the author’s last, first and middle names separated by underscore characters. So I’ll go to the URL of that form, then click on the “Create it” link.

Now here is one of the tricky parts of the existing interface. The page type starts as “type/page”, and as you can see it doesn’t have any of the fielded elements that we saw in previous examples. What you have do do is change the page type to “type/author” and then you get the fielded HTML form. So I’m going to go through here and fill in some of the parts. Then go down to the edit summary field and write a summary of this change, then click save. Now that Open Library knows who I am, let’s create the record for the book.

You’ve seen the structure of the URLs to book pages before — a “b” followed by a slash followed by the book title with spaces replaced by underscore characters. I’ll put that in the URL field and get the default page type. This needs to be changed to “type/edition” in order to get the bibliographic record fields. There. Now I’ll go through here and enter the data. When we get down to the author field we enter it in the same format that we used to create it — an “a” followed by a slash followed by the name with spaces replaced by underscores.

So we’ll just finish up here and come down to the edit summary field, put something in here, and hit save. This record is now in the system, and you can see the public display here along with the links on the right because I entered an ISBN. I haven’t quite figured out how to get a cover image into the system yet — I expect there is a file upload interface somewhere, but I haven’t found it.

Conclusions

So that’s all there is, and I don’t say that in a way to denigrate the work that has been done by the development team so far. As the URL and site banner indicate, it is a demonstration system — and a compelling demonstration it is. All sorts of questions immediately come to mind, of course — will there be a controlled vocabulary or authority control built into the system, can data be exported out of records — and, for that matter, can end-users bulk import data into the system, are there Web2.0 niceties like tagging and RSS feeds in the works, and so forth.

Even with all of those questions, Open Library is one of those mind-bending, assumption-shattering projects that, at least for me, is challenging my thoughts about what library service could be and should be. Congratulations to the team at the Internet Archive, and I’m looking forward to future enhancements and directions for the project.

The text was modified to remove a link to http://demo.openlibrary.org/b/adventures_of_Tom_Sawyer on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/b/adventures_of_Tom_Sawyer?m=history on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/b/adventures_of_Tom_Sawyer?b=3&a=2&m=diff on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/b/Eric_Meyer_on_CSS on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/a/Murray_Peter_E on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/b/Library_Patron_Privacy on January 19th, 2011.

A recent posting in the Chronicle of Higher Education “Wired Campus” section describes the new iTunes U portal, “a spot on the site that will collect college lectures, commencement speeches, tours, sports highlights, and promotional material, all available at no cost.” (If you have iTunes on your desktop/laptop, you can use this link to visit iTunes U in the iTunes Store.) Now, according to the Apple press release, “content from iTunes can be loaded onto an iPod® with just one click and experienced on-the-go, anytime, making learning from a lecture just as simple as enjoying music.”

How about iTunes U as a content delivery platform for libraries. What kind of content could we put into iTunes U? Here at OhioLINK, we have the Borror Laboratory of Bioacoustics Recorded Sounds and Foreign Language Video Instruction that are open to the public. (Granted, we might have to do some file format conversions to meet requirements for iTunes U.) With more students and researchers using devices such as iPods and services such as iTunes U, as long as we are not entering into an exclusive agreement for delivering such content, why not meet the users where they are.

A post by Bill Harris at “Dubious Quality” with the title Information got caught up in my Technorati filter for disruptive change in libraries. Geoff Engelstein, a colleague of Bill’s mentioned this in an e-mail:

We were a generation of information explorers. They [Geoff's thirteen– and eleven-year-olds] are a generation of editors.

The context is a reflection on Bill’s part of the trials and feelings of success when conducting research: “you’d have to pull out a rack in the card catalog according to the alphabetized subject and flip through the cards. If you got lucky, the title of a book or a brief description would point you in the right direction. Then you had to actually find the book, skim through it, and hope that you’d find some information.” Bill even includes a link to a bibliographic instruction page showing how an actual card catalog works.

Geoff’s observation is that “the big skill [his kids] are learning is what information to dump and what to keep” — hence the comparison to the work of an editor.

This has to get one thinking — are the services we are building for the information explorers or the information editors?

“Participatory Digital Libraries” is the name of a talk Paul Jones, Director of ibiblio.org, gave this morning at OCLC’s Kilgour Auditorium. Known as “The Public’s Library,” ibiblio is a large, diverse digital library. His talk offered insight on how ibiblio works and commentary for applying the same successful techniques in library projects. This is a summary of the key points of his talk; errors of transcription and omission are undoubtedly my own.

Digital Archives in Action

In one part of the presentation, Paul describes the array of curatorial practices used for ibiblio collections. First, however, he distinguished his activity from that of tradition archives. “Archives,” in his words, “look for interesting collections; we look for interesting people creating interesting collections in interesting ways.” Put another way, ibiblio experimented with getting to the front end of the content creation process by finding people as they were creating the collections.

They collected the collectors and empower them with tools to build the content. As a result of seeding many such experiments, ibiblio has many different, idiosyncratic ways of building collections.

• folkstreams.net — This is an IMLS-funded project that is a “rescue mission” for documentary films created in the 1960s. The collection is curated by well-respected film makers.
• Roger McGuinn’s Folk Den — A single individual that has been recording a folk song a week for 12 years. The contents of the collection? “What ever interests Roger McGuinn.”
• Documenting the American South — After residing for 10 years on ibiblio, this collection is now being migrated to a UNC Libraries server. Curated by professional archivists, it has lots of money invested in best-practice output for digitization, description, and standards-based (e.g., TEI Lite) artifacts.
• Moonshine — A project created by students creating on online exhibition to materials in the university archives.
• Degree Confluence — A participatory infrastructure that allows people to contribute pictures and diary entries from points on the earth’s surface where integer values of latitude and longitude intersect.

Paul also mentioned a bit about blogs. His observation is that researchers are not putting content into Institutional Repositories, yet they are putting content into blogs, photo sharing sites, and the like. Many of these content systems that researchers are using have RSS capabilities, and the metadata in an RSS feed is at times better than what we ask users to provide in an IR. Perhaps our “IR” capabilities could be about RSS harvesting this content and preserving it.

Five ‘Big Ideas’

In a subsequent paper published in the Communications of the ACM, Paul described the ibiblio effort this way: “By adopting not only the open-source tools, but also the open-source philosophy encouraging community interaction and contributor involvement, digital libraries can open new horizons to new communities as well as greatly improve traditional services.” He touched on five ‘big ideas’ and how they impact the digital library arena.

1. The Long Tail. ibiblio made many little bets by seeding many collectors, as opposed to focusing on blockbusters (as say what the Smithsonian or California Digital Library does).
2. Democratizing Innovation. No one is ultimately happy with how you present things to them. Enabling users to modify things creates happier and more satisfied users, and better products result from looking at what users do.
3. Wealth of Networks. A community working together to remix and create knowledge increases the knowledge capital in a system. This book also discusses the value of market and non-market interactions.
4. Free Culture. Discusses the social utility of information wanting to be free. Who owns the cultural heritage and who preserves it and why these are important questions.
5. Small is the New Big. He touched on this book briefly, but my notes and memory fail me at this point.

(See Paul’s presentation page for demonstrations of these ideas in action — each of these books has a free online version and/or a remix and/or a blog that discusses the ideas presented both before and after formal publication.)

Paul also mentions one that he hopes spend some time working on: Laws of Simplicity. We have simple ideas (e.g. the web’s core HTML and HTTP standards), but not simple tools for our users.

Experts And (not ‘Versus’) Passionate Amateurs

This heading is my summary of a portion of his talk, and the theme struck a chord with me. How does the role of experts intersect with the role of people just trying to get their jobs done? He states quite emphatically that there is a middle ground between formal description (‘cataloging’) and user-driven description (‘tagging’). How do we build a community of professionals and enthusiasts who fill the space between content objects with new knowledge? This is already starting to play out in other arenas — how will “Encyclopedia of Life” and “Wikipedia” interact? In one “material will…be authenticated by scientists”¹ and in the other “thousands of people have contributed information to different parts of this project, and anyone can do so”².

The text was modified to remove a link to http://www.alaoweb.org/cgi-bin/dada/mail.cgi/archive/announce/20070501093221/ on January 20th, 2011.

The text was modified to update a link from http://www.hyperionbooks.com/titlepage.asp?ISBN=1401302378 to http://www.hyperionbooks.com/book/the-long-tail-revised-and-updated-editionwhy-the-future-of-business-is-selling-less-of-more/ on January 20th, 2011.

The text was modified to remove a link to http://www.free-culture.cc/ on January 20th, 2011.

Footnotes

1. http://www.eol.org/faqs.html#faq3.2
2. http://en.wikipedia.org/wiki/Wikipedia:Overview_FAQ#Who_is_responsible_for_the_articles_on_Wikipedia.3F

Jenny Emanuel, Electronic Services Librarian at University of Central Missouri, posted an invitation to complete a survey on how library professionals think of themselves to several mailing lists. As part of the ALA Emerging Leaders 2007 program, she is part of a team look for options on rebranding the librarian profession in the digital world. This looks like it will have interesting results; if you consider yourself a “library professional” take the survey yourself: http://www.surveymonkey.com/s.asp?u=371423757475.

Out of all of the questions, number 10 struck me as the heart of the matter:

10. How strongly do you agree with the following statements?

• Librarians excel in customer service.
• Librarians are on the cutting edge of the publishing industry.
• Librarians are experts at designing databases and special computer resources.
• Librarians assist people in learning new technologies.
• Librarians wear cardigans and glasses.
• Librarians are experts at exploiting the Internet.
• Librarians are on the cutting edge of technology tools (i.e. blogs, wikis, podcasts).
• Librarians are highly educated professionals.
• Librarians need to do a better job of marketing themselves.
• Librarians need a new image for the digital age.

I did have philosophical differences with question #13:

13. Please fill in the blanks in the sentence:

Only a librarian can deliver _____ to _____.

I couldn’t get past the mental image of librarian-as-gatekeeper. I could answer the question “only a library can deliver…” — but that is an entirely different question.

Believe it or not, I also had a problem answering question #2:

Would you describe yourself as a librarian?

I think I used to be a librarian — primarily at a time before I had a library degree. Now I think I’m a ‘library technologist’ — working on the application of technology in the library arena. Since working in a consortial office puts me at some distance from actual users, I’m not sure I can claim the title ‘librarian’ any more.

Interesting questions, no? Then go take the survey yourself. The results will be presented in a poster session at ALA Annual on Friday, June 23.

Update 19-Jan-2011: I’ve decommissioned this service. Talkr seems to be unavailable, and I haven’t had time to find a replacement.

If reading the thoughts of the Jester via this blog wasn’t enough, you can now hear this witty (witless?) insights read to you through your favorite podcast player. I’ve been messing with some technology this weekend for a mashup of my own.

First, start with the Talkr service, which will take the text of your RSS feed posts and convert them to an audio file of a computer generated voice speaking the text to you. The audio file is included as an attachment in a new RSS feed of your post content. In the sidebar of DLTJ, you can subscribe to audio version of this blog using the “Subscribe to Postcast” graphic.

But wait, there’s more! A second graphic leads to a magic iTunes URI that will add the audio version of the Jester as a podcast in your iTunes setup. That one was a little more difficult in that the RSS coming out of Talkr doesn’t have all of the necessary iTunes markup (and in some cases, the markup itself needs to be futzed with). So a quick pass through an XSLT script gives Apple the feed that they need to make this work in iTunes.

You might ask “Why iTunes?” followed closely by “Don’t they represent the new evil empire by the way they lock subscribers into using their hardware only?” and ending with “Do you really want to encourage that?” First, as an aside, I’ve already sold part of my soul to Apple — I’ve been sold on their operating system and portable computer hardware for years. But that is really okay because I own a piece of Apple’s soul — 96 pieces to be exact (unless the stock has split again since the last time I checked). Secondly, and more relevant perhaps, is a pragmatic belief to meet the users where they are. If there are potential listeners of DLTJ using iTunes and I can make it easier for them to listen to this blog on the platform of their choice without impacting those who don’t want to make iTunes their choice, then that’s fine. I had this discussion with a fellow librarian about making the OhioLINK Digital Video Collection visible through iTunesU; as long as it is not an exclusive arrangement and the no-iTunesU users are served just as well as those who access the content through iTunesU, then I don’t see a problem with “meeting the users where they are.” (Oh, and yes, I still do not own an iPod — I have a generic MP3 player that suits me just fine.)

By the way: “Can You Hear Me Now?” refers to the tagline for Verison Wireless commercials, a wireless carrier in North America. I’m not sure how well that translates outside this region.

Update [20070330T1030]: Along with the WordPress upgrade, I tweaked the RSS feeds so that the version that Talkr sees does not have the FeedBurner FeedFlares in them. The flares were coming across in the audio as “The original story included an image at this point.” Repeatedly. Quite annoying, I imagine.

The text was modified to remove a link to http://talkr.com/ on January 19th, 2011.

The text was modified to remove a link to http://dltj.org/audio.rss on January 19th, 2011.

The text was modified to remove a link to http://www.apple.com/education/products/ipod/itunes_u.html on January 19th, 2011.

Last month was an interesting month for discussion and news of JPEG2000 as an archival format. First, there was a series of posts on the IMAGELIB about the rational for using JPEG2000 for master files. It started with a posting by Tom Blake of Boston Public Library asking these questions:

What can I do with a JPEG200 that I can’t do with a TIFF, a good version
of Zoomify, and a well-designded DAMS?

Don’t you need to rely on a proprietary version/flavor of JPEG2000 and a
viewer to utilize its full potential?

Bill Snead from Duke offered pointers in a follow up message to Aware’s “Why JPEG2000?” whitepaper, the Olsen-Melville case study from the University of Connecticut, and Princeton’s statement of use of JPEG2000 as surrogates from a TIFF master.

I leapt into the conversation by offering an opinion that with JPEG2000 is a compelling replacement for a TIFF-based practice because:

1. JPEG2000 offers a single format for both access and preservation of digital imagery. Eliminating the complexity of managing derivatives in the creation, processing, and delivery of images is a good thing, I think. Said another way, the archival master can be the same file as the production master with the access derivatives being generated on-the-fly based on the inherent-to-JPEG2000 scaling capabilities. Also, if one’s preservation master is your access master, then one will know very quickly if something is wrong with the preservation master — it no longer renders in your access system.
2. It is truly lossless compression — one can get back to a bit-for-bit identical TIFF if desired.
3. The standard has built-in support for bundling (embedding) metadata with the image bitstream. For the long term, that means we just need to be concerned about managing one file and not a series of image-plus-metadata files based on a (opinion-> “fragile”) strategy of holding file names constant and varying the file extension. ¹
4. JPEG2000 is at the core of a number of technologies that are bigger than we are with consituencies that have deeper pockets than ours to ensure JPEG2000 support is perpetuated, well, indefinitely. For instance, JPEG2000 is at the heart of the DICOM medical imaging standard that is gaining wide adoption. Also, motion pictures are being delivered to theaters using Motion JPEG2000. ²
5. The JPEG2000 is an open standard with defined and emerging protocols for guaranteeing compliance with the standard. (This is one area, I think, that our community could help support the JPEG2000 standard, by the way. We have a vested interest in making sure images that we capture with one software are processable by other bits of software.) If the text of the standard is printed to acid free paper or etched onto a nickel plate, we’ve got a very good shot at creating a program to read these files in whatever future computing system comes about.

One note about #3 above. I would not argue that an access system based around JPEG2000 files would use the embedded metadata as the core of the DAMS. Rather, I would proposed that the authoritative version of the metadata would be in the JPEG2000 file itself. The surrounding DAMS database would be used for efficiencies in accessing and manipulating the metadata. If the DAMS was blown away (or if one wanted to migrate to a new DAMS), the subsequent asset management system could be built based on the metadata stored in the JPEG2000 artifacts. Any kind of modification to the metadata wouldbe written back to the JPEG2000 file and the requisite checksums would need to be recalculated. (If were storing our metadata in the same trusted fashion as the image bitstream, then we would be calculating checksums on the metadata anyway.)

One of the concerns about JPEG2000 is some language from the JPEG2000 website about how “undeclared and obscure submarine patents may still present a hazard…” to open use of the standard. This seems like lawyer CYA to me as nothing has come up that I’m aware of in the seven years after the standard was ratified. Here, too, there are deeper pockets than that of libraries and cultural heritage institutions that would fight something like this. And, if in the end it is found that a patent would cause an embargo ‘unlicensed’ versions of JPEG2000 codecs for some period of years, we can always run a batch conversion back to TIFF until the embargo period is up and/or something else better comes along.

To sum up, I believe that JPEG2000 is a compelling and real replacement for TIFF. I’m not the only one, though. In section 2.3 (“The Establishment of a Scalable Workflow for Digitizing a Wide Variety of Materials”) of the Harvard University Library Open Collections Program Final Report, Harvard announced the “Adoption by Office of Information Systems of the jpeg2000 standard for archival master images stored in the Digital Repository Service.” That is farther than the previous early adopter’s statement by the Library of Congress in the Digital Format Registry entry for JPEG2000 where they said “[a]s adoption and implementation increases in other sectors, the use of the format for the Library’s master images may become more and more appealing.”

The text was modified to remove a link to http://ocp.hul.harvard.edu/report/final/pages_6.html on January 19th, 2011.

The text was modified to remove a link to http://ocp.hul.harvard.edu/report/final/ on January 19th, 2011.

Footnotes

1. It could equally be argued, of course, that wrapping the image codestream and the associated metadata in a METS or MPEG21 file achieves the same end. I’m not sure which is better — the metadata in the image file or the image file inside a metadata wrapper.
2. I can’t confirm this because it seems wildly outrageous, but a reliable source told me that very soon motion picture film stock will not be produced anymore and that all films will be shot and edited digitally. I don’t know if Motion JPEG2000 is used in the filming or editing process, but if it is then I’m even less worried about the future of reading JPEG2000 files.

Two previous posts on dltj.org have described the OAI Object Reuse and Exchange (ORE) project and the theory behind what has become known as the ‘Web Architecture’. These two areas meet up now in this post which describes the issues surrounding the raw Web Architecture as applied to a web of scholarly communication and a basic outline of what the ORE project hopes to accomplish.

Problems With the Web Architecture

The concepts behind the Web Architecture are clearly successful. I believe it is safe to assert that the genius behind the creation of Tim Berners-Lee and his colleagues is the simplicity with which the vast web of world wide connections has sprung into existence with relatively little coordination. That said, some of the fundamental concepts behind the Web Architecture do not fit well with the web of interactions known as “scholarly communication.”

The first issue is aggregation. The Web Architecture does not provide a way to describe a finite set of Resources and relationships as a citable complex digital object resource structure. As scholarly communication becomes more than just papers — it can also now include data sets, supplementary graphics, primary source material as well as references to previously publish objects — this concept of aggregation becomes important.

Second, the relationships between Resources are usually untyped and link type ontologies are not well defined. (Capital-R “Resources” carries the meaning of this term as defined by the standards related to the Web Architecture; see the previous posting for a definition and examples.) Is this link within the text of a document a citation? A data set? An explanatory graphic? In general, it is not good practice to try to guess the relationship based on the contents of the URI itself. In fact, the Web Architecture technical report suggests “agents making use of URIs SHOULD NOT attempt to infer properties of the referenced resource.” ¹ In the absence of additional standards (such as OAI-ORE) layered on top of the core Web Architecture, this notion of “URI Opacity” encourages independence between an identifier in one document and the Representation of another object. One thing the ORE work seeks to accomplish is to build a framework for the semantics of links between objects in a scholarly communication environment.

The Problems From a Scholarly Communication Perspective

Take, for example, a paper in an example repository as described by this graphic. The article, identified by the number “012345″ has six Resources with five Representations: an HTML splash page generated by the repository software (Resource #1), the article in PDF format (Resource #2), the article in Postscript format (Resource #4), metadata in Dublin Core XML (Resource #5), metadata in BibTex format (Resource #6), and the article in a format decided by agent/server content negotiation (#3). ². Keep in mind that views of digital object must be bound to Resources in order to be reference-able (e.g. they must have URIs).

Although it may be possible to infer that all six Resources are related by comparing the leading fragment of the URIs, the Web Architecture principle of URI opacity dictates that we shouldn’t make those assumptions. Furthermore, even if we could determine that they are related based on examining the URIs, we do not have a consistent vocabulary to define that relationship. Is “…meta/bibtex” the citation data for this article or is it the list of citations used in the article?

Modeling Complex Objects

Because the Web Architecture does not allow for the definition of a boundary for a compound digital object, the ORE project proposes the definition of a Resource — called the ORE Model, for lack of a better name at the moment — that formally expresses a bounded aggregation of resources and relationships that corresponds to a compound digital object. Put another way, an instantiation of the ORE Model is a map of other resources that expresses the boundaries of the compound digital object. A URI identifies the compound digital object — the ORE Resource — and a service request on that URI returns a Representation that is some serialization of the ORE Model.

The preliminary version of the model describes two types of relationships: intra-aggregation relationships (inside the boundaries of the compound digital object) and inter-aggregation relationships (to Resources outside the boundary of this compound digital object). The intra-aggregation relationships come in two forms: hasPart (where one Resource contains other Resources, such as books contain chapters or journal issues contain articles) and hasView (where the target Resource is a semantically equivalent presentation format, such as Word and PDF versions of an article). The inter-aggregation relationship has only one verb, “hasRelationshipTo,” which simply means the target of the relationship is considered outside the boundaries of the complex digital object. From a base verb of “hasRelationshipTo” other communities can apply specialized relationships.

The result describes a connected sub-graph with a finite set of resources and relationships among those resources to form a compound digital object plus relationships to resources that are external to the aggregation. With that in place, we can consider services that can be applied to portions of the graph.

ORE Services

One half of the work of the ORE project is to define a model for compound digital objects in a Web Architecture environment. The other half of the work is to define the meaning of services that exchange instances of the model to form the basis of a Web Architecture-aware scholarly communication environment.

Conceived based on the experiences with the OAI Protocol for Metadata Harvesting (PMH), there are three archetypes of services.

• Harvest: a request for a batch of instances that correspond to the ORE model from a set of ORE Resources.
• Obtain: A request for an instance that corresponds to the ORE Model from a specific ORE Resource.
• Register: A request to add new nodes or relationships to an ORE aggregation.

Service requests against the ORE Resource URI are the access points for these activities.

For more information…

This is a basic introduction to the work of the technical committee so far. For a more in-depth view into the outcomes of the first face-to-face meeting, including expanded definitions and examples of what was outlined here, see the Report of the January 2007 ORE-TC Meeting. In addition, there is a interview with Herbert van de Sompel recorded at the CNI 2006 Fall Task Force to go along with a project briefing presented at that meeting. (Keep in mind that these were recorded and presented before the first technical committee meeting, so some of the concepts of the implementation have changed.) Pete Johnson, a member of the ORE technical committee, posted his thoughts on the topic on his blog: Prospecting for ORE, More ORE, and More ruminations on compoundness and complexity (and metadata). The presentation slides from Carl Lagoze’s talk at Open Repositories 2007 are also available, of which Chris Wilper and Jim Downing posed summaries and reactions. Also keep an eye on the OAI-ORE tag on Technorati for more updates and reactions.

The text was modified to remove a link to http://technorati.com/tag/OAI-ORE on January 19th, 2011.

Footnotes

1. “Architecture of the World Wide Web, Volume One” paragraph #98. Available from http://www.w3.org/TR/2004/REC-webarch-20041215/#p98 Accessed Feb 16 2007.
2. The Web Architecture allows for the Representation of a Resource to be decided through content negotiation between the agent/browser and the server.

There was a great crowd at the University of Windsor “Future of the ILS” symposium. The presentation is available from http://dltj.org/wp-content/uploads/2006/11/200611-uwindsor-soa/. An outline of the presentation is given below with links into the presentation slides. Amanda Etches-Johnson has also posted a great summary of the presentation on her blog, “Blog Without A Library.”

The presentation is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/2.5/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA. Other rights are available; please contact the author for more information.

• What Is Service Oriented Architecture?
  • In Summary…
  • It’s About an Architecture for Designing Systems
  • It’s About Discrete Business Processes
  • It’s About Reuse
  • It’s About Web Services as a Means to an End
  • It’s About Standards and Open Protocols
• Why Should I Care?
  • ‘The ILS Market Implodes! Film at 11…’
  • Can We Afford Not To Care?
  • The Alternatives
• What Might a Library SOA Look Like?
  • Ground Rules
  • The Business Processes of a Traditional ILS
  • Traditional and Non-traditional Services for Discovering Content
  • Traditional and Non-traditional Services for Describing Content
  • Non-traditional Uses of Traditional ILS Functions
  • Non-Traditional Locations of Traditional and Non-Traditional ILS Functions
• Who Else Is Talking About This?
  • Who Else Is Talking About This? (Blogs)
  • Who Else Is Talking About This? (Articles)
  • Who Else Is Talking About This? (DLF Project)
  • Who Else Is Talking About This? (CDL Project)
  • Who Else Is Talking About This? (eFramework Project)

[20061116T1222 Added a link to Amanda Etches-Johnson's summary. Thanks, Amanda!]

[20100317T1539 Replaced links to slides. The host where the presentation was originally located is no longer in operation.]

This is part three of a continuing series on the application of the Service Oriented Architecture (SOA) design pattern to library systems. In the first part, the SOA concept was compared to a transportation network and the basic foundation for defining SOA was set down. The second part described what a “service” in SOA could be and proposed an example using OCLC’s WorldCat interface with item status information being pulled from a library catalog system. That part also left off with a teaser about the juxtaposition of “inventory control system” with “local catalog system” — a foreshadowing of the topic of this post: what to do about the Monolithic (er… “Integrated”) Library System.

How did we get where we are today?

Before the advent of computers, libraries frequently used a card catalog to index its holdings. Computers were used to automate the card catalog, thus the term automation system. Since the late 1980s, multi-tasking modules allowed business functions to be integrated. Instead of having to open up separate applications, library staff could now use a single application with multiple functional modules. ¹

As this article points out, libraries have a grand tradition of applying technology to its business problems. These “integrated library systems” (ILS) — so named for their unified functions of public search, circulation (check-in/check-out), acquisitions, and cataloging (description) — propelled library staff productivity to magnitudes not seen before. “Those systems,” notes Marshall Breeding, “came quite close to delivering comprehensive automation for libraries.”²

So why change? Marshall goes on to say, “the ILS remains shackled to the antiquated print-based paradigm for library automation.” Today’s integrated library system is ill-equipped to handle the desired integration of formats (audio, video, journal article) and delivery mechanisms (online versus on-site) of our users. It is also not a nimble platform; as a single, all-encompassing application, bug fixes and new functionality across the entire must be wrapped in a software distribution to be released to users.

This post is not intended to be an exhaustive examination of the problems with the software design model of current ILS applications; there is a growing body of analysis and commentary on that point. Rather, the rest of this post seeks to point a way forward — a path that involves the dis-integration (“un-integration”? or, as Marshall put it, “re-integration”?) of the integrated library system.

What is the Integrated library System?

Discovery Tool / Indexing Tool

Speaking of Roy’s mantra, I may be showing a bit of a bias here towards one particular solution by pairing the Discovery Tool with an Indexing Tool. The underlying presumption here is that there will be One Common Index for all library content. Given the issues surrounding metasearch (particularly the speed at which a metasearch query can be executed), getting a zippy response for a SOA interaction really requires that an index already exist that is used to answer the query. This bias is fueled, admittedly, by OhioLINK’s fortunate position of storing and serving much of the content under license from publishers/producers from our own servers. When you actually have access to the content and underlying metadata, you can think about creating One Common Index for all of that content. So use of the Discovery Tool is inexorably linked to the Indexing Tool.

Inventory Control Tool

A library’s inventory control challenge is modestly similar to that of a video rental store: we need to track what is within our confines and what has been “rented.” That leads us right into the next tool.

Point-of-Sale (a la video rental) Tool / User Directory Tool

If that analogy holds, then their customer database is like out patron database. In any case, we will need a User Directory Tool where demographic information is kept as well as information about users’ “rentals.” The User Directory Tool can also be a source of authentication and authorization as well as other functions such as storing recommendations and annotations. (More on this topic to come at a later time.)

Ordering and Receiving Tool

Description Tool

Unlike the integrated library systems in use today, however, our next generation Description Tool must be multi-lingual: MARC ⁶, while very expressive to the point of obsessive, cannot capture all of the information about every object. If metadata is going to truly encapsulate the descriptive fidelity of the object, we need to look at standards other than MARC. For example, OhioLINK’s current media content repository has object that are digitized bird calls; the description of those audio files includes elements such as genus and species, the type of microphone used to record the bird call, and a notation of whether the bird was seen or not seen. Odds are those elements will never find their way into MARC, so the Description Tool must be able to pull together schema such as the Nature Services Species Schema, some other schema that describes recording devices, and something that we’ll likely have to make up to represent whether the bird was observed. (See, I told you we were obsessive about description.) In addition, a really good Description Tool would pull in authority files and ontologies of various sorts, beyond the Name Authority File and Library of Congress Subject Headings that enable similar topics to be grouped together. Above all, this tool needs to be easy to use to promote a quick workflow and response SOA Service requests.

What’s Next

Thanks go out to Richard Wallis at Talis and Lorcan Dempsey at OCLC for their comments so far. I’m on the home stretch towards a plateau in this series and which point I’ll take a step back, absorb your observations, and reflect them back out in subsequent postings. Everyone else, please feel free to join in on the conversation as well, whether it be through comments here or commentary elsewhere.

[20060921T0820 Missed the "p" in Lorcan's last name -- my humble apologies, Lorcan.]

The text was modified to update a link from http://en.wikipedia.org/wiki/Integrated_library_system&oldid=76840266#History to http://en.wikipedia.org/wiki/Integrated_library_system?oldid=76840266#History on January 13th, 2011.

Footnotes

1. Adapted from “Integrated library system.” Wikipedia, The Free Encyclopedia. 20 Sep 2006, 19:29 UTC. Wikimedia Foundation, Inc. 20 Sep 2006 http://en.wikipedia.org/wiki/Integrated_library_system&oldid=76840266#History.
2. Breeding, Marshall. “Re-Integrating the integrated library system.” Computers in Libraries. 25(25): Jan 2005. 20-Sep-2006 http://www.librarytechnology.org/ltg-displaytext.pl?RC=11340.
3. A note out to those of you outside the library community who still seem to be reading this series, if the blog statistics are to be any guide: you can help us out with your ideas here — many of us have had our heads buried in ILS systems for so long (my entire career, for instance) that it is difficult to think outside that box. Let me know in the comments if I’ve missed anything.
4. “A Conversation with Werner Vogels.” ACM Queue. 4(4): May 2006. 19-Sep-2006 http://www.acmqueue.com/modules.php?name=Content&pa=showpage&pid=388
5. For a definition and description of Point-of-Sale systems, I recommend this Wikipedia article.
6. Machine Readable Cataloging