We’re taking a break this week from the HarperCollins e-book story; although the commentary continues from librarians (and a few authors), there hasn’t been anything new (that I’ve seen) from HarperCollins itself. There is still plenty more to look at, though. First up is a report from the health care sector on the applicability of open source and open systems. Next is an interview with a financial analyst that sees the end of the “big deal” for library journal subscriptions. And lastly is a list of web archive services that you could use to find old copies of web pages.

Feel free to send this to others you think might be interested in the topics. If you find these threads interesting and useful, you might want to add the Thursday Threads RSS Feed to your feed reader or subscribe to e-mail delivery using the form to the right. If you would like a more raw and immediate version of these types of stories, watch my FriendFeed stream (or subscribe to its feed in your feed reader). Comments and tips, as always, are welcome.

Open Source, Open Standards, and Health Care Information Systems

Recognition of the improvements in patient safety, quality of patient care, and efficiency that health care information systems have the potential to bring has led to significant investment. Globally the sale of health care information systems now represents a multibillion dollar industry. As policy makers, health care professionals, and patients, we have a responsibility to maximize the return on this investment. To this end we analyze alternative licensing and software development models, as well as the role of standards. We describe how licensing affects development. We argue for the superiority of open source licensing to promote safer, more effective health care information systems. We claim that open source licensing in health care information systems is essential to rational procurement strategy.

This might be a useful data point for libraries considering the adoption of open source for their mission-critical applications. Two U.K. authors have published a report that reviews general benefits of open source and open standards, noting in one heading that “Open Standards Facilitate Competition Between Open Source Software and Proprietary Software”. They also compare the open source software development practices with those of proprietary software development and look at barriers to the adoption of open source software. A great deal of the analysis is particular to health care information systems, but the report would be a useful template to applying the same analysis to core library systems. [Via ACM TechNews]

Reynolds, Carl J., & Wyatt, Jeremy C. (2011). Open Source, Open Standards, and Health Care Information Systems Journal of Medical Internet Research, 13 (1) DOI: 10.2196/jmir.1521

The Demise of the Big Deal?

Interview question: You, however, believe that publishers will simply have to accept that their revenues are going to fall, because there really is no more money?

Claudio Aspesi: I have no doubt that — over time — adjustments would be made. But it remains to be seen if they need all the 2,200/2,400 journals that the each of the largest publishers maintain today.

You know, my job is not to pass judgement on how people run their business or to decry capitalism, only to advise investors whether they should buy or sell stocks.

I can observe, however, that there is something unhealthy about an industry which has managed to alienate its customers to the point their membership associations increasingly focus time and attention on how to overturn the industry structure. It is not a good thing to have your customers spend their time trying to put you out of business.

Richard Poynder interviews Claudio Aspesi, a financial analyst based at the sell-side research firm Sanford Bernstein. Aspesi issued a report last year that was critical of the financial outlook of Reed Elsevier and more recently has downgraded the outlook to “underperform”. This interview gets into the reasoning behind Aspesi’s decision.

Archives of Dead Web Pages: Wayback, Cache, and More

The Web changes constantly, and sometimes that page that had just the information you needed yesterday (or last month or two years ago) is not available today. At other times you may want to see how a page’s content or design has changed. There are several sources for finding Web pages as they used to exist. While Google’s cache is probably the best known, the others are important alternatives that may have pages not available at Google or the Wayback Machine plus they may have an archived page from a different date. The table below notes the name of the service, the way to find the archived page, and some notes that should give some idea as to how old a page the archive may contain.

Although this list is over three years old, many of the services are still active. One addition of note is a beta test version of the Internet Archive’s Wayback machine; it includes an improved interface and a more up-to-date archive of pages.

It is an all e-books edition of DLTJ Thursday Threads this week. The biggest news was the announcement of the policy change by HarperCollins for ebooks distributed through OverDrive. Beyond that, though, was an announcement of a new sharing model and program through the Internet Archive. Lastly is a slidecast recording of a presentation by David Lewis on the future of library collections.

Before continuing, a quick apology and explanation. E-mail readers received a pair of extra Thursday Threads messages and RSS subscribers got a dump of unrelated posts; I’m sorry. The cause was an update of this blog’s WordPress software to version 3.1 and a conflict (maybe this one) with the SimpleTags plugin. I believe all is well, but I won’t know until this post is published.

Feel free to send this to others you think might be interested in the topics. If you find these threads interesting and useful, you might want to add the Thursday Threads RSS Feed to your feed reader or subscribe to e-mail delivery using the form to the right. If you would like a more raw and immediate version of these types of stories, watch my FriendFeed stream (or subscribe to its feed in your feed reader). Comments and tips, as always, are welcome.

HarperCollins Puts 26 Loan Cap on Ebook Circulations

In the first significant revision to lending terms for ebook circulation, HarperCollins has announced that new titles licensed from library ebook vendors will be able to circulate only 26 times before the license expires.

Mention of the new terms was first made in a letter from OverDrive CEO Steve Potash to customers yesterday. He wrote [emphasis in original]:

[W]e have been required to accept and accommodate new terms for eBook lending as established by certain publishers. Next week, OverDrive will communicate a licensing change from a publisher that, while still operating under the one-copy/one-user model, will include a checkout limit for each eBook licensed. Under this publisher’s requirement, for every new eBook licensed, the library (and the OverDrive platform) will make the eBook available to one customer at a time until the total number of permitted checkouts is reached.

Though the letter leaves the publisher unnamed, HarperCollins confirmed today to [Library Journal] that it is the publisher referred to.

In an odd one-two punch, this past week saw a disturbance in the status quo of e-book licensing. The first punch came in the letter from OverDrive [PDF] (part of which is quoted in the Library Journal article excerpted above). The second in that Library Journal article when we learned that the publisher pushing for the change of terms is HarperCollins. Since then it has been the source of a great deal of discussion by librarians and a few authors, much of it in the form of tweets with the hash-tag “#hcod” (short for HarperCollinsOverDrive). Damage control comes in the form of open letters from OverDrive and HarperCollins. There has been a call for a boycott. Bobbi Newman, one of the first to jump on the story, is maintaining a list of news articles and commentary.

As you can see, much has already been said about the issue, and since collection development is not my specialty, you probably shouldn’t look to me for an informed opinion. (If pressed, I will suggest that it is a perfectly reasonable collection development policy to not buy access to material with terms that are not in the library’s and patron’s best interest.) Instead, there is so much oddness in this new policy that I find I can’t put myself in HarperCollins’ shoes. First, using OverDrive as a proxy for announcing this policy change seems wrong (and, frankly, unfair to OverDrive). Then as word spreads, you don’t make your own announcement, but rather talk to a reporter from Library Journal. Then as news spreads through the day, you send a single tweet. In fact, you don’t really publicly respond until five days after the twitter universe and biblio-blogosphere have been talking about it. And it is pretty much a non-engaging, public relations response. (You do get credit, though, for allowing open comments on your blog post. But some of that credit is taken back because you aren’t using a company branded blog. Really? A typepad.com blog? One of the tenants of most information literacy courses I’ve seen is to look for the source of the information, and it requires extra effort to take this blog seriously because it isn’t in the harpercollins.com domain space.)

If I were to guess, this seems like a trial balloon that was badly floated. I certainly can’t fault HarperCollins for trying something new in the ebook licensing world, but this one has fallen flat.

Internet Archive and Library Partners Develop Joint Collection of 80,000+ eBooks To Extend Traditional In-Library Lending Model

Today [February 22, 2011], a group of libraries led by the Internet Archive announced a new, cooperative 80,000+ eBook lending collection of mostly 20th century books on OpenLibrary.org, a site where it’s already possible to read over 1 million eBooks without restriction. During a library visit, patrons with an OpenLibrary.org account can borrow any of these lendable eBooks using laptops, reading devices or library computers. This new twist on the traditional lending model could increase eBook use and revenue for publishers. …

Any OpenLibrary.org account holder can borrow up to 5 eBooks at a time, for up to 2 weeks. Books can only be borrowed by one person at a time. People can choose to borrow either an in-browser version (viewed using the Internet Archive’s BookReader web application), or a PDF or ePub version, managed by the free Adobe Digital Editions software. …

Publishers selling their eBooks to participating libraries include Cursor and OR Books. Books purchased will be lent to readers as well as being digitally preserved for the long-term. This continues the traditional relationship and services offered by publishers and libraries.

This press release from the Internet Archive largely went unnoticed on the eve of the #hcod onslaught. It was covered in the Chronicle of Higher Education’s Wired Campus blog and in Library Journal. The latter has a few more helpful details: “IA founder Brewster Kahle and director Peter Brantley also told LJ that small independent publishers Cursor, OR Books, and Smashwords will donate ebooks license-free to the Open Library for lending to all Open Library members. With this venture, IA hopes to establish a “first-sale precedent” for e-lending, according to Brantley.” One must be from one of the participating libraries to check out books. My experience with most Internet Archive efforts is that the initial announcement is very subtle and not picked up widely, then slowly grows to something substantial. I expect this project will follow much the same path and will have a noticeable imprint on the profession in a few years.

Slidecast of David Lewis’ “Collections Futures” Talk

• Context
  • The Big Shift
  • Interlude with Clay Shirky
  • A Bit of Disruptive Innovation Theory
• Collections in “A Strategy for Academic Libraries in the First Quarter of the 21st Century”
• What Will Be Easy and What Will Be Hard

So far in DLTJ Thursday Threads I’ve intentionally avoided pointing to items inside this blog — preferring to link to events, resources, and conversations elsewhere. I’m going to make sort-of-an-exception in this case because what I’m ultimately pointing to is not my work. It is a slidecast (recorded audio synchronized to slides) of David Lewis’ presentation at the 2010 Annual RLG Partnership Meeting. Starting with a foundation from John Hagel III, John Seely Brown and Lang Davison called the “Shift Index” [PDF], Clay Shirky’s “How Social Media Can Make History” TED Talk, and Clayton Christensen’s disruptive innovation theories, David walks through the possibilities for three strategic issues facing academic libraries: Complete the migration from print to electronic collections; Retire legacy print collections; and Migrate the focus of collections from purchasing materials to curating content. The slidecast is about 75 minutes long and well worth the time as a thought-provoking view of what libraries should be doing to survive the next few decades.

This year the ALCTS Forum at ALA Midwinter brought together three perspectives on massaging bibliographic data of various sorts in ways that use MARC, but where MARC is not the end goal. What do you get when you swirl MARC, ONIX, and various other formats of metadata in a big pot? Three projects: ONIX Enrichment at OCLC, the Open Library Project, and Google Book Search metadata.

Below is a summary of how these three projects are messin’ with metadata, as told by the Forum panelists. I also recommend reading Eric Hellman’s Google Exposes Book Metadata Privates at ALA Forum for his recollection and views of the same meeting.

ONIX Enrichment at OCLC

Renee Register, Global Product Manager for OCLC Cataloging and Metadata Services, was the first to present on the panel. Her talk looked at a new and evolving product at OCLC on the enhancement of ONIX records with WorldCat records, and vice versa. ¹

As libraries, Renee said “our instincts are collaborative” but “our data and workflow silos encourage redundancy and inhibit interoperability.” Beyond the obvious differences in metadata formats, the workflows of libraries differ dramatically from other metadata providers and consumers. In libraries (with the exception of CIP and brief on-order records) the major work of bibliographic production is performed at the end of the publication cycle and ends with the receipt of the published item. In the publisher supply chain, bibliographic data evolves over time, usually beginning months before publication and continuing to grow for months and years (sales information, etc.) after publication. Renee had a graphic showing the current flow of metadata around the broader bibliographic universe that highlighted the isolation of library activity relative to publisher, wholesaler, and retailer activity.

Diagram of the Process of Enhancing ONIX Records, from OCLC Services for the Publisher Supply Chain Webinar, August 2009

In her experience, Renee said that libraries need ways to enable our metadata to evolve over time and allow for publisher-created metadata to merge effectively with library-created metadata. The bibliographic record needs to be a “living, growing” thing throughout the lifecycle of a title and beyond. In concluding her remarks, she offered several resources to explore for further information: the OCLC/NISO study on Streamlining Book Metadata Workflow, the U.K. Research Information Network report on Creating Catalogues: Bibliographic Records in a Networked World, the Library of Congress Study of the North American MARC Records Marketplace, the Library of Congress CIP/ONIX Pilot Project, and the OCLC Publisher Supply Chain Website.

From MARC to Wiki with Open Library

Karen said right at the start that the Open Library project is different from most of what happens in libraries — it is “someone outside the library world making use of library data” — although the goal is arguably the same as others — “One web page for every book ever published.” As such, the Open Library isn’t a library catalog as librarians think of it in that it is not a representation of a libraries inventory. It has metadata for every book it can know about and a pointer to places where the book can be found, including all of the electronic books in Internet Archive (Open Content Alliance, Google Public Domain, etc.) as well as pointers back to OCLC WorldCat. Karen’s role for the project is that of “Library Data Informant.” The Internet Archive decided that they needed someone who understood library data in order to try to use it. From Karen’s perspective, she is trying to be a resource for project but not give them any guidance on how to implement the service. She is curious to see what the project would do when bibliographic data is viewed from a non-librarian perspective. If they have questions, or if they have assumptions about data that are wrong, then she intervenes.

Karen went on to briefly describe the Open Library system. Open Library doesn’t have records; rather, it has field types and data properties. In this way, it uses semantic web concepts. “Author” is a type, “Author birthdate” is another type, and so forth. There are no set field types, so if the project gets data from source for which a type doesn’t yet exist, it can create a new one. Each type can have data properties such as string, boolean, text, link, etc. Nothing is required and everything is repeatable. Everything — types, properties, and values — gets a URI (a URI is an identifier like a URL, but conceptually a superset of the universe of URLs). Titles, authors, subjects, author birthdates, and so on have URIs. Lastly, the underlying data structures are based on wiki principles: all edits are saved and viewable, anyone can edit any value, anyone can add new types or properties, anyone can develop their own displays, etc.

The data that is now in Open Library came from a variety of sources. They started with a copy of books from the Library of Congress, and continue to receive the weekly updates. They performed a crawl of Amazon’s book data. They have gotten some from publishers, libraries, and individual users. The last is perhaps the most interesting because it is mainly people outside the western world who are otherwise having trouble getting their works recognized.

Problems, Issues, Challenges, and Opportunities with the Data

Names -

“These library forms of names? Honestly no one but us can stand them.” Even something as simple as the form of last-name-comma-first-name is troublesome. No one else uses this form of the name: Amazon, Wikipedia, etc. In processing these, any information between parenthesis has been deleted, birth and death dates move into separate field types.

Titles -

In working with the Open Library developers, this is one place that Karen tried insisting on applying a library practice: knowing the initial article. For us, this is important for sorting books in alphabetical order. The developer response — why do we have to sort in alphabetical order? “Where else but library catalogs to we see things sorted in alphabetical order? Not in Google, not in Amazon, not anywhere. Alphabetical order is not in the mindset anymore.” They also found that the title might include extraneous data. Amazon, for instance, appends the series title in parenthesis to the main title. This is a demonstration of how other communities are not as concerned about strongly typing and separating information into fields. Amazon, of course, has reasons for series information into the main title: it helps sell books.

Product dimensions -

Publishers and distributors need to know characteristics of an item such as height, width, depth, and weight; they, of course, need to put it in a box and ship it. Libraries, concerned about placing the item on the shelf, record just height. Recording pagination is different, too: libraries use odd notations “ill. (some col)” and “xv, 200p.” versus simply “200 pages.”

Birthdates -

Librarians use birthdates to distinguish names; if there is no need to distinguish a name, birth and death dates are not added. Someone looking at this from the outside would ask ‘Why don’t all authors have birth and death dates?’ This can be useful information for viewing the context of an item, not just to distinguish author names. Open Library ran author names against Wikipedia to pick up not only birth and death years, but also the actual dates.

Subject headings -

Open Library using Library of Congress Subject Headings was out of the question. In processing the data, the Open Library developers just broke them apart into segments and used them. But because they were able to do data mining on the subject field types, they did find statistical relationships between the disassembled precoordinated headings and were able to present those to the user.

The View of the Data -

Rather than a traditional library view of long lists of author-title, the Open Library (in its next version coming in February) will have several different views into the mass of data: Authors; Books (what we would call FRBR ‘manifestations’); Works; Subjects; and eventually places, publishers, etc. For example, when searching for an author one would get the author page. On it would be all of the works from the author as well as other biographical information. It looks similar to a WorldCat identities page, except it is the actual user interface built into the system. Similarly, every work will have a page, and at the bottom of it one will see all of the editions of the work. Also, each subject will have a page, and one will see a list of works with that subject as well as authors who write on that subject. As Karen said, “The subject itself becomes an object of interest in the database, not just something that is just tacked on to the bottom of the library record.”

Data mining -

With the data in this format, it is possible to perform data mining actions against it. For instance, simple data mining such as country of publication, popular places that appear, etc. When they had the problem of author names — knowing when to reverse surname and forname — they ran the names against Amazon and Wikipedia and retained the ones where they found the order of the entry was the same. The Open Library developers are also experimenting with data mining to find publisher names. Publisher names, of course, vary dramatically, but by using ISBN prefixes they can pull together related items into a “publisher” view.

Karen suggested watching the Edward Betts’s site, one of the developers of the Open Library project with an eye on the data mining aspects. She said it is fun to look at our data when it can be viewed from this different point-of-view. She also said to watch out for a new version of the Open Library website coming in February.

Google Book Search Metadata

On the input side, Google is getting bibliographic metadata from over 100 sources in a variety of formats. MARC records are coming from libraries, union catalogs, commercial providers (OCLC), publishers/retails (one publisher supplies records in MARC format). Google also gets ONIX records from commercial providers (such as Ingram and Bowker), publishers, and retailers. Google is especially interested in data from non-U.S. retailers because it is a source of information about books published outside the United States; it helps facilitate discovery of items that they may not otherwise encounter in the publisher and library programs. Google also receives records in a variety of “idiosyncratic formats” — for example, publisher-contributed metadata (via the Publisher Partner Program); information associating books with jacket images; name authority records (from LC); reviews; popularity signals (sales data as well as anonymized circulation data from some library partners, useful for feeding into the relevancy ranking algorithm); and internally-generated metadata (for instance, whether a book is commercially available or not). Google processes all of this information to come up with a single record that describes a book. At this point they have over 800 million bibliographic records and one trillion bits of information in those records.

All of these records from all of these sources are processed and remixed with Google’s parsing algorithms about twice a week. The first step is to transform the incoming records into a “less verbose format” for storage and processing. It is a SQL-like structure that allows elements of the metadata to be queried. Records are then parsed to extract specific bits of information, transform the bits as necessary, and write the information to an internal “resolved records” data structure (a subset of the data coming from the input formats). In the presentation, Kurt had examples of how making inferences from data coming from both MARC and ONIX can be troublesome. Parsing also involves extracting “bibkeys” from the records to aid in matching across sources of data. Four types of identifiers are extracted from bibliographic records: OCLC numbers, LCCNs, ISBNs, and ISSNs. They provide usually useful signals when matching bibliographic and help with assertions that two records describe the same manifestation. Google also tries to parse item data when present in records representing multi-volume works, enumeration and chronology. They will also treat barcode as a form of a “bibkey” if they get it from a library. The parsing algorithm will also split records containing multiple ISBNs representing different product forms (e.g. hardback, paperback, etc.).

With all of this data parsed into records, Google starts its clustering process where records are examined and attached to each other. Bibkeys provide significant evidence for relating records to each other, but bibkeys are not always present in a record (non-U.S. records and older records frequently contain no bibkeys). The algorithms then fall back on text similarity matching using title, subtitle, contributor and other fields such as publisher and publication year. The results are clusters of records representing the same manifestation. An algorithm then attempts to derive the “best-of” record for a single cluster from all of the parsed input records. This is done in a field-by-field voting process based on the trustworthiness of individual fields from record sources.

Kurt went into some of the challenges facing the team building the clustering and best-of record creation algorithms. For instance, in dealing with multivolume works they know of 5 numbering schemas with 3 number types in 15 different languages. Enumeration is now showing in the public display, but the development team is still working with unparsable item data due to inconsistent cataloging practices between institutions…and sometimes inconsistencies within an institution. Another problem is non-unique identifiers. In the current data set ISBN 7899964709 is shared by 75 books and ISBN 7533305353 is associated with 1413 books. There are also poor quality or “junk records”. Kurt said his favorite was “The Mosaic Navigator” by Sigmund Freud published in 1939. These are hard to identify with an algorithm, and they rely on reports of problems that enable the developers to go in and “kill” the troublesome record. Another example is a book by Virginia Woolf where the incoming record had conflicting information; it had two 260 fields that contained different dates (1961, correct, and 1900) with fixed field information that strongly suggested that 1900 was the single date of publication. When the data problem is systematic, they can identify it and compensate for it. Kurt’s example for this case was “The United States Since 1945″ published in 1899. This one was highlighted in Geoffrey Nunberg’s criticism of Google Books metadata. In this case, there was a source of metadata from Brazil that when they didn’t know the date of publication would use 1899. When Google went back and looked at the date distribution of books there was a huge spike in 1899. Once Google knew about it they were able to go in and kill that information from that source of records. ³

In closing, Kurt said that Google is committed to engaging with the library community on improving metadata and metadata processing.

The text was modified to update a link from http://www.niso.org/publications/white_papers/Stream lineBookMetadataWorkflowWhitePaper.pdf to http://www.niso.org/publications/white_papers/StreamlineBookMetadataWorkflowWhitePaper.pdf on January 19th, 2011.

The text was modified to remove a link to http://www.oclc.org/speakers/bios/register_renee.htm on February 11th, 2011.

Footnotes

1. For those not familiar with ONIX, it is a suite of standards promulgated by EDItEUR for the interchange of information on books and serial publications. It is primarily used as the communication channel between the publishing industry through distribution chains to retail establishments.
2. By the way, it seems like BISAC is an acronym for “Book Industry Systems Advisory Committee”, the former name of the Book Industry Study Group.
3. A side note: Google isn’t the only one tripped up by this. If one searches for the ISBN of the item, 0195038487, you get to more than one site that has the same incorrect publication date. At least Google is attempting to clean up the data!

Ah, it is the beginning of September when thoughts turn to going back to school, the days turn a little colder (in the northern hemisphere) and the smell of lawsuit briefs is in the air. Well, okay — the latter might not be what you expect, but this is a special September, after all. Postponed from May, the deadline for filing comments in the Google Book Search settlement is coming up. And everyone is weighing in (“again” for some) on the details of the settlement. A couple of highlights.

The American Library Association (ALA), the Association of College and Research Libraries (ACRL) and the Association of Research Libraries (ARL) again offered its support for the settlement, if only the court would promise to extend vigorous oversight of pricing and privacy practices of Google and the Books Rights Registry. This came in the form of a supplemental filing to the brief the three organizations filed in May (just prior to the first comment deadline).

Google Search for Open Book Alliance

Lest you think the fun be over too soon, the deadline for filing briefs has been extended yet again from close of business tomorrow (Friday, September 4th, 2009) until 10:00am Tuesday. Apparently, the court’s electronic filing system will be unavailable from 2pm today until 8am on Tuesday the 8th. The main settlement website says explicitly that the deadline for rights holders to opt out of the settlement remains September 4th.

But seriously, if you are looking for thoughtful commentary on the commentary, I recommend James Grimmelmann’s blog, The Laboratorium. Although there isn’t a single page that brings together all of his postings about the Google Book Search settlement, he helpfully prepends “GBS:” to the title of all such postings. If you are looking to participate in the discussion surrounding the settlement, the best place I know of is the interactive version of the settlement notice hosted at The Public Index. There you can comment and watch the comments of others on a section-by-section basis, along with a catalog of documents and links from others regarding the settlement.

The next big event after the filing deadline is the Final Fairness Hearing, scheduled for 10am on October 7, 2009 (or, at least, scheduled for that day and time at the moment). At the fairness hearing, we get to hear from the the court as it considers whether to grant final approval of the Settlement. Somehow, though, I don’t think even that will be close to the final word on the settlement. Stay tuned…

The text was modified to update a link from http://www.wo.ala.org/districtdispatch/wp-content/uploads/2009/09/supplementbrief-FINAL.pdf to http://wo.ala.org/gbs/ala-acrl-arl-brief-to-court/ on January 20th, 2011.

The text was modified to update a link from http://sla.org/ to http://www.sla.org/ on February 11th, 2011.

New York Judge Denny Chin recently issued two rulings in the Google Book Search settlement. In the first, he ‘ the request by the Internet Archive to intervene as a defendant in the lawsuit (and thus, presumably, be on firmer founding to guide aspects of the settlement). In his response, Judge Chin said:

The Court has received requests for pre-motion conferences by the Internet Archive, Lewis Hyde, Harry Lewis, and the Open Access Trust, Inc. seeking leave to intervene in this action. I have construed their letters as motions to intervene, and the motions are denied. The proposed interveners are, however, free to file objections to the proposed settlement or amicus briefs, either of which must be filed by the May 5, 2009 objection deadline.

(The Open Access Trust is a proposal to form a legal trust for the revenue generated by unclaimed orphan works.)

In the second, Judge Chin granted a four month extension to the deadline for class members to opt out of the settlement or file objections. The new deadline is now September 4th. Requests for the extension came from a group of authors (including heirs of Steinbeck) in an April 24th letter and a group of academic authors represented by Berkeley School of Law Professor Pamela Samuelson (who recently wrote an eloquent post about the settlement on the O’Reilly Radar blog). Attorneys in the case requested a two month extension.

In related news, the court has received and logged three letters that object to the settlement: from authors Hope Ryden and Lee Killough as well as Jenny Darling & Associates in Australia. More may come next week from those anticipating the May 5th deadline, but I expect most will continue to flow in around the new September 4th date.

Found via two articles and a tweet by Tim O’Reilly.

The text was modified to remove a link to http://www.publishersweekly.com/article/CA6654190.html on January 28th, 2011.

The text was modified to remove a link to http://www.publishersweekly.com/article/CA6654845.html on January 28th, 2011.

We are starting to see objections to the Google Book Search Settlement this month in advance of the May 5th deadline set up by the court. The first comes from the consumer advocacy group Consumer Watchdog (found by way of the American Libraries news feed). They have submitted a letter to the U.S. Justice Department asking the antitrust division to delay the settlement until the “‘most favored nation’ clause favoring Google is removed and the deal’s ‘orphan works’ provision is extended to cover all who might digitize books, not only Google.” The letter in PDF is available on the Consumer Watchdog website. The objections revolve around the provision that require the Books Rights Registry to give Google the same terms as anyone else who enters into agreements with the Registry (noting that more favorable terms might be required by a new party in order to compete with Google) as well as the fact that the copyright infringement protection for digitizing orphan works only extends to Google.

The American Libraries news piece also says: “ALA, in conjunction with the Association for Research Libraries and ALA’s Association of College and Research Libraries, plans to file an amicus brief with the court about the settlement.” We’ll likely see that before the May deadline.

Another objection came from the Open Content Alliance. On behalf of the OCA, Internet Archive filed an intervention request with the court seeking to become a party to the settlement. The document, located on the Scribd service (an odd choice, if you were to ask me), is asking to add itself as a defendant because the rights of internet content providers and of the public were not represented in the negotiations. In the paragraph in the middle of the letter, it says “All other persons, including Internet content providers such as the [Internet] Archive, would not be able to use orphan works broadly without being exposed to claims to infringement.” And further down: “Google has negotiated for itself certainty in its use of orphan works under the terms of the settlement through the mechanism of the [Books Rights Registry], whereas marketplace competitors are able to negotiate with the [Registry] only for commercial exploitation of those works with identified rightsholders.”

Stay tuned — I think this is going to get more interesting in the next few weeks.

Update

17-Apr-2009

The O’Reilly Radar blog has a post by Pamela Samuelson, professor at the University of California at Berkeley with a joint appointment in the School of Information and the School of Law, on this very topic. While not a formal filing with the court, she comments on the legal implications of the settlement in very clear language.

This column argues that the proposed settlement of this lawsuit is a privately negotiated compulsory license primarily designed to monetize millions of orphan works. It will benefit Google and certain authors and publishers, but it is questionable whether the authors of most books in the corpus (the ‘dead souls’ to which the title refers) would agree that the settling authors and publishers will truly represent their interests when setting terms for access to the Book Search corpus.

Earlier this week, Aaron Swartz of the Internet Archive announced the demonstration website of the Open Library project, a new kind of book catalog that brings together traditional publisher and library bibliographic data in an interface with the user-contributed paradigm of Wikipedia. Okay, I’ll pause for a moment while you parse that last sentence. Think you got it? Read — and watch — further.

Open Library has been mentioned a bit in the blogs this week, but not to the extent I thought was worthy of the magnitude of the project. So I recorded a screencast introduction (in Flash Video format below followed by a rough transcript) that looks at not only the browsing side of the system but also the record editing and record creation aspects of Open Library. As I say at the end of the recording, Open Library is one of those mind-bending, assumption-shattering projects that, at least for me, is challenging my thoughts about what library service could be and should be. Congratulations to the team at the Internet Archive, and I’m looking forward to future enhancements and directions for the project.

Rough transcript of the screen cast is below.

Introduction

Hello, and welcome to this screencast overview of the Open Library project. Open Library is an effort by the Internet Archive to create a comprehensive catalog of every book. As the project’s “about” page says, “Not every book on sale, or every important book, or even every book in English, but simply every book.” The about page goes on to describe the characteristics of Open Library project — that it is a project enabled by Internet technology because no physical space could hold it and that it aims to pull together records from publishers and libraries. It is also a project in the same vein as Wikipedia, meaning that any user can create and edit the records in the system.

In this overview, I’ll lead you through searching and browsing the Open Library’s demonstration website from the perspective of any modern library catalog interface. Then I’ll show you where it deviates from traditional library catalogs by exposing the underlying wiki nature of the database; we’ll examine the changes that users have made and we’ll even make a change ourselves. And finally I’ll show the process of creating entirely new records in the system. So let’s get started.

Searching

We’re looking at the home page of the Open Library project demonstration site. In the middle is a search box with a suggested search — “tom sawyer adventure”. That is a good suggestion so we’ll click on Go. Open Library returns a classic, relevance ranked list of matching records with some book covers along the left side and a faceted list of refinements along the right. So right away you can see that there are some authority control problems here in the author names — Twain comma Mark, Mark comma Twain, and Twain comma Mark with birth and death dates — and here in the language field. But I have high hopes that the developer team will find some intriguing ways to address these problems.

Back over here in the results area we have the various editions of Samuel Clemen’s “The Adventures of Tom Sawyer” — let’s pick the 1876 edition to see the full record display — there. We have the publisher, publication date and place, language, and a summary or review of sorts at the bottom. We also see signs of the availability of full text — over here in the options box there is a download from the Internet Archive link, a “Scan Sponsor” field here and a “View this book” graphic. This is one of the items scanned by the Open Content Alliance and made available by the Internet Archive through the Open Library project. A very nice interface for paging through the book. So one could imagine that the Open Library could become the primary vehicle by which Open Content Alliance materials are made available to the public.

So let’s go back here to the metadata page. Remember in the introduction that I said that the data was malleable in a wiki-like fashion. The Open Library developers created a system that allows for user-contributed updates (a la Wikipedia) to fielded data (like your classic bibliographic record). The two hints that the record is modifiable are this big edit button in the middle of the metadata and this more subtile “[history]” link near the top of the page. Let’s start with the history link to see what has been done to this record.

This page should look familiar to those who have worked with wikis before. It shows a listing of edits that were made to this record from most recent to the very first edit, who made the change (identified by IP addresses in this case because the people making the changes were not logged to an account at the time), an editor-supplied comment about what was done, and when the change was made. We can go back in time and see the page at a particular version through the links under the “When” column, or we can use the compare function to see the difference between two version. In the case of the changes between version 2 and version 3, we see that the editor added “Canada” as the place of publication. On this page you start to see the fielded nature of this wiki structure, but the best place to see it is look at the record edit screen itself.

These are all full-text fields on this page with no controlled vocabulary. You’ll note the absence of any MARC field names here, but as you scroll through you’ll see the evidence of MARC and AACR2 in the field labels. Down at the bottom is an edit summary to describe the changes made to the record, then save, preview and delete version buttons — all classic wiki functions.

Editing

Now, I’d like to show the full record editing process, but since I don’t have this Mark Twain book in hand, I’m going to bring up another record that I created yesterday — “Eric Meyer on CSS“. Before showing the editing process, let’s linger here a moment at the “options” box along the right side. Since this is a more modern book (as opposed to the Tom Sawyer book we saw first), there are additional options here for purchasing the book through these various vendors or borrowing the book through a very nice link into Open Worldcat and two web-based book trading sites.

But back to the metadata. There is one error and one omission in this record — perhaps this is a subtile demonstration of problems that creep in with user-generated content. First, the error, is that there is an extra digit in the ISBN-10 field, which is a big problem because the links in the options box use the ISBN as a linking field and at the time of this recording they don’t work. They will work in a moment, though. The second problem is that I forgot to put in the publication date. But hey, no problem, all I need to do is “Edit” this record.

So we are back to the edit screen, and I’m going to scroll down and fix the ISBN-10 field like so, then scroll down a little further and add the publication date. Then I’ll scroll all the way to the bottom and type in an edit summary — “Fixed the ISBN and added a publication date” — and hit save. We’re now back at the metadata display screen and the link to Open Worldcat now works. So, as an aside, one wonders what the folks in Dublin, Ohio, think about this. It is competition on the one hand since Worldcat is also aiming to be the most comprehensive catalog of books in the world. On the other hand, perhaps there is room for cooperation by somewhat getting vetted changes to Open Library records into the OCLC union catalog. Who knows?

Creating a New Record

Alright, back to current reality. Let’s add a record to Open Library, and in this case I’m going to use an ARL SPEC Kit that I wrote a number of years ago called “Library Patron Privacy”. First let’s run a search in Open Library to see if it is there, and no, it isn’t. The only way I’ve figured out how to enter a new item is to go to the URL where the page would be located and get the classic wiki “This page does not exist. Create it?” message.

One of the quirks I found in the system is that I have to create author wiki pages before book wiki pages — otherwise I’ll get a Python error message on the screen. I’ve reported this to the Open Library developers, but in the meantime just know authors need to be created before their books. Which is to say that authors have wiki pages in Open Library in addition to books. The structure of URLs to Open Library author pages is the letter “a” followed by a slash followed by the author’s last, first and middle names separated by underscore characters. So I’ll go to the URL of that form, then click on the “Create it” link.

Now here is one of the tricky parts of the existing interface. The page type starts as “type/page”, and as you can see it doesn’t have any of the fielded elements that we saw in previous examples. What you have do do is change the page type to “type/author” and then you get the fielded HTML form. So I’m going to go through here and fill in some of the parts. Then go down to the edit summary field and write a summary of this change, then click save. Now that Open Library knows who I am, let’s create the record for the book.

You’ve seen the structure of the URLs to book pages before — a “b” followed by a slash followed by the book title with spaces replaced by underscore characters. I’ll put that in the URL field and get the default page type. This needs to be changed to “type/edition” in order to get the bibliographic record fields. There. Now I’ll go through here and enter the data. When we get down to the author field we enter it in the same format that we used to create it — an “a” followed by a slash followed by the name with spaces replaced by underscores.

So we’ll just finish up here and come down to the edit summary field, put something in here, and hit save. This record is now in the system, and you can see the public display here along with the links on the right because I entered an ISBN. I haven’t quite figured out how to get a cover image into the system yet — I expect there is a file upload interface somewhere, but I haven’t found it.

Conclusions

So that’s all there is, and I don’t say that in a way to denigrate the work that has been done by the development team so far. As the URL and site banner indicate, it is a demonstration system — and a compelling demonstration it is. All sorts of questions immediately come to mind, of course — will there be a controlled vocabulary or authority control built into the system, can data be exported out of records — and, for that matter, can end-users bulk import data into the system, are there Web2.0 niceties like tagging and RSS feeds in the works, and so forth.

Even with all of those questions, Open Library is one of those mind-bending, assumption-shattering projects that, at least for me, is challenging my thoughts about what library service could be and should be. Congratulations to the team at the Internet Archive, and I’m looking forward to future enhancements and directions for the project.

The text was modified to remove a link to http://demo.openlibrary.org/b/adventures_of_Tom_Sawyer on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/b/adventures_of_Tom_Sawyer?m=history on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/b/adventures_of_Tom_Sawyer?b=3&a=2&m=diff on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/b/Eric_Meyer_on_CSS on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/a/Murray_Peter_E on January 19th, 2011.

The text was modified to remove a link to http://demo.openlibrary.org/b/Library_Patron_Privacy on January 19th, 2011.