Earlier today, the Digital Public Library of America (DPLA) Steering Committee put out a call for a “Beta Sprint” to bring to the surface “innovations that could play a part in the building of a digital public library.” From the announcement:

The Beta Sprint seeks, ideas, models, prototypes, technical tools, user interfaces, etc. – put forth as a written statement, a visual display, code, or a combination of forms – that demonstrate how the DPLA might index and provide access to a wide range of broadly distributed content. The Beta Sprint also encourages development of submissions that suggest alternative designs or that focus on particular parts of the system, rather than on the DPLA as a whole.

1. File a 400-word statement of interest with the DPLA by June 15th (which includes an acknowledgment of the intellectual property policy).
2. Make a final submission by September 1st.

Participants will be notified of the results of the review in late September or early October 2011 and will be invited to a DPLA meeting in October in Washington, DC.

In the video introducing the beta sprint, John Palfrey points to the March’s Concept Note and the description of research tracks on the DPLA wiki. Jennifer Howard on Wired Campus blog of The Chronicle of Higher Education has a blog post about the DPLA beta sprint with a few quotes from some of the principle people.

This is an interesting way to gather more input about what the DPLA will look like and interact with local libraries. It reminds me of a traditional “Request for Information” process but is much more wide open in its call for new concepts from anyone — not just established players. The breadth of possible submissions will make the task of selecting proposals to be presented to a wider audience a challenging process. Submissions can be anything from data models to wireframe user interface mockups to running code.

Anyone Interested in Working on Library-Friendly DRM?

Jerome McDonough of the Graduate School of Library & Information Science at the University of Illinois at Urbana-Champaign presented a paper this summer at the Balisage conference with the title Structural Metadata and the Social Limitation of Interoperability: A Sociotechnical View of XML and Digital Library Standards Development.¹ The title is very hard to penetrate, but the contents of the paper lay bare a theory for why we don’t have large, swirling pools of shared digital objects that cross institutional silo boundaries.

Jerome lays the issue out right away. Paraphrasing from several papers, he says:

Despite its success, however, XML has not lived up to many librarians’ expectations within one area, that of interoperability…. Digital library developers have expected that shared use of an XML standard for structuring of content and metadata (what is commonly called “structural metadata” within the digital library community) would ensure content interoperability and provide a clean division between content and higher level tools and services designed to work with standardized encodings of that content. In practice, this goal has proved extraordinarily elusive. Experiments conducted by participants in the Library of Congress National Digital Infrastructure for Preservation Program (NDIIPP) to test the exchange of digital objects between repositories failed even when participants were using the same XML-based encoding format and producing valid XML instances to exchange.

What Jerome is referring to is the ability to readily move objects from one repository to another. This would seem inherently doable on the surface — the offering repository and the receiving repository are both using XML and perhaps even the same “structural metadata standard” (METS, MPEG-21, etc.). These standards provide “a structural grammar for the encoding of complex digital objects” — the kind of thing needed to move these complex digital objects around various repositories. Jerome lays out two reasons why this doesn’t occur. First, there is a tremendous amount of flexibility in the metadata standards as a result of efforts to make each of the standards abstract enough to encode every conceivable structure. Document authors have choices in the depth of levels of description, labeling of object components, and arrangement of the object structure relative to creation of one or more interrelated descriptive metadata files. The second issue he identifies is the problem of “standards independence”, or the desire by the document standard author to have his/her metadata schema stand alone. Relying on other schemas may decrease the usefulness of the new standard to other organizations and environments.

The paper is a rich history of structural markup standards that have lead the profession to where it is today. He concludes with suggestions for the digital library community to move past this problem. One solution is to declare that our community is more concerned with using the flexibility inherent in the metadata standards for local needs than we are with sharing digital objects across silos. Counter to this is to refine standards to reduce the flexibility so as to increase the chances that the standards will promote interoperability. Jerome also offers the idea of promoting the activity of converting between various metadata formats to a more recognized and valued level. (He notes, for instance that the XSL stylesheets created by the Library of Congress that convert MODS into MARC/XML and back are omitted from the ‘Standards of the Library of Congress’ web page.)

Via Lorcan Dempsey.

Footnotes

1. McDonough, J. (2008). Structural Metadata and the Social Limitation of Interoperability: A Sociotechnical View of XML and Digital Library Standards Development. InBalisage: The Markup Conference Proceedings 2008. Montréal, Canada. Retrieved October 2, 2008, from http://balisage.net/Proceedings/html/2008/McDonough01/Balisage2008-McDonough01.html.

David Lowe, Preservation Librarian at the University of Connecticut, is coordinating a survey of JPEG2000 use for digital imagery. The survey asks questions about the use of the JPEG2000 file format (for archival purposes or for access systems), tools used (both JPEG2000 toolkits and software that embeds JPEG2000 toolkits), and considerations of mathematically lossless versus visually lossless compression settings.

This is his announcement:

I am writing to solicit your help with a survey of library-related digital project staff regarding the implementation of the JPEG 2000 standard for digital images (specifically still images and not motion). We estimate that this task will take approximately 15 minutes of your time. It is available now at: http://www.surveymonkey.com/s.aspx?sm=WXFAJwyRNZZilRWzrnum_2fw_3d_3d

The survey will remain active until October 31, 2008. Afterward, we will post the results via a report uploaded to our institutional repository, digitalcommons.uconn.edu.

Please note that in our report, personal information from the survey will not be revealed, and any comments used will remain unattributed unless the respondent prefers to be credited and indicates that desire in a separate email to me directly at david.lowe@uconn.edu.

Thank you for your help,

David Lowe
Preservation Librarian
UConn Libraries

I encourage you to take the survey as well.

On Monday, I had the honor and pleasure of speaking at the NISO workshop “Getting the Most Out of Your Institutional Repository” on the topic of The Third Wave of Library Information Stewardship. The presentation abstract was:

[Academic] Libraries are gearing up for the third wave of information under our stewardship. In the first wave, libraries purchased, made discoverable, and managed information from commercial sources in physical forms (e.g., paper-bound monographs, traditional serials, and microform archives). In the second wave, libraries licensed, made discoverable, and supported information from commercial sources in digital form (e.g., electronic journals, index/abstract databases, and image collections).

Libraries are now entering the third wave: selecting, publishing, and curating locally-produced digital content (institutional repositories, pre-print archives, and other locally unique collections). In this third wave, we need the skills and techniques of all of the previous stages, plus a need to learn a few new tricks. This presentation offers an overview of the selection, publication, and curation of locally-produced digital content. The speaker will also end with a glimpse of the fourth wave.

This DLTJ posting is a placeholder for a link to the anticipated recording of the presentation (I’ll update the page when the recording is available) and as a place for attendees to offer comments on the talk. I’ll create a separate posting for my impressions of the meeting and what I learned from the other presenters.

One correction I must make: I gave the conference organizers an older version of my biographical statement. It said that I was heading a project called the Digital Resource Commons that is bringing hosted repositories to OhioLINK members. That (insert one or more of of: mission, honor, duty, challenge, responsibility, dream, nightmare…) is now in the extraordinarily capable hands of John Davison on the OhioLINK staff. Content repositories at institutions (which, as I said in my talk, shouldn’t necessarily be equated to “institutional repositories” as we know them now) remains a key professional interest and activity, just not with the awesome (insert another word from the list above) of running the project. ¹

The text was modified to update a link from http://www.niso.org/news/events_workshops/ir07/agenda.html to http://www.niso.org/news/events/niso/past/ir07/agenda.html on January 19th, 2011.

Footnotes

1. See the “New Title, New Challenges” posting from earlier this year for more background on the job change.

As a part of work for an OhioLINK strategic task force, I have been exploring the creation and operation of regional/collaborative/shared digitization centers. This is a report of findings to date after an open call for information. The report is structured with questions to be explored when considering a regional digitization center followed by narratives from conversations with the Collaborative Digitization Program (formerly the Colorado Digitization Program), the Mountain West Digital Library, and the Ohio Historical Society. My thanks go out to Leigh Grinstead, Liz Bishoff, Karen Estlund, Angela O’Neal, and Phil Sager for their assistance.

I am still interested in talking with collaboratives about similar programs, both “on the record” and in private conversations. Please get in touch with me if you would like to chat.

Questions for Exploration

1. Who does the digital conversion? Is it staff at the hosting institution (where the equipment is located), or does the institution contributing the materials perform the scanning operation at the host institution?
2. Where and when is the description done? Description of the digitized item was universally done by the contributing institution, but the location (at the scanning center or at the contributing institution) and timing (before conversion, during conversion, or after conversion) vary. Answers to the first question — whether contributing staff members are performing the conversion — affect this question.
3. Is contribution of materials to the Ohio Digital Resource Commons required? If the regional scanning center is used to digitize materials (or in the case where the consortium is subsidizing a scanning service to perform the digital conversion), what conditions are put in place for contributing those materials to the central repository.
4. What is the cost structure for use of the regional scanning center? Options range from complete subsidy by consortium (most notably when the CDP funded the equipment at their regional scanning centers) to contributing institutions being charged at cost recovery rates by hosting institutions. There can be various cost structures applied, such as a consortial subsidy for equipment and training with labor supplied by contributing institution or contracted from hosting institution.
5. What training is offered? Topics for training range from optimal use of digitization equipment to digitization project planning to metadata creation standards. The training can be based on group instruction, one-on-one consultation with contributing institutions, or a combination of both.
6. Who will fill the role of “metadata editor”? The need for a collective expert in metadata creation was found in many of the projects. This person is typically charged with training contributing staff on the appropriate use of local metadata conventions, coaching individual staff on particular projects, and reviewing records that will become part of a consortial database.

Report from the CDP

The original climate that generated the interest in scanners was one where institutions did not want to outsource digitization because they didn’t want the materials leaving their direct control. The CDP, on the other hand, did not want institutions to buy cheap scanners that would result in lower-quality scans. The CDP purchased the scanners ($2,500 each, in 1999 dollars) plus desktop workstations and created a training program for the use of the equipment. The center provided the location for the equipment and hands-on assistance with using the equipment; it was incumbent on the staff at the institution with materials to be digitized to perform the scanning themselves. Before using the equipment, those performing the scanning had to attend a CDP “Introduction to Digital Imaging” training session on the proper use and techniques for obtaining high-quality images. Scanning projects that received CDP funding had priority over other users of the equipment, but there was little contention for the equipment at the centers. Descriptive metadata could be keyed at the time of the scan; CDP provided a web-based template (called “DC builder”) or staff could use their own system (an ILS, ContentDM, etc.). Contribution of the associated metadata to the CDP union catalog of metadata — Heritage West — was required for CDP-funded grant projects. Images were locally hosted.

In 2003, the CDP conducted an evaluation of the scan centers in the form of focus groups. They found two different responses. First, institutions on the western slopes of Colorado made much heavier use of their scan centers. These institutions tended to be smaller and/or economically disadvantaged, and the availability of the hardware and software to conduct the scanning operations was more critical. In front range libraries, where institutions tend to be better-funded, the centers were not as heavily used for projects and tended to be used for training and demonstration sites; these participants felt that the greatest value in the CDP came from the professional networking and training opportunities the locations, the grants that the CDP provided and the creation of the best practices and website that brought all this together.

The regional scan centers are not used today. The primary reason is the diffusion of experience within the community that was spurred by the early success of the centers. As funding cycles continued, institutions purchased their own equipment (generally replicating the equipment at the centers) so as avoid the need to transport materials and staff to a regional center for larger projects. The focus of grant funding within CDP also shifted from image-based collections to EAD and sound collections. There is still an “Introduction to Digital Imaging” workshop, but the focus is now on requirements for in-house equipment and/or what to seek from a vendor in an RFP. The imaging workshop is typically taught as part of a three-day workshop series with an “Introduction to Digital Project Management” (storage, preservation, handling socially sensitive materials, how to display them) before and “Introduction to Metadata” (focused on Dublin Core) after. The biggest issue facing the consortium now is oversized scanning; only the very largest university library would have this kind of equipment.

The Colorado model of teaching staff at the institution on the use of the equipment along with the creation of regional scan centers was picked up by several states: North Carolina; Alabama; Kansas; Missouri; and Wyoming (except that regional centers were not practical due to the wide population dispersion). Tennessee is working under a current IMLS grant to build three regional center plus a suite of mobile equipment.

Liz suggests that we should look for ways to support collaborative efforts within the institution with museum and archives on the campus. The Florida Center for Library Automation (FCLA) has a program in place where the local university library is the contact in partnerships with public libraries and local museums.

Leigh noted that in recent years, CDP projects include a “metadata editor” that is hired to look at record quality and work with individual institutions to improve records. Having a metadata editor, someone familiar with the CDP guidelines and the field in general, look at four or five records at the very early stage of a project is critical to the success of the quality of the rest of the project. In a recent project, nearly half of the partners were going to export records out of a local collection management system. It was discovered that the records were not consistent; local implementation/practice of cataloging standards has a higher overall impact than community best practice. Having a local cataloger participate on the team migrating the records from a local system to a central system is key to success.

Mountain West Digital Library

Content is ingested into the MWDL either through digital conversion centers at the regional institutions or through the efforts of the contributing institution. The regional institutions perform digital conversion for contributing institutions at cost-recovery rates. These regional centers use the equipment and staff at the hosting institution, and are very busy at times resulting in difficult choices to balance needs of the host institution with that of requests from other institutions.

The regional centers also conduct on-site training and technical education at contributing institutions about best practice for digital conversion. The on-site program includes a technical evaluation of the equipment to be used to ensure that it can produce conversions that meet the minimum requirements for the MWDL. In the course of this evaluation, staff at contributing institutions are taught some technical aspects of scanning such as the actual DPI scanning capabilities of the hardware versus interpolated resolutions (and why this is important). Staff also receive an introduction to Photoshop for de-skewing and other standard practices. They also are instructed in the Western Standard Metadata Best Practices. This hands-on approach — with the contributing institution’s people, equipment and materials — enables strong connections between the contributing institution and the hosting institution. It makes the contributing institution feel like a part of the larger program. Most contributing institutions with local digital conversion operations will FTP files to the ContentDM server at the regional institution; some institutions contribute materials through the transportation of a portable hard drive.

Each regional hosting center is responsible for the digital preservation of the material on their server. At this point there is no common agreement across the MWDL for standards on digital preservation; this is an area of work to be addressed in the near future by the cooperative.

The cooperative is starting work in several new areas. First is the digital conversion and posting of oral histories from contributing institutions. While it is anticipated that such resource will be highly valued, quick progress on this project is hampered by the same permission problems that face similar projects with relatively old audio. The cooperative also has a state LSTA grant for a Utah institutional repository with a portal connected to the MWDL. The state archives are also beginning a program to post state government documents, including items related to the Olympic Games held in Utah. The hands-on approach to training staff at contributing institutions is very labor-intensive and uneven across the project participants. MWDL is in the process of hiring a program director, and a component of that job description is to provide this kind of training across the project.

Ohio Memory Project (Ohio Historical Society)

OHS staff will work with contributing institutions on a one-on-one basis as well as conduct workshops on introduction to scanning. Many of the participants are from contributing institutions that have had staff changes and the new staff want to know how to use the equipment purchased during the grant-funded phase of the project. There is also a desire for more advanced training, such as Photoshop basics.

OHS staff recognize that a compromise is needed between institutional capabilities for scanning and the very best practices in the field. They have found it hard to establish and mandate an absolute standard for the parameters and quality of digital scans. Institutions that scan their own materials are responsible for their own storage and preservation.

The Technical Committee and Liaison Group of the OAI Object Reuse and Exchange effort met last week in New York City to hammer out face-to-face some of the last remaining issues before work could begin on a draft specifications document. In preparation for that meeting, we worked on a white paper that is officially called “Compound Information Objects: The OAI-ORE Perspective” — but it might more accurately be called “Thoughts on Compound Documents”. The outline of the white paper looks like this:

1. Introduction and Motivation
2. An OAI-ORE Interoperability Layer for Compound Objects
3. Exposing Logical Boundaries in the Web Graph
4. Named Graph Publishing
5. Named Graph Authorship
6. Generalizing Named Graph Publishing
7. Named Graphs for Compound Objects: Issues for Exploration
   1. Connectedness and the Containment Node
   2. Boundary
   3. Identification
   4. Referencing
   5. Versioning
   6. Publishing Named Graphs versus Publishing RDF Documents
   7. Discovery of Named Graphs
8. Conclusion

Sections 1-6 set up our collective thoughts on compound documents in which we borrow the concept of “named graphs” from the semantic web community. One of the things learned over our two-day face-to-face meeting was that “Named Graphs” in the semantic web community means a lot more than the “named graphs” term we use in this document, so a) don’t read too much into the use of our term “Named Graph”; and b) the term is likely to change as the specifications are drafted to avoid confusion with the semantic web community. Section 7 sets up the issues that were identified in conference calls and e-mail with the scheme outlined in the previous sections. In the face-to-face meeting we came to some conclusion on all of them except Versioning, and I’ll try to give insight as to the decisions made last week in a future post.

We (the members of the Technical Committee and the Liaison Group) felt comfortable enough with the document to publish it on the open web and solicit comments. Feel free to leave comments here or contact me directly; all comments will be forwarded back into the Technical Committee.

OhioLINK is engaged in building a “trusted digital repository” on behalf of its membership. As we build it, we want to have an understanding of what “trusted” means, and so we are engaging in an audit process to assess whether we can claim to be trustworthy. This process is panning out to have four major phases:

1. Research common and best practices for preservation.
2. Evaluate the OhioLINK policies and processes against common and best practices.
3. Perform a gap analysis between where we are now and where common and best practices suggest we should be.
4. Propose and adopt policies and processes that get us closer to the ideal common and best practices.

This is a report at the end of phase 1. Earlier this year, two major reports were released that address how one measures a “trustworthy repository.” The two reports are summarized below, followed by a recommendation.

Trustworthy Repositories Audit & Certification: Criteria and Checklist

The nestor working group says a trusted, “long-term digital repository is a complex and interrelated system” (nestor 2006). However, more than just the “digital preservation system” drives the management of the digital materials. In determining trustworthiness, one must look at the entire system in which the digital information is managed, including the organization running the repository: its governance; organizational structure and staffing; policies and procedures; financial fitness and sustainability; the contracts, licenses, and liabilities under which it must operate; and trusted inheritors of data, as applicable. Additionally, the digital object management practices, technological infrastructure, and data security in place must be reasonable and adequate to fulfill the mission and commitments of the repository.

A trusted digital repository will understand threats to and risks within its systems. As articulated by Rosenthal et al. (2005), these potential threats include media failure, hardware failure, software failure, communication errors, failure of network services, media and hardware obsolescence, software obsolescence, operator error, natural disaster, external attack, internal attack, economic failure, and organizational failure. Constant monitoring, planning, and maintenance, as well as conscious actions and strategy implementation will be required of repositories to carry out their mission of digital preservation. All of these present an expensive, complex undertaking that depositors, stakeholders, funders, the designated community(ies), and other digital repositories will need to rely on in the greater collaborative digital preservation environment that is required to preserve the vast amounts of digital information generated now and into the future.

The TRAC:CC contains 84 criteria broken out into three main sections: Organizational infrastructure; Digital object management; and Technologies, technical infrastructure, and security. Within each of these sections are various subsections and under the subsections are the criteria themselves.

1. Organizational infrastructure
   1. Governance & organizational viability
   2. Organizational structure & staffing
   3. Procedural accountability & policy framework
   4. Financial sustainability
   5. Contracts, licenses, & liabilities
2. Digital object management
   1. Ingest: acquisition of content
   2. Ingest: creation of the archivable package
   3. Preservation planning
   4. Archival storage & preservation/maintenance of AIPs
   5. Information management
   6. Access management
3. Technologies, technical infrastructure, and security
   1. System infrastructure
   2. Appropriate technologies
   3. Security

Some sample criteria are:

• A1.1 Repository has a mission statement that reflects a commitment to the long-term retention of, management of, and access to digital information.
• A2.2 Repository has the appropriate number of staff to support all functions and services.
• A3.5 Repository has policies and procedures to ensure that feedback from producers and users is sought and addressed over time.
• A5.4 Repository tracks and manages intellectual property rights and restrictions on use of repository content as required by deposit agreement, contract, or license.
• B2.5 Repository has and uses a naming convention that generates visible, persistent, unique identifiers for all archived objects (i.e., AIPs).
• B2.9 Repository acquires preservation metadata (i.e., PDI) for its associated Content Information.
• B3.4 Repository can provide evidence of the effectiveness of its preservation planning.
• B4.4 Repository actively monitors integrity of archival objects (i.e., AIPs).
• B5.3 Repository can demonstrate that referential integrity is created between all archived objects (i.e., AIPs) and associated descriptive information.
• C1.1 Repository functions on well-supported operating systems and other core
• infrastructural software.
• C1.5 Repository has effective mechanisms to detect bit corruption or loss.
• C1.7 Repository has defined processes for storage media and/or hardware change (e.g., refreshing, migration).
• C1.9 Repository has a process for testing the effect of critical changes to the system.
• C3.3 Repository staff have delineated roles, responsibilities, and authorizations related to implementing changes within the system.

TRAC:CC states that the “criteria are written to be applicable to any kind of digital repository or archives.” As such, criteria should be placed within the context of vision and goals of the Ohio DRC. One demonstrates compliance with the criteria through documentation (evidence), transparency (open examination of the evidence), adequacy (degree to which the evidence meets the vision/goals), and measurability.

Digital Repository Audit Method Based On Risk Assessment

The process has six stages, some with multiple tasks:

1. Identify organizational context
   • Specify mandate of your repository or the organization in which it is embedded
   • List goals and objectives of your repository
2. Document policy and regulatory framework
   • List your repository’s strategic planning documents
   • List the legal, regulatory, and contractual frameworks or agreements to which your repository is subject
   • List the voluntary codes to which your repository has agreed to adhere
   • List any other documents and principles with which your repository complies
3. Identify activities, assets and their owners
   • Identify your repository’s activities, assets and their owners
4. Identify risks
   • Identify risks associated with activities and assets of your repository
5. Assess risks
   • Assess the identified risks
6. Manage risks
   • Manage the risks

The report includes a catalog of risks taken from other checklists and repository audits that can be used to spur the thinking of the auditor.

Recommendation

I’m looking for information about the formation and management of regional digitization centers for one of the OhioLINK strategic task forces. For our purposes, a “regional digitization center” is a place that has the hardware, software, and human expertise to convert a variety of media to digital form. (We’re primarily looking at small format imaging, but could also include broadside imaging, audio capture, and video transformation.) There is plenty of information to be found about the services that centers provide and even more evidence of regional groups wanting to create these centers, but precious little about the operation of the centers themselves. (As in zilch in professional literature searches, and only a few hits via general web searching.) The kinds of things I’m looking for are:

• Operational structures, ranging from staff at the center that do the actual digitization to centers that just provide equipment and the organization wanting the digitized materials providing the staffing.
• Cost structures for initial center development and ongoing support.
• Project plans for building and promoting regional digitization centers.

Here’s what I’ve found so far. There is a really good message from Liz Bishoff from the time she was the Executive Director of the Colorado Digitization Program. There is also this bit from an article Liz Bishoff wrote for First Monday:

To assure that the institutions had access to equipment that supported these standards, the CDP established five regional scan centers. These centers provide the Colorado institutions with relatively easy access to scanning equipment, assistance by trained staff in scanning, and access to the union catalog and local databases via the Web. Each institution has to do their own scanning. Training sessions on scanning and metadata are being conducted throughout the spring and summer, 2000 at these regional scan centers. It is hoped that the combination of consulting on scanning, training, and quality equipment will result in a consistent quality image, as well as developing expertise at the local institution level.¹

The same themes are repeated in an interview from the NINCH Guide to Good Practice. Beyond the CDP project, though, the findable information drops off quickly. I’ve found that the Making of Modern Michigan project has spawned quite a number of regional centers, as have other efforts. I’m going to get in touch with folks in Colorado and Michigan, but in the meantime if you know of something I’ve missed (or you run one of these kinds of centers yourself) please let me know.

The text was modified to remove a link to http://lists.mdch.org/public/digistates/2003-September/000020.html on January 20th, 2011.

Footnotes

1. Bishoff, L. (2000). Interoperability and standards in a museum/library collaborative. First Monday, 5(6). Retrieved May 18, 2007, from http://www.firstmonday.org/issues/issue5_6/bishoff/.

“Participatory Digital Libraries” is the name of a talk Paul Jones, Director of ibiblio.org, gave this morning at OCLC’s Kilgour Auditorium. Known as “The Public’s Library,” ibiblio is a large, diverse digital library. His talk offered insight on how ibiblio works and commentary for applying the same successful techniques in library projects. This is a summary of the key points of his talk; errors of transcription and omission are undoubtedly my own.

Digital Archives in Action

In one part of the presentation, Paul describes the array of curatorial practices used for ibiblio collections. First, however, he distinguished his activity from that of tradition archives. “Archives,” in his words, “look for interesting collections; we look for interesting people creating interesting collections in interesting ways.” Put another way, ibiblio experimented with getting to the front end of the content creation process by finding people as they were creating the collections.

They collected the collectors and empower them with tools to build the content. As a result of seeding many such experiments, ibiblio has many different, idiosyncratic ways of building collections.

• folkstreams.net — This is an IMLS-funded project that is a “rescue mission” for documentary films created in the 1960s. The collection is curated by well-respected film makers.
• Roger McGuinn’s Folk Den — A single individual that has been recording a folk song a week for 12 years. The contents of the collection? “What ever interests Roger McGuinn.”
• Documenting the American South — After residing for 10 years on ibiblio, this collection is now being migrated to a UNC Libraries server. Curated by professional archivists, it has lots of money invested in best-practice output for digitization, description, and standards-based (e.g., TEI Lite) artifacts.
• Moonshine — A project created by students creating on online exhibition to materials in the university archives.
• Degree Confluence — A participatory infrastructure that allows people to contribute pictures and diary entries from points on the earth’s surface where integer values of latitude and longitude intersect.

Paul also mentioned a bit about blogs. His observation is that researchers are not putting content into Institutional Repositories, yet they are putting content into blogs, photo sharing sites, and the like. Many of these content systems that researchers are using have RSS capabilities, and the metadata in an RSS feed is at times better than what we ask users to provide in an IR. Perhaps our “IR” capabilities could be about RSS harvesting this content and preserving it.

Five ‘Big Ideas’

In a subsequent paper published in the Communications of the ACM, Paul described the ibiblio effort this way: “By adopting not only the open-source tools, but also the open-source philosophy encouraging community interaction and contributor involvement, digital libraries can open new horizons to new communities as well as greatly improve traditional services.” He touched on five ‘big ideas’ and how they impact the digital library arena.

1. The Long Tail. ibiblio made many little bets by seeding many collectors, as opposed to focusing on blockbusters (as say what the Smithsonian or California Digital Library does).
2. Democratizing Innovation. No one is ultimately happy with how you present things to them. Enabling users to modify things creates happier and more satisfied users, and better products result from looking at what users do.
3. Wealth of Networks. A community working together to remix and create knowledge increases the knowledge capital in a system. This book also discusses the value of market and non-market interactions.
4. Free Culture. Discusses the social utility of information wanting to be free. Who owns the cultural heritage and who preserves it and why these are important questions.
5. Small is the New Big. He touched on this book briefly, but my notes and memory fail me at this point.

(See Paul’s presentation page for demonstrations of these ideas in action — each of these books has a free online version and/or a remix and/or a blog that discusses the ideas presented both before and after formal publication.)

Paul also mentions one that he hopes spend some time working on: Laws of Simplicity. We have simple ideas (e.g. the web’s core HTML and HTTP standards), but not simple tools for our users.

Experts And (not ‘Versus’) Passionate Amateurs

This heading is my summary of a portion of his talk, and the theme struck a chord with me. How does the role of experts intersect with the role of people just trying to get their jobs done? He states quite emphatically that there is a middle ground between formal description (‘cataloging’) and user-driven description (‘tagging’). How do we build a community of professionals and enthusiasts who fill the space between content objects with new knowledge? This is already starting to play out in other arenas — how will “Encyclopedia of Life” and “Wikipedia” interact? In one “material will…be authenticated by scientists”¹ and in the other “thousands of people have contributed information to different parts of this project, and anyone can do so”².

The text was modified to remove a link to http://www.alaoweb.org/cgi-bin/dada/mail.cgi/archive/announce/20070501093221/ on January 20th, 2011.

The text was modified to update a link from http://www.hyperionbooks.com/titlepage.asp?ISBN=1401302378 to http://www.hyperionbooks.com/book/the-long-tail-revised-and-updated-editionwhy-the-future-of-business-is-selling-less-of-more/ on January 20th, 2011.

The text was modified to remove a link to http://www.free-culture.cc/ on January 20th, 2011.

Footnotes

1. http://www.eol.org/faqs.html#faq3.2
2. http://en.wikipedia.org/wiki/Wikipedia:Overview_FAQ#Who_is_responsible_for_the_articles_on_Wikipedia.3F

Here is a pair of papers that I’d like a chance to digest at some point. The first is “Recommending Related Papers Based on Digital Library Access Records” by Stefan Pohl, Filip Radlinski, and Thorsten Joachims. According to the notes on the paper, it is to appear in proceedings of JCDL’07. The abstract:

An important goal for digital libraries is to enable researchers to more easily explore related work. While citation data is often used as an indicator of relatedness, in this paper we demonstrate that digital access records (e.g. http-server logs) can be used as indicators as well. In particular, we show that measures based on co-access provide better coverage than co-citation, that they are available much sooner, and that they are more accurate for recent papers.

This is a two-page summary, with the meatier version being “Using Access Data for Paper Recommendations on ArXiv.org”, a masters thesis written by Stefan Pohl. This one is about 70 pages. The abstract is:

This thesis investigates in the use of access log data as a source of information for identifying related scientific papers. This is done for arXiv.org, the authority for publication of e-prints in several fields of physics.

Compared to citation information, access logs have the advantage of being immediately available, without manual or automatic extraction of the citation graph. Because of that, a main focus is on the question, how far user behavior can serve as a replacement for explicit meta-data, which potentially might be expensive or completely unavailable. Therefore, we compare access, content, and citation-based measures of relatedness on different recommendation tasks. As a final result, an online recommendation system has been built that can help scientists to find further relevant literature, without having to search for them actively.

Stefan’s work would seem to bring the old adage “Do as I Do (access), Not as I Say (cite)” to bear on information retrieval. More fluid and dynamic than using PageRank — or web citation — “co-access” would seem to allow the wisdom of the crowds to become more apparent.