NELLCO's Universal Search Solution Project

Boundaries are being blurred between the academic and commercial Web, between library resources, between the citation and the item itself. Students have no patience with these arbitrary boundaries; they want information, and they want it now, wherever it may be located. ((Cox, Christopher. An Analysis of the Impact of Federated Search Products on Library Instruction Using the ACRL Standards. portal: Libraries and the Academy. 6(3), July 2006, pp. 253-267.))

Earlier this year, the New England Law Library Consortium (NELLCO) announced that they had received a grant from the Institute of Museum and Library Services to build a "Universal Search Solution" -- a 'one-box' search into a unified index of a range of electronic resources. Indexed databases include OPACs, subscription-based resources, and selected free web resources. It is a two year grant to build and implement the tool for NELLCO members and release the code into open source. Index Data will be contracted to build the tool.

The tool "will be based on open standards and open source software, and will result in the creation of a ... master index of material, including participating library catalogs, as well as subscription-based databases and open content, special collections, and other resources that a participating library wishes to make discoverable to its patrons." ((From the NELLCO press release.)) The project specifications include the concept of differentiating search results based on the user's institutional affiliation -- search results from the user's OPAC and hits from commercial data sources in the unified index that are limited by license to just that user's institution. "Technologically, the Universal Search Solution will combine multiple technologies (consolidated indexing and data storage; metadata harvesting; and metasearching) to put together a single window to disparate resources." ((From the Index Data press release.))

I spoke with Tracy Thompson earlier this week about the project. The concept is an out-growth of an earlier pilot to use a Google Search Appliance (GSA) to create the unified index. The pilot was technologically successful, but ran into problems in the business model for the GSA because the pricing was based on the number of "documents" and from the GSA perspective each record indexed in a database was a document. (To give a perspective, the GSA page says the base model "starts at $30,000 to search up to 500,000 documents" ((This figure is on the GSA page at the time this posting is being published, but it is also on the 8-Aug-2007 copy of the page in the Internet Archive.)) -- and how many bibliographic records are in our OPACs, just as a baseline for costing out a GSA?)

Current Status

Tracy said the IMLS grant period starts on December 1st, and they are in the process of hiring a project coordinator (the search has not yet been announced). There will be a kick-off meeting for NELLCO participants in early December.

In broad strokes, the first year is geared mainly towards building the tool. In the second year, the IMLS grant will fund the implementation of the tool for interested NELLCO members. After the grant period, NELLCO expects they will be able to sustain the implementation by subscription fee from the members. As described earlier, this is a consortial product/service, so there are already some economies of scale to be gained by not implementing redundant hardware/support at many local institutions.

A Different Model for Open Source Development

The Index Data press release goes on to say:

The grant also offers a new model for libraries to obtain affordable software services that are under their control. It combines the powerful financial leverage of IMLS and the organizational capabilities of NELLCO and its membership with the software development expertise of Index Data to bring cutting edge open source software and services to all types and sizes of libraries -- affordable and with commercial support -- but without vendor lock-in.

The tension between the inherent values of commercial, closed-source software vendors and higher education institutions (with an acknowledging nod towards commercial support for open source solutions) has been discussed on DLTJ before. The IMLS/NELLCO/Index Data collaboration suggests a new way for software tools to be built: a granting agency funds the initial hurdle of software development, and then the project transitions to self-supporting either through subscription charges that are in the self-interest of participants or through the sweat equity of participant-developers.

The text was modified to update a link from http://www.nellco.org/IMLS/Press%20Release.pdf to http://www.nellco.org/index.cfm?fuseaction=Feature.showFeature&featureid=21&pageid=4 on January 20th, 2011.

The text was modified to remove a link to http://www.uwec.edu/library/aboutUs/who/cox.htm on January 20th, 2011.

The text was modified to update a link from http://www.nellco.org/index.cfm?page=contact to http://www.nellco.org/general/?type=CONTACT on November 13th, 2012.

The text was modified to remove a link to http://www.imls.gov/results.asp?program=-1&inst=+New+England+Law+Library+Consortium&city=&State=0&year=12&keyword=&description=on&sort=year&imageField.x=0&imageField.y=0 on November 13th, 2012.

The text was modified to update a link from http://www.nellco.org/index.cfm?fuseaction=Feature.showFeature&featureid=21&pageid=4 to http://www.librarytechnology.org/ltg-displaytext.pl?RC=13319 on November 21st, 2012.