It has been years since I’ve done meaningful work with JPEG2000, but I still try to keep tabs on what is happening in that community. In that vein, Rob Buckley — formerly of Xerox Research and now on his own with a consulting business — pointed me to an announcement about a JPEG2000 Summit happening at the Library of Congress next month.

The plan for the meeting is:

The program starts with a half-day tutorial on JPEG 2000 on the morning of the first day, followed by two sessions of presentations. The program will conclude on the afternoon of the second day with a small invitation-only working session to identify key objectives and collaborations for broadening the understanding and use of the technology.

Registration is free but required because of the limited space. If I were in or near D.C., I’d surely be there but my day job doesn’t align well enough to the agenda to justify the trip. I continue to think that use of JPEG2000 is a significant enough upgrade to TIFF practices for archiving and presentation to justify the cost of making a transition. (That statement says nothing about a wholesale conversion of archive TIFFs to JPEG2000s; existing TIFFs can stay as they are but current practice should take advantage of the features offered with JPEG2000.) The organizers are ending the summit with “a small invitation-only working session to identify key objectives and collaborations for broadening the understanding and use of the technology.” I’m looking forward to the output of that working group.

With the close of the year approaching, this issue marks the 14th week of DLTJ Thursday Threads. This issue has a publisher’s view of Amazon’s strong-arm tactics in book pricing, research into the possibility that academic authors could game Google Scholar with spam, demonstrations of how Amazon’s Mechanical Turk drives down the cost of enlisting humans to overwhelm anti-spam systems, and a story of multispectral imaging adding information in the process of digital preservation.

As the new year approaches, I wish you the best professionally and personally.

Books After Amazon

What happens when an industry concerned with the production of culture is beholden to a company with the sole goal of underselling competitors? Amazon is indisputably the king of books, but the issue remains, as Charlie Winton, CEO of the independent publisher Counterpoint Press puts it, “what kind of king they’re going to be.” A vital publishing industry must be able take chances with new authors and with books that don’t have obvious mass-market appeal. When mega-retailers have all the power in the industry, consumers benefit from low prices, but the effect on the future of literature—on what books can be published successfully—is far more in doubt.

Onnesha Roychoudhuri publishes this view of Amazon’s marketing practices in the lastest issue of the Boston Review. From the publisher’s pespective, the strong-arm tactics described sound horrible. But the story also points to cracks appearing — at least for the bigger publishers. That may leave smaller, independent publishers in a big squeeze. [Via OCLC Research's Above-the-Fold]

Academic Search Engine Spam and Google Scholar’s Resilience Against it

Abstract: In a previous paper we provided guidelines for scholars on optimizing research articles for academic search engines such as Google Scholar. Feedback in the academic community to these guidelines was diverse. Some were concerned researchers could use our guidelines to manipulate rankings of scientific articles and promote what we call ‘academic search engine spam’. To find out whether these concerns are justified, we conducted several tests on Google Scholar. The results show that academic search engine spam is indeed—and with little effort—possible: We increased rankings of academic articles on Google Scholar by manipulating their citation counts; Google Scholar indexed invisible text we added to some articles, making papers appear for keyword searches the articles were not relevant for; Google Scholar indexed some nonsensical articles we randomly created with the paper generator SciGen; and Google Scholar linked to manipulated versions of research papers that contained a Viagra advertisement. At the end of this paper, we discuss whether academic search engine spam could become a serious threat to Web-based academic search engines.

Joeran Beel and Bela Gipp have this article in the most recent issue of Journal of Electronic Publishing. In addition to being able to game Google Scholar, the authors note that Microsoft Academic Search and CiteSeer (as well as their own academic search engine currently under development — SciPlore) have the same issues. Although it is possible, we don’t know if it is being done — or even if there would be an penalties in the academic community for doing so.

Mechanical Turk: Now with 40.92% spam

At this point, Amazon Mechanical Turk has reached the mainstream. Pretty much everyone knows about the concept. Post small tasks online, pay people cents, and get thousands of micro-tasks completed. Unfortunately, this resulted in some unfortunate trends. Anyone who frequents just a little bit the market will notice the tremendous number of spammy HITs. (HIT = a task posted for completion in the market; stands for Human Intelligence Task). “Test if the ads in my website work”. “Create a Twitter account and follow me”. “Like my YouTube video”. “Download this app”. “Write a positive review on Yelp”. A seemingly endless amount of spam HITs come to the market, mainly with the purpose of spamming “social media” metrics. So, with Dahn Tamir and Priya Kanth (MS student at NYU), we decided to examine how big is the problem. How many spammers join the market? How many spam HITs are there?

This post from Panos Ipeirotis, Associate Professor at the IOMS Department at Stern School of Business of New York University, describes a review of activities posted to Amazon’s Mechanical Turk service. Spam is everywhere, and it appears that the Mechanical Turk is reducing the friction between buyers and workers of spam activity. [Via Ron Murray]

Cutting-Edge Imaging Helps Scholar Reveal 8th-Century Manuscript

With a manuscript like the St. Chad Gospels, multispectral imaging—a series of scans, each based on a single part of the color spectrum—allows his team to create images that have the equivalent of three-dimensional detail, down to revealing the thickness of brush strokes on letters and illustrations. Cockled pages can be virtually flattened out so that all their details can be studied. Studied color band by color band, the chemical composition of ink can be determined.

This article by Jennifer Howard at the Chrnoicle of Higher Education reviews the story of how 8th-century documents in England were digitized by scholars at the University of Kentucky. It caught my eye because of the mention of multispectral imaging; this is something that the JPEG2000 file format can natively store. Digitization at this level doesn’t just provide alternative, online access to documents — it actually adds new information to the process of researching those documents. [Note: the link is behind a publisher paywall. If you would like to see it, send me an e-mail and I'll forward you a short-term link from the Chronicle's website.]

Earlier this month, the JPEG 2000 Implementation Working Group, the Wellcome Trust Library, and the U.K. Digital Preservation Coalition hosted a free one-day seminar called JPEG2000 for the Practitioner. The presentation slides are now linked to the seminar program and is a short report of the event by Christy Henshaw of Wellcome Library. The presentation slides by themselves carry a great deal of depth even without a recording of the audio. In particular I can recommend “What did JPEG 2000 ever do for us?” by Simon Tanner and “JPEG 2000 standardization – a pragmatic viewpoint” by Richard Clark. As brief introductions to where we’ve been with JPEG 2000 and where we could go.

Hat tip to Ron Murray for pointing this out to me.

Week #2 of this new project to highlight interesting tidbits from the previous seven days. Well, things that were interesting to me that I hope will be interesting to DLTJ readers. Time will tell.

Technical Debt: A Perspective for Managers

What is Technical Debt? It’s all “those internal things that you choose not to do now, but which will impede future development if left undone” [Ward Cunningham]. On the surface the application looks to be of high quality and in good condition, but these problems are hidden underneath. QA may even tell you that the application has quality and few defects, but there is still debt. If this debt isn’t managed and reduced, the cost of writing/maintaining the code will eventually outweigh its value to customers.

Technical Debt is like a credit card that charges a high interest rate, just leaving the team with an outstanding balance cost. In this case, the costs are represented by time and effort needed to work around the problems. The longer the team takes to pay off the debt, the more interest is accumulated (in the form of additional workarounds) and the higher the costs for the business.

This definition of the amorphous stuff that gets in the way of moving faster really resonates with me.

A Case of Taking QR Codes to the Park

[Fort Smith Park Superintendent Bill Black] sat through a few conference sessions held by the Arkansas Parks and Tourism Department about information technology, where he heard about QR (or Quick Response) codes—which are two-dimensional bar codes that can be used in a variety of ways. A company can choose from any number of sites that will generate a QR code for free and put that code almost anywhere—on a website, a postcard, or even a T-shirt. Then smartphone users use the camera on their phones to scan the bar code—some phones have the scanning technology built in, but older iPhones and the like will have to download a free app—and are instantly taken to whatever content is linked to the bar code.

“On the drive home I got thinking about how it might work for interpretation purposes,” Black says, and he began to consider how this technology might be deployed to provide information to park visitors.

Econtent Magazine has this brief use case for QR Codes as a way to link to more information in a national park. Usage of QR Codes seem to be creeping up, helped in no small part by efforts at Google in its Favorite Places and URL Shortner services. They aren’t exactly common yet, but this is a place where libraries might get ahead of the game. There have been several experiments with QR Codes in OPACs and other services, for instance, and some great thinking about how they could be used. Is there an education role for libraries in helping patrons use this new technique for connecting to information?

WebP, a new image format for the Web

Most of the common image formats on the web today were established over a decade ago and are based on technology from around that time. Some engineers at Google decided to figure out if there was a way to further compress lossy images like JPEG to make them load faster, while still preserving quality and resolution. As part of this effort, we are releasing a developer preview of a new image format, WebP, that promises to significantly reduce the byte size of photos on the web, allowing web sites to load faster than before.

On the heels of the mention here last week of the bounty to add JPEG2000 support to Firefox comes this announcement from Google of a new image format for websites that is supposedly better than JPEG2000. Lots of buzz around this, but not much in the way of commitment to support it yet. I suppose the real test will be whether WebP will be supported in Firefox before JPEG2000…

General Counsel’s Role in Shoring Up Authentication Practices Used in Secure Communications

The major Internet browsers all currently use the Certificate Authority Trust Model to verify the identity of websites on behalf of end-users. (The Model involves third parties known as certificate authorities or “CAs” issuing digital certificates to browswers and website operators that enable the end-user’s computer to cryptographically prove that the same CA that issued a certificate to the browser also issued a certificate to the website). The CA Trust Model has recently come under fire by the information security community because of technical and institutional defects. Steve Schultze and Ed Felten, in previous posts here, have outlined the Model’s shortcomings and examined potential fixes. The vulernabilities are a big deal because of the potential for man-in-the-middle wiretap exploits as well as imposter website scams.

Is ‘https’ and ‘SSL’ as secure as you believe it is? These researchers point out that it is only as good as your trust in the Certificate Authorities to issue SSL certificates to the appropriate web site owners and to keep safe the secrets necessary to make ‘https’ work. Read this so that you have an informed sense of how secure your communications on the web actually are.

M.I.T. Weighs Charges for Online Lectures

The Massachusetts Institute of Technology has announced that it is considering charging for access to online lectures and class notes, which are currently available free on the Web. Speaking at the Organization for Economic Cooperation and Development’s Institutional Management in Higher Education conference in Paris this month, Lori Breslow, director of M.I.T.’s Teaching and Learning Laboratory, said that free access “may not be the best economic model, so we are now looking seriously at new e-learning opportunities.”

I only saw this brief mention of this in the New York Times. Were MIT seriously considering reversing its ground-breaking course to open up access to its lectures, I think there would be more talk. Maybe I missed other discussion, but if this turns out to be the case then the open courseware movement has been dealt a serious blow.

I’m starting something new on DLTJ: Thursday Threads — summaries and pointers of stories, services, and other stuff that I found interesting in the previous seven days. This is culled from entries that I post to my FriendFeed lifestream through various channels (Google Reader shared items, citations shared in Zotero, Twitter posts, etc.), but since I know not everyone is using those services, it might be useful to post the best-of-the-selected here once a week. Why Thursday? Somewhere long ago I read that Thursday at 11am is the best time to put a post on a blog because Thursday lunch through Friday are the most active time for readers. I have no idea whether that is true or not, but lacking any evidence to the contrary, Thursday morning will do fine. (Obviously I’m a little late on this first one, but I’ll try to do better next time. Or not — maybe this will be a one-off weekly thing.)

MagCloud — On-demand printing of magazines

MagCloud, the revolutionary new self-publishing web service from HP, is changing the way ideas, stories, and images find their way into peoples’ hands in a printed magazine format. Whether you are a novice or experienced publisher, MagCloud offers you a way to create commercial quality magazines, printed on demand with no upfront costs or minimum print runs. MagCloud is creating new ways to bring consumers and publishers together in a web-based marketplace where choice, flexibility and print on demand are the cornerstones of the community.

Could be useful for short-run, professional printing. I learned about this via a conference call with the editorial board of the NISO International Standards Quarterly.

Chris Anderson: How web video powers global innovation (TED Talk)

TED’s Chris Anderson says the rise of web video is driving a worldwide phenomenon he calls Crowd Accelerated Innovation — a self-fueling cycle of learning that could be as significant as the invention of print. But to tap into its power, organizations will need to embrace radical openness. And for TED, it means the dawn of a whole new chapter …

TED curator Chris Anderson takes the stage to talk about what he has seen as the impact of putting TED talks on the net specifically as well as the general case for the impact of services like YouTube on worldwide culture. This is definitely gets one thinking about the power of the visual medium. Closer to home, it also should get one thinking about assisting library patrons in creating and curating this content, no?

Plain English

Every field has its own jargon that’s meaningless to everyone else. Sometimes you want to translate a given -ese into lay terms while preserving the original text. Plain English is designed to facilitate this. The premise is straightforward: The original text is highlighted in yellow. When you click on a phrase, it toggles to the re-written simpler version, in gray. Buttons at the top allow you to toggle the whole thing at once. The words are stored in a simple JSON file.

From the laboratory of Slate Magazine comes this technique for toggling between one set of words and its translated form. I first found this on the NPR Planet Money blog in a post titled The Fed, Translated Into English. They used it to “translate” Fed-speak (e.g. the very dense statements released by the U.S. Federal Reserve) into more common language.

Google New

The one place to find everything new from Google.

Found via Jason Griffey’s post on his American Libraries Perpetual Beta blog. I noted there my frustration that Google New didn’t have an RSS feed to make this list of new things more machine-actionable. I still think that this missing feed functionality is strange, and if I get a chance at some point I’ll try to feed the page through Yahoo! Pipes to make one.

Rising Into the Public Domain: The Copyright Review Management System (CRMS) at the University of Michigan

Interview with John Wilkin, Associate University Librarian for Library Information Technology and Executive Director, HathiTrust and Principal Investigator for CRMS

Interesting insight into how the University of Michigan is tackling the 1923-1963 orphan works problem. (Found via James Grimmelmann)

$1000 bounty offered for JPEG2000 support in Firefox

We’ve waited long enough. Apparently Firefox needs to be dragged kicking and screaming into the early 2000′s. I have a financial interest in seeing this implemented, so I’m going to step up.

I’m going to offer a $1000 bounty for native JPEG2000 support in Firefox, on Windows, Mac, and Linux.

Comment #155 on this feature request has someone putting up real money to have a developer integrate JPEG2000 into the Firefox browser. The ensuing discussion gives a glimpse into how hard and how easy it could be.

White House Issues IPv6 Directive

Network World reports: Federal CIO Vivek Kundra has issued a directive requiring all U.S. government agencies to upgrade their public-facing Web sites and services by Sept. 30, 2012 to support IPv6, the long-anticipated upgrade to the Internet’s main communications protocol. Kundra’s memo mandates that agencies use native IPv6 instead of transition mechanisms that translate between IPv6 and the current standard, which is known as IPv4.

You may not have heard this, but we’re running out of IP addresses. An IP address is the thing computers use to find each other on the net (and not to be confused with domain name system (DNS) addresses — the human friendly things that we put on our business cards and advertisements). In the current version of the Internet Protocol (IPv4), we only have about 4 billion addresses and we’ve used up 95% of them. There has been a big press this year to move to the next generation Internet Protocol (IPv6) that will give us 340 billion billion billion billion addresses (or roughly 50 billion billion billion addresses for each person alive in 2012 when the 4 billion addresses of the existing Internet Protocol run out). The entry of the federal government into the push for IPv6 is expected to accelerate adoption of the new standard.

Today at IBC2009 Fast Forward Video (FFV) announced the launch of the JPEG2000 Alliance, “a consortium of broadcast industry leaders dedicated to ensuring that JPEG2000 continues to develop into a leading compression standard.” According to the press release:

In addition to developing their own JPEG2000 technologies and products, these companies will collaborate to ensure widespread acceptance, deployment, and support of the compression standard for the benefit of the media and video industries. Activities will be centered on educating and creating awareness about the benefits of JPEG2000, promoting interoperability between standards and system devices, and promoting the development of tools by members and industry peers.

In addition to FFV, charter members of the JPEG2000 Alliance include 360 Systems, Analog Devices, Barco, Digital Rapids, Doremi Labs, the Fraunhofer Institute, Front Porch Digital, intoPIX, Media Links, Media Matters and Miranda Technologies. I’m taking the time to post this (and I’m hoping you are taking the time to read it) to give a sense of how JPEG2000 is being used outside the cultural heritage community.

I’m not sure what “IBC” stands for (their website doesn’t expand the acronym), but according to their “about” page they are “the leading international forum for the electronic media industry.” Based on the nature of the six partners behind the IBC (International Association of Broadcasting Manufacturers, Institution of Engineering and Technology, IEEE Broadcast Technology Society, Royal Television Society, Society of Cable Telecommunication Engineers, and Society of Motion Picture and Television Engineers), it seems to be geared towards broadcast media. There is a conference going now in Amsterdam where they expect to have over 49,000 attendees and 1,300 exhibitors.

Although my day-to-day work takes me farther away from working with digital collections in general and JPEG2000 specifically, I still have a Google News search set up looking for hits on JPEG2000 topics. An entry appeared yesterday that gives some interesting insight into how motion JPEG2000 is being used in broadcast video transmission: “HBO Opens T-VIPS Video Gateways: Norweigan Vendor Helps Premium Net Ship Content Coast to Coast”

The article describes how HBO is using video gateways based on the JPEG2000 standard “to transport high-definition programming from its New York City studios to the HBO Communications Center.” The device, a TVG430 HD JPEG2000, encodes and decodes HDTV signals in motion JPEG2000 for transmission over gigabit ethernet. (Take a look at the data sheet [PDF] for all of the fine details about the product.) The article also describes some of the operational advantages and disadvantages of real-time motion JPEG2000 transmission:

For HBO and other clients, JPEG2000 has proven to have a number of advantages over MPEG formats for video-signal transport, Dolvik said. MPEG signals that are repeatedly encoded and decoded have much poorer image quality than JPEG2000 signals, and JPEG2000 does a significantly better job of error correction. In addition, the latency for JPEG2000 signals is about 120 milliseconds, compared with as much as two to four seconds for MPEG.

A downside to JPEG2000 is that it requires significantly more bandwidth than MPEG. This isn’t a major problem for sending content over IP networks, in which bandwidth has become much less expensive, but it is a significant issue for “the last mile” connection into homes where bandwidth is often extremely limited.

Very interesting to read, even if it doesn’t have a direct impact on libraries and other cultural heritage institutions. It does show, though, that JPEG2000 is gaining market share and mind share in other fields.

The text was modified to update a link from http://www.t-vips.com/sites/default/files/datasheets/0409_Datasheet_tvg430.pdf to http://www.t-vips.com/sites/default/files/datasheets/Datasheet_tvg430.pdf on January 28th, 2011.

The text was modified to remove a link to http://www.t-vips.com/sites/default/files/datasheets/Datasheet_tvg430.pdf on June 9th, 2011.

The lead article in the September/October issue of D-Lib Magazine release yesterday is on djatoka, the open source JPEG2000 Image Server from Los Alamos National Laboratory. The authors, Ryan Chute and Herbert Van de Sompel describe their effort in the article abstract:

The ISO-standardized JPEG 2000 image format has started to attract significant attention. Support for the format is emerging in major consumer applications, and the cultural heritage community seriously considers it a viable format for digital preservation. So far, only commercial image servers with JPEG 2000 support have been available. They come with significant license fees and typically provide the customers with limited extensibility capabilities. Here, we introduce djatoka, an open source JPEG 2000 image server with an attractive basic feature set, and extensibility under control of the community of implementers. We describe djatoka, and point at demonstrations that feature digitized images of marvelous historical manuscripts from the collections of the British Library and the University of Ghent. We also call upon the community to engage in further development of djatoka.

The article is very easy to read and is a great overview of how they built the djatoka image server. LANL has a demonstration site with images of the Magna Carta from the British Library. The University of Ghent has also deployed a djatoka installation with some digitized pages of a Gregorian choir book. (The text of the site is in Dutch, I think, but you can click on the square boxes to the right of “Fol.” to bring up the images.) LANL has also put together a screencast demonstration of djatoka, included below.

djatoka is available under the GNU Lesser General Public License. The software has a site on SourceForge with forums for discussion. It runs as a Java servlet, so it is pretty much cross-platform. In the image server is the Kakadu JPEG2000 toolkit and the IIPImage JavaScript Viewer toolkit. One other key piece is a fascinating use of an OpenURL ContextObject to carry the service request information from the browser through the image server to the caching and rendering pieces.

Congratulations and kudos to Ryan, Herbert, and the team at LANL for putting together this great piece of software and releasing it as open source.Ryan Chute, Herbert Van de Sompel (2008). Introducing djatoka: A Reuse Friendly, Open Source JPEG 2000 Image Server D-Lib Magazine, 14 (9/10) DOI: 10.1045/september2008-chute

David Lowe, Preservation Librarian at the University of Connecticut, is coordinating a survey of JPEG2000 use for digital imagery. The survey asks questions about the use of the JPEG2000 file format (for archival purposes or for access systems), tools used (both JPEG2000 toolkits and software that embeds JPEG2000 toolkits), and considerations of mathematically lossless versus visually lossless compression settings.

This is his announcement:

I am writing to solicit your help with a survey of library-related digital project staff regarding the implementation of the JPEG 2000 standard for digital images (specifically still images and not motion). We estimate that this task will take approximately 15 minutes of your time. It is available now at: http://www.surveymonkey.com/s.aspx?sm=WXFAJwyRNZZilRWzrnum_2fw_3d_3d

The survey will remain active until October 31, 2008. Afterward, we will post the results via a report uploaded to our institutional repository, digitalcommons.uconn.edu.

Please note that in our report, personal information from the survey will not be revealed, and any comments used will remain unattributed unless the respondent prefers to be credited and indicates that desire in a separate email to me directly at david.lowe@uconn.edu.

Thank you for your help,

David Lowe
Preservation Librarian
UConn Libraries

I encourage you to take the survey as well.

Download iCal file

The JPEG2000 in Archives and Libraries Interest Group of the LITA division of ALA is pleased to present a program on Archiving in Practice with JPEG2000 on Sunday, June 29th from 8am to 10am in Ballroom E, Anaheim Convention Center.

The lead presentation will be given by Justin Dávila, Digital Media Workflow, Business and Technology consultant. The formal presentation will be followed an invitation to the audience to present five-minute lightning talks on the topic of JPEG2000 for cultural heritage archiving and access. More details can be found in the announcement on the j2kArcLib.info website.

Update

20080717T1338 : Slides from the presentation are posted on the ALA Presentations website.

The text was modified to remove a link to http://j2karclib.info/node/120 on January 19th, 2011.

The text was modified to remove a link to http://www.lita.org/ala/lita/litamembership/litaigs/jpeg2000/jk2kig.cfm on June 9th, 2011.