Download iCal file

The JPEG2000 in Archives and Libraries Interest Group of the LITA division of ALA is pleased to present a program on Archiving in Practice with JPEG2000 on Sunday, June 29th from 8am to 10am in Ballroom E, Anaheim Convention Center.

The lead presentation will be given by Justin Dávila, Digital Media Workflow, Business and Technology consultant. The formal presentation will be followed an invitation to the audience to present five-minute lightning talks on the topic of JPEG2000 for cultural heritage archiving and access. More details can be found in the announcement on the j2kArcLib.info website.

Update

20080717T1338 : Slides from the presentation are posted on the ALA Presentations website.

The text was modified to remove a link to http://j2karclib.info/node/120 on January 19th, 2011.

The text was modified to remove a link to http://www.lita.org/ala/lita/litamembership/litaigs/jpeg2000/jk2kig.cfm on June 9th, 2011.

On the ImageLib mailing list, Rob Lancefield (Manager of Museum Information Services for Wesleyan University) posted a link to the Universal Photographic Digital Imaging Guidelines (UPDIG) for image creators. The introduction says: “These 12 guidelines — provided as a Quick Guide plus an in-depth Complete Guide — aim to clarify the issues affecting accurate reproduction and management of digital image files. Although they largely reflect a photographer’s perspective, anyone working with digital images should find them useful…. This document, prepared by the UPDIG working group, represents the industry consensus as of September 2007.” The listed members of UPDIG leads one to believe that this is a professional photography group. One thing in the introduction to the guidelines caught my eye, though:

The chapter on archiving now has a discussion of JPEG as an archival format.

Note that the authors do indeed mean JPEG (circa 1994), not JPEG2000. The chapter on archiving lists the pros and cons of a number of formats, to include JPEG. The following bullet points are excerpted from the text.

• Conversion to TIFF files: By converting images to TIFF format [from camera RAW], the photographer is storing the images in the most accessible file format… There is a downside, however. TIFF files are much larger than RAW files… Another downside to conversion to TIFF is that it precludes the use of better RAW converters that are surely coming in the future.
• Archiving JPEG files: Conventional wisdom holds that the TIFF format holds a quality advantage over the JPEG format. This holds true only if the JPEG file is saved at less than 10 quality using the Photoshop standard. When using JPEG quality 10 or 12, the artifacts are either non-existent or insignificant. Higher bit-depth is really the only advantage of using TIFF over JPEG 10 or 12 (in terms of image quality)… Update 2008-02-11: Please see below.
• Archiving RAW files: If a photographer chooses to archive the RAW file, then he will be preserving the largest number of options for future conversion of the files… This, too, has its downside. RAW files will likely have to be converted to a more universal file format at some time in the future.
• Archiving DNG files: RAW files can be converted to DNG, a documented TIFF-based format created by Adobe that can store the RAW image data, metadata, and a color-corrected JPEG preview of the image. The DNG file format provides a common platform for information about the file and adjustments to the image… DNG is likely to be readable long after the original RAW format becomes obsolete, simply because there will be so many more of them than any particular RAW file format… There’s a downside to DNG, of course. Conversion to DNG requires an extra step at the time of RAW file processing; it does not take terribly long, but it is an extra process.

Update 2008-02-11: Ken Fleisher noted in the comments that the excerpt above was truncated before his reasoning was described. In the interest of clarity, the full text of this bullet point on the UPDIG site is:

Archiving JPEG files: Conventional wisdom holds that the TIFF format holds a quality advantage over the JPEG format. This holds true only if the JPEG file is saved at less than 10 quality using the Photoshop standard. When using JPEG quality 10 or 12, the artifacts are either non-existent or insignificant. Higher bit-depth is really the only advantage of using TIFF over JPEG 10 or 12 (in terms of image quality). Some have argued that that JPEG, because of the way it encodes data, compromises color. This is a misconception. When using the highest quality settings, there is no loss of color fidelity. Therefore, if JPEG files are saved at 10-12 quality, and if they do not require much pixel editing before use, archiving JPEG files is not a bad concept, and it can save a lot of space. For many picture archives, the economics of storing large numbers of files dominates all other considerations, and JPEG offers a feasible solution to the problem.

The notes at the end of the chapter say: “The archiving JPEG section is based on research and analysis by Ken Fleisher.”

So I wonder what is going on here. Does the cultural heritage community have a different definition of the word archive from the professional photography community? Are there sufficient differences in our goals that warrant the differences in practices?

This topic is of interest because the program of the JPEG2000 in Archives and Libraries Interest Group of the Library and Information Technology Association (LITA) will be holding a panel at the ALA Annual Conference in Anaheim this summer on using the JPEG2000 file format for archival purposes. Part of the discussion will center around the notion of visually lossless versus data lossless compression. This mention of lossy-yet-high-quality JPEG compression seems to fit into the same topic.

The text was modified to remove a link to http://www.ala.org/ala/lita/litamembership/litaigs/jpeg2000/jk2kig.cfm on June 9th, 2011.

Download iCal file

Saturday, January 12th from 1:30pm to 3:30pm — j2kIG Meeting at Chestnut room in the Radisson Plaza hotel

The JPEG2000 Interest Group of LITA will be holding a business meeting to discuss plans for a program at the ALA Annual Conference in Anaheim in June. Anyone with an interest in the use of JPEG2000 in Archives and Libraries is welcome to attend the meeting, whether or not you are a member of LITA.

Agenda

1. Finalize plans for program at ALA Annual in Anaheim
2. Interest group renewal and solicitation for an IG chair
3. Information sharing about projects, issues, successes/failures

Proposed ALA Annual Program Details

Proposed Program Title: Archiving in Practice with JPEG2000

Program Restrictions (i.e., speaker date restrictions, conflict times, etc.): None at this time

Program Day and Time:

Tentative Program Description (75 words or less): The topic is surrounding the use of JPEG2000 as an archival format. Part of the presentation would be on the use of JPEG2000 as an archival format, timed to follow the release of an in-depth study on this topic by Stephen Abrahams, Stephen Chapman, and John Kuntz for the IS&T conference in the fall. Participants in the IG meeting would like to see this perspective balanced with a speaker offering considerations on why one should choose not to adopt the standard. Another part of the program would be on the process for considering the adoption of the JPEG2000 file format in practice, perhaps someone from the Library of Congress related to the NDNP project. A third component of the program would be from a vendor and/or open source tool user on the available toolsets and what to consider when adopting the new file format. [This more extensive description is offered in leu of an in-person meeting with PPC. It will be trimmed for the published program copy.]

Target Audience and Estimated Size: Digital library practitioners, archivists; approximately 150-200 in attendance

Possible program track (identify 1, 2 and 3 choices):
1st choice: Digital Information & Technologies
Subtrack: n/a
2nd choice: Transformation & Innovations
Subtrack: n/a
3rd choice: Collection Management & Technical Services
Subtrack: Digital Collection Development

Minutes of the JPEG2000 Interest Group have been posted to the j2kArcLib.info website. Comments there are restricted to registered users of the site (although registration is freely available), so feel free to post comments here.

The text was modified to remove a link to http://j2karclib.info/node/113 on January 19th, 2011.

The text was modified to remove a link to http://j2karclib.info/j2kIG on January 19th, 2011.

At the meeting we will be sharing observations and experiences with JPEG 2000 for access and preservation of still and moving pictures as well as discussing ideas for advocacy and spreading information. Membership in LITA is not required to attend the meeting. Get this meeting as an iCal file suitable for importing into most calendar programs.

38.9077° by -77.0353°

The text was modified to remove a link to http://j2karclib.info/j2kig on January 19th, 2011.

OhioLINK was excited and privileged to participate in the second annual Google Summer of Code — a program to inspire young developers and provide students in Computer Science and related fields the opportunity to do work related to their academic pursuits during the summer, and to support existing open source projects and organizations. This is the first of three posts summarizing the efforts of three students; this one details the work of Juan Pablo Garcia Ortiz, a Ph.D. candidate at the University of Almeria in Spain, to build a JPEG2000 JPIP Streaming Server and Client Browser Viewer Applet. This is an edited version of his final report.

The final applications are the ‘jpip_server‘ and the ‘J2KViewer‘ — both of which can be found in the OhioLINK subversion repository. Please note that both are built atop the Kakadu JPEG2000 code library, only a portion of which is included in the OhioLINK subversion repository. ¹

JPIP Streaming Server

The multi-threaded mechanism is implemented with the Linux ‘pthread’ library and uses one thread for each client socket. The source code is written in C++, using the STL library to simplify the code, making it more legible, and orienting it to the object oriented programming philisophy. The different modules of the program are:

• report.cpp (report.h): For the class ‘report’, used to generate the applications logs.
• sock.cpp (sock.h): For the class ‘sock’, used to simplify the sockets work in Linux.
• j2k.cpp (j2k.h): Contains the class ‘j2k_image_file’, used to abstract the work with J2K images. It also contains the necessary code to initialize the Kakadu message handling system.
• jpip.cpp (jpip.h): Contains several classes to support a basic version of the JPIP protocol, over the Kakadu library.
• jpip_server.cpp: The main module.

It is recommended to start from a freshly unpacked (but not yet build) version of the Kakadu library source code (version 5.1). To build this JPIP, follow these steps:

1. At the top level of the Kakadu source directory, checkout the ‘jpip_server’ directory from the OhioLINK Subversion respository. The Subversion URI is https://drc-dev.ohiolink.edu/svn/jpip_server/
2. Change the current directory into ‘jpip_server’ and call the script ‘kakadu_rebuild‘. The script will create the Kakadu directory structure, apply some necessary patches, and build the Kakadu libraries. Please note that this script was written for Kakadu version 5.1 and the patches may not be required in subsequent versions.
3. Run ‘make’ (inside the jpip_server directory)

The accepted parameters by the server can be seen executing ‘jpip_server -u’. The images served by the server must be, by default, in the same directory. The command line option ‘-ipath’ allows you to define a base path for all the requested images. This server can be tested with the ‘kdu_show’ application of the Kakadu package. (kdu_show is a Windows application, but can be executed in Linux with WINE without problems.)

JPIP Browser Applet

1. Checkout of the ‘J2KViewer’ directory from the OhioLINK Subversion respository. The Subversion URI is https://drc-dev.ohiolink.edu/svn/J2KViewer/.
2. From within the J2KViewer directory, run ‘make’.

The build process creates a JAR file containing the viewer applet. It’s correct use would be to sign it with a official signature and include it into the appropriate Web page as an applet. The applet accepts the following parameters:

• “Image“: required, specifies the JPEG2000 image to view. If it’s a local image (not typical), this would be a path name (for example ‘/home/jportiz/image.jp2‘). If it’s a remote image (the typical usage) through the JPIP server, it would be a JPIP URI (for example, ‘jpip://server:9000/image.jp2‘). Do not include any JPIP parameters in this URI.
• “MiniViewWidth“: optional, specifies the width, in pixels, of the mini-view.
• “MiniViewHeight“: optional, specifies the height, in pixels, of the mini-view. The final mini-view size will be the maximum multiple by 2 size of the image that can be included within the specified mini-view size. By default, the mini-view size is 250×250.

To be able to test easily the applet, you can run ‘make test‘ to create a ‘test.html’ file and a ‘test’ script. This script launches the applet viewer with the ‘test.html’ page. It is necessary to modify the ‘Image’ parameter of this HTML page to set the appropriate image location.

To test the applet within a web browser, you can run ‘make test-signed‘ to create a signed JAR file, then open ‘test-signed.html’ with a Java compatible browser. Before opening the HTML page, it is necessary to modify the ‘Image’ parameter within the page to set the appropriate image location.

The viewer has a simple UI. It has a toolbar with two buttons, one to select the interaction mode (zoom or pan) and another one to show the mini-view window. There is a status bar where is showed the real image
size and the current scale, and in the right side, the current number of bytes read. In the center panel is the image — initially scaled by 2 fo fit it in the applet size. This behaviour can be modified easily in the code.

The mini-view window can be moved to any part of the main window. If it’s closed, it can be showed again with the associated button of the toolbar. The mini-view contains a red rectangle outlining the actual view of the main window. This red rectangle can be moved by dragging it and the corresponding view in the main window will be updated.

The applet requires that the Kakadu JNI library is stored in the user home directory (‘C:\Document And Settings\User’ in Windows and ‘/home/user’ in Linux). The applet detects if this library exists and if it is not found the applet allows it to be download automatically from the OhioLINK repository and stored it in the user home directory.

This is a brief description of the implemented classes:

• ChunkedInputStream: An InputStream derived class to decode HTTP chunked messages.
• HTTPMessage: The base class of all the HTTP message classes.
• HTTPRequest: Identifies a HTTP request.
• HTTPResponse: Identifies a HTTP response.
• HTTPSocket: Derives from Socket and allows HTTP messages to be sent and received in an easy way. In only supports send HTTP requests and receive HTTP responses (a client socket).
• ImagePanel: A GUI panel that can be included in any image browser. This panel has the necessary methods to load images and control its visualization. It supports the display of a mini-view of the image. All of the image navigation is controlled directly within this panel.
• ImageWindow: An interface to implement all the parent windows that include an ImagePanel obejct.
• J2KCache: A wrapper class for the Kdu_cache Kakadu class. This encapsulation allows another J2K engine to be used while avoiding the need to modify the design and code of the applet.
• J2KEngine: Contains all the necessary code to initialize the J2K engine. In our case, the Kakadu J2K engine.
• J2KException: A Exception derived class to identify the exceptions generated by the J2K code.
• J2KImage: This class contains all the necessary code to work with J2K images, local files as well as remote URIs. If the image is local, the Kakadu library functions are used to open it. If the image is remote, a J2KReader object is used to retreive the content.
• J2KImageView: Defines an image view, renderable on the screen. This image view is defined by its coordenates, size and resolution level.
• J2KReader: An implementation of a basic JPIP client to read the necessary data of a specific image view.
• J2KRender: Implements the rendering thread of the image browser. This thread is always generating the current image view. It stops when this view is rendered and the image content is completed (if the image is remote, this means that all of the required image codestream blocks have been received).
• J2KViewer: The main class of the application.
• JpipConstants: Defines several global constants related to the JPIP protocol.
• JpipDataInputStream: An InputStream-derived class to extract JPIP data segments from an input.
• JpipDataSegment: Contains the information of a JPIP data segment, which can be either an EOR message or a data-bin segment.
• Mutex: A simple mutex implementation in Java, used to avoid threads conflicts when accessing to the image data.
• StringInputStream: An InputStream-derived class to extract strings from inputs without buffering. Useful to decode HTTP headers and chunks lengths.

The viewer follows the JPIP philosophy in the code structure, especially with remote images. That is, when it is displaying a remote image, there are three threads running in parallel: one for the main GUI, a second for the JPIP comunication that is making the requests and filling the cache with the received data, and a third to render the cache content on the user’s display.

The Java JPIP viewer has been tested in Windows and in Linux. In Windows it has been used the Sun JVM/JRE 1.5 without problems. However, in Linux, this JVM/JRE version seems quite unstable, requiring the previous version, 1.4.2. In Linux the Blackdown JVM 1.4.2 as well as the Sun JVM 1.4.2 have been tested with succesful results.

Future Development Goals

Another feature that was part of the original design but was not completed due to time pressures was to integrate the JPIP server and client into the FEDORA digital object repository system. There is also room for a lot of more features: support of the HTTP-TCPtransport protocol, allow for the display of the image metadata, more GUI utilities like the ability to save the image as a local file for image processing, increase the set of supported JPIP parameters, etc.

Gratitude and Acknowledgements

The text was modified to remove a link to http://drc-dev.ohiolink.edu/browser/jpip_server/ on January 13th, 2011.

The text was modified to remove a link to http://drc-dev.ohiolink.edu/browser/J2KViewer/ on January 13th, 2011.

The text was modified to remove a link to http://drc-dev.ohiolink.edu/browser/jpip_server/kakadu_rebuild on January 13th, 2011.

Footnotes

1. Editorial comment: although not released with an open source license, the Kakadu JPEG2000 code library is available at a very reasonable cost. Juan Pablo’s efforts initially started with one of the open source JPEG2000 libraries, but it was quickly determined that the Kakadu library was required to meet the demands of the project proposal.