Without careful filing, it can be hard to find a specific image. However, most image file formats (including all we recommend here) include space for metadata that facilitate cataloging and searching with a digital asset management programs (such as Extensis Portfolio or Expression Media). Embedding IPTC metadata in image files early in a workflow, paired with logical file numbering and naming conventions, speeds up cataloging and facilitates searching.
Storing multiple copies of image files on redundant hard drives is arguably the most convenient and least expensive solution to guard against data loss. Additional back-ups to CD-R or DVD-R media may be prudent. Note, however, that different brands and batches of CD-R and DVD-R media have different archival qualities. Use only media with high archival ratings; burn discs at lower speeds; do not use adhesive labels on discs; and write on discs only with non-solvent markers designed specifically for inscribing CDs and DVDs, writing only on the hub of the disc.
A prudent practice is to maintain a complete copy of your photo archive off-site. However, be aware of one potential problem with any type of online copy: If there is corruption in your primary copy and you have your system set to automatically clone it to the backup, you may corrupt the backup.
Over time, as operating systems change and new storage media emerge, you will likely need to migrate your digital image files. Most businesses and institutions have been slow to recognize the need for digital asset management, so it generally falls to photographers to maintain digital image archives. It’s wise to discuss archiving responsibilities with clients to avoid losing image files.
archival image formats
Archiving raw image files from Digital SLR cameras will likely be a key challenge in the long-term maintenance of a digital archive. Because each camera model creates a unique file type, it’s likely some files will become unreadable in the future. Photographers should develop a comprehensive strategy to confront this issue, accounting for the probable obsolescence of file formats and for the need for file format migration.
File format migration refers to the practice of converting image files to a different (newer) storage format. This might mean conversion to TIFF files, conversion to DNG files, or conversion to some future format not yet invented. Each choice holds certain advantages and disadvantages for image quality, storage needs, and workflow requirements.
conversion to tiff files
By converting images to TIFF format, the photographer stores them in the most accessible file format. And because TIFF is an open standard, it should be readable for a very long time. TIFF also offers a workflow advantage: By converting to TIFF, you probably eliminate the need to reconvert the files again for many years, perhaps even for your lifetime. So images can be converted and archived with confidence they will be safely accessible long into the future.
There is a downside, however. TIFF files are much larger than raw files. Converting image files to 16-bit TIFFs can make the files up to 10 times larger than raw files and 15 times larger than compressed DNGs. This clearly increases the cost of file storage (assuming other costs stay the same). Although TIFF files have several compression options (LZW, ZIP, and JPEG), none is recommended due to the lack of universal support for a compressed TIFF format. This is especially important if you do not know exactly who will receive the file. Another downside to TIFF conversion is that it precludes the use of better raw converters surely coming in the future. Just as Photoshop CS3 converts raw files better than Photoshop CS2, it is likely the file conversion programs available several years from now will do an even better job.
archiving jpeg files
Conventional wisdom suggests the TIFF format holds a quality advantage over the JPEG format. This holds true only if the JPEG file is saved at less than 10 quality, using the Photoshop standard. Artifacts are either non-existent or insignificant at JPEG quality 10 or 12. Higher bit-depth is really the only advantage of using TIFF over JPEG 10 or 12 (in terms of image quality). Some have argued that JPEG, because of the way it encodes data, compromises color. This is a misconception. When using the highest quality settings, there is no loss of color fidelity. Therefore, if JPEG files are saved at 10 to 12 quality, and if they do not require much pixel editing before use, archiving JPEG files is not a bad concept. And it can save a lot of space. For many picture archives, the economics of storing large numbers of files dominates all other considerations, and JPEG offers a feasible solution to the problem.
archiving raw files
A photographer who chooses to archive raw files preserves the largest number of options for future file conversions. As software improves, new conversions of the image file might offer better color fidelity, better noise reduction or better upsizing interpolation.
This, too, has its downside. Raw files will likely require conversion to a more universal file format at some time in the future. For a single photographer, this may involve converting and re-cataloging hundreds of thousands of image files. If this conversion is not accomplished before that particular format becomes generally unreadable, then the conversion may simply never happen. And those images may be lost. Additionally, since raw files are proprietary, it may not be safe to alter these files. This means any work on these files, such as adding metadata or tonal adjustments to the image, should not be stored in the file itself. Instead, use either “sidecar” files or store these adjustments in a database. But the added files can become a file-management challenge that may present a serious roadblock in the future, as you try to include such work in a conversion file. Although some software can add metadata to raw files, there can be conflicts with Adobe Bridge not reading the processing information in the XMP file after metadata has been added by another program. Always test a backed-up image for anomalies before adding metadata to proprietary raw files.
archiving dng files
A raw file can also convert to DNG format, a documented, TIFF-based format created by Adobe. DNG can store the raw image data, metadata and a color-corrected JPEG preview of the image. The DNG file format provides a common platform for information about the file and adjustments to the image. Photoshop opens DNG in Adobe Camera Raw just as it does raw files, offering the full range of conversion options available with the original. Cataloging applications such as Expression Media and Extensis Portfolio can see metadata changes and image adjustments made in Camera Raw.
DNG will likely be readable long after most original raw formats become obsolete, simply because there will be so many more DNG images than those in any proprietary raw format. DNG also offers a lossless compression that can reduce the file size up to a third. In addition, DNG allows embedding the raw file itself, so it can be re-converted later if desired — although since this doubles the file size, embedding the original file may not be desirable.
There is, however, a downside to DNG: Conversion requires an extra step at the time of raw file processing. While this does not take terribly long, it is an extra process. Moreover, by converting a raw file to DNG, you currently preclude using the camera manufacturer’s software for converting this file. If you like the conversions you get from the manufacturer’s software more than the conversions you get from Photoshop, then DNG may not be acceptable.
The DNG converter attempts to copy “Undocumented Maker Notes” to the metadata of the DNG file. These maker notes might include information that could be useful for converting the file at a later date (such as “dust reference” information, or information helpful in correcting chromatic aberration). By converting to DNG today, you may lose this data, since it is currently undocumented and it may not copy correctly.
archiving jpeg2000 (.jp2, .jpx, .jpf)
Created by the Joint Photographic Experts Group committee in 2000, JPEG2000 is a wavelet-based, image-compression standard. JPEG2000 offers a compression performance gain of around 20 percent compared to the standard JPEG. For lower or higher compression rates, the improvement can be somewhat greater. It has, however, notably higher computational and memory demands, which for most photographers means opening and closing JPEG2000 files takes more time. Unlike the standard, baseline JPEG, JPEG2000 does offer a fully lossless option. Improved compression options and performance, however, are not the main benefit of JPEG2000. The real achievement of JPEG2000 is “smart decoding.” This feature enables a decoding application (or plug-in) to access and decode only the required portion of the code stream. This means a single JPEG2000 image can supply multiple, reduced-resolution versions of the original. These might include specific file sizes, and/or a high-quality, high-resolution view of a specific portion of the image. This makes JPEG2000 an excellent format if you require the ability to smoothly zoom, pan and rotate images. Creating compressed images that contain different quality levels allows master images in an archive to supply multiple derivatives, saving time and bandwidth. This makes an image archive much more efficient. In addition to this array of output options, JPEG2000 can handle very large images, at least up to a terabyte. JPEG2000, unlike standard, baseline JPEG, supports high-bit-depth (up to 16 bits per channel vs. 8 for standard JPEG) and high-dynamic-range images. JPEG2000 also underpins the MJ2 and JPM formats for motion images (each frame is a JPEG2000-compressed image) and compound images (images, graphics and text). These additional features make the JPEG2000 format a potentially valuable option for archiving film, video and historical materials.
JPEG2000 has not been widely adopted by photographers. Raw and DNG are viewed as better solutions for their specific needs. And JPEG2000 is not a good candidate to replace standard JPEG as a digital capture format, since the former requires far greater compression time. Still, photographers may want to become more familiar with JPEG2000 and be prepared to deliver in one of its variants, since many cultural heritage and digital preservation communities use it as the basis for their collections. Unlike photographers, these institutions are less concerned with improved rendering options over time. Their focus is preserving a specific rendering intent in the best way possible, with the most efficient storage and delivery options. JPEG2000 does these things well. The unanswered question is whether this "niche" adoption of JPEG2000 will ensure its long-term viability as an archival format. Some of the cultural institutions finding JPEG2000 to be the best current solution for their archiving needs include The Library of Congress, the Harvard University Library, Library and Archives Canada, Chronicling America website and the Google Library Project. For more information, see the Digital Preservation Coalition Technology Watch Report.
archiving hd photo/jpeg xr
Microsoft has thrown its hat in the ring of digital formats. Its technology was originally called Windows Media Photo, then renamed HD Photo, and Microsoft is now working with the Joint Photographic Experts Group to make the format a JPEG standard, JPEG XR (for extended dynamic range). HD Photo has shown no compelling advantages over JPEG2000 for archival purposes. But HD Photo does have a faster compression algorithm compared to standard JPEG, and this offers some potential advantages as a digital capture format. Since the main drawbacks to the standard JPEG format are the 8-bit tonal depth limit and limited dynamic range, JPEG XR could provide a welcome step. Covered by Microsoft’s Open Specification Promise, JPEG XR avoids problems associated with proprietary raw formats. The future of JPEG XR depends on camera makers offering it as a capture format. If and when this happens, then JPEG XR would become a potential archival format as well.
HD Photo is also the basis for the Microsoft Seadragon project and the related Photosynth photo viewing technology. These new technologies could also drive the adoption of HD Photo, since they both use HD Photo’s ability to decode only the needed portions of an image, allowing for rapid screen draws as images are combined, zoomed, and panned. The latest Windows operating systems include full support for HD Photo.
| NOTE: The archiving JPEG section is based on research and analysis by Ken Fleisher. Peter Krogh prepared the rest of this analysis of archival image formats. Peter offers a more in-depth discussion of these issues, as well as details related to the creation and maintenance of a digital archive, in The DAM Book: Digital Asset Management for Photographers (O’Reilly, 2006). It’s available directly from the author, from O’Reilly or from Amazon and similar distributors. |
archiving 32bit hdr images
HDR Imaging is a rapidly evolving field with very few set standards at this point. HDR image formats include: Radiance HDR, OpenEXR, JPEG-HDR, Windows WDP, TIFF LogLuv 24/32, Cineon DPX, TIFF float, PFM, and TGA. Radiance HDR and OpenEXR currently offer the most application support.