A few weeks ago, wrote about my rapidly failing film and negatives from Cord Industries. At the end of that post I alluded to the fact that neither hard drives or CD disks are considered safe bets for long term image storage.
Now ZNet blogger Robin Harris is posting about something called “silent corruption” on hard disks. If, like me, you store thousands of images on disk, it’s’s enough to keep you awake at night.
Harris links to a study by CERN, the world’s largest particle physics lab. The study suggests that data stored on hard disks isn’t as safe as we might like. While CERN is concerned with raw data, it would appear that our disk-stored photos are equally at risk.
Your backup is only as good as your storage media
The term data corruption doesn’t sound all that terrible. You might think that minor corruption would only affect a few pixels here and there. If that were the case, someone with good Photoshop skills could restore the corrupted image. Such is not the case.
A couple of years ago I had a nearly full Compact Flash card go corrupt on me. It worked fine while I was shooting, but when I inserted the CF card into my reader, I was greeted with a “Card Unreadable” error message.
Fortunately, I was able to recover most of the images using data recovery software. However, several images were truly and totally corrupt.
The recovery program was able to copy the bad images off the card. but there wasn’t much hope for the data they contained. The images were super-imposed over one another, in much the same way as if you made a double exposure on a piece of film. In addition, there were weird bands of color passing all through the corrupt images. I consider myself something of a Photoshop expert and I can tell you there was no hope of restoring those images. They were well past salvage.
Now imagine the same scenario on a huge hard drive with several terrabytes of stored images. if the drive becomes corrupt, how many images could you stand to lose? Sure you can back your images up, and you should. If both the primary and backup sources fail at the same rate, however, where does that leave you?
Recently I started a project that involved digitizing some Kodak 126 negatives from the early sixties. The negatives were over forty years old, but still perfectly useable. I have some my parents’ film negatives that are more than seven decades old. You still could get decent prints off them. Will we be able to say that about digital?
Can digital survive as long as film?
I’m not advocating a return to film and I’m not going to be Chicken Little and start waiting for the sky to fall. On the other hand, I don’t want to be complacent and sit idly by while my collection of digital originals becomes so much silicon dust. Even with a rigorous backup plan, however, there is a chance that silent corruption could trash valuable, irreplaceable images.
On the other hand, i have been involved with computers and hard drives since the early ’80s. I have only had one one hard disk fail so badly that I couldn’t recover the data off it. That drive suffered a power failure during the start up procedure, and the heads actually slammed against the platter, making 1.6GB of information unreadable. Sad, but not a disaster. What if it had been 1.6TB of information? And what if the back-up had quietly become riddled by errors while in storage?
There is a chance that I still may be around in another forty years. If I am, I beleive that most of my film negatives will still be usable. You have to wonder, however. How many of the digital images I shoot today will I be able to access in 2047?
Are you concerned about the life of your images stored on hard drives and CDs? What steps are you taking to ensure that your digital images survive? Let us know what you think.
You bet I am concerned. I store my pictures in my internal and external HDD and to an additional CD or DVD. But I agree that it may not be enough.
I guess my solution is to print in archival paper the ones I care the most. You can always scan. By the way, that’s how I got some of my photos from the early 80’s which their negatives were long gone.
Hi Padu;
Glad to know that I’m not the only one taking these warnings seriously. Your suggestion to print to archival paper is good, but I hate to give up the flexibility of a digital file. A RAW file has so much information packed into it that you can’t obtain from a print.
On the other hand, having a print is better than no image at all. So it might be a viable option. Yet, part of the attraction of digital files is how easy it is to store thousands of images in a small footprint. A part of me groans at the thought of printing out hundreds of images just to store them — to say nothing about the cost.
Still, I may consider this avenue for my best images at least.
Thanks for taking the time to share your input.
— Tom
film can be destroyed/spoiled as well. so can prints. and it is much harder to keep them safe guarded for backups.
there are DAT/DDS tapes. It’ll still be cheaper then the other alternative. There is blu-ray in future. Hard drives are cheap and you could have two backups locked away, and it’ll be easier, and cheaper then the alternative.
Not a big deal.
Omer;
You are right, film can fade or spoil — but the point of the post is that the vast majority of my older negatives are still in excellent condition. With proper storage, most of my negatives and slides will out live me.
The thing about the CERN study is that data on hard drives can become corrupt with out warning. So having backup hard drives locked away might not protect your images. And the entire drive may not fail — just certain sectors. If your priceless image happens to be stored on that sector.. you better have a good backup. Data corruption happens, and I would be pretty miserable if I lost important images.
I wanted to drive home the importance of multiple backups and the fact that hard drives may not be the best storage medium for archival digital storage.
DAT tapes might be the way to go, but so many people prefer backing up to a hard drive or CD that tape drives haven’t come down in price the way hard drives have. There was a time when tape drives were about the same price or even cheaper than hard drives. Due to high demand, the price of a HD has fallen dramatically, while the slower selling tape drives haven’t come down in price at all.
Maybe when the general public come around to the notion that DAT tape is much better at storing archival data, sales will increase and the prices will come down.
Thanks for sharing your opinion
— Tom
Tom,
I recommend what I call backup in depth:
-Backup all files every night to a local USB or FireWire hard drive/array for rapid restores if your main drive dies
-Backup all documents/images/movies to DVD every 3-6 months and store in a cool, dark place. This protects against silent data corruption that occurs after the backup.
-Purchase a remote backup service such as Carbonite or Mozy that offer unlimited storage for ~$50/yr. This protects against local catastrophes. Sadly, neither does a very good job with Macs today, but I’m sure they will get there.
Thanks for the link to Storage Bits.
Robin
The issue of silent corruption is taken care of by modern file systems like SUN’s zfs.
The good news for Mac users is: Apple is in the process of adopting zfs. The initial Mac OS X 10.5 release will only have a read-only version of zfs, and it will only be a command-line option. But developer seeds that enable read-write are already out, and it’s clear that either as part of the 10.5 maintenance releases or as part of Mac OS X 10.6 Apple will arrive at full-fledged zfs support.
The issue of silent corruption is taken care of by modern file systems like SUN’s zfs.
The good news for Mac users is: Apple is in the process of adopting zfs. The initial Mac OS X 10.5 release will only have a read-only version of zfs, and it will only be a command-line option. But developer seeds that enable read-write are already out, and it’s clear that either as part of the 10.5 maintenance releases or as part of Mac OS X 10.6 Apple will arrive at full-fledged zfs support.
Other issues are in-memory corruption. Again, back in the days before computer price wars, all computer RAM had parity bits. These days only server grade products do this. So if you’re a Mac user and worried about this, get a Mac Pro, they have all ECC RAM that fights that aspect of the problem.
Between zfs with a RAID-5 zpool and ECC RAM, things are pretty good on the “silent corruption” side of things. Obviously, you’ll still need off-site backup to cover local catastrophic failure (fire, flood, EMP, etc.) Low-grade replacement for off-site backup are uploads to sites like flickr, but it’s better than nothing. Ideally, have a few hundred GB worth of web space and a backup software that’s capable of using WebDAV or ftp as a backup target.
If all these are not enough to save your pictures, you’re likely having more important issues to worry about…
Very informative post! Normally I don’t comment on blogs but this post deserves it 😀
Yeah learn something everyday.
Hm there a bug with the blog using OPera the text is weird :/
feed him one day old pinkie mice snakes can eat thing twice to three occasions their size but 1.5 times the dimension of their grith is better
Oh how I idolised the music euphony from the decade, everything seemed to be way many yeasty than penalization.