Hopefully, many photographers are doing the right thing and backing up their precious images to keep them safe. Problem is, are those particular storage methods any good long-term?
How many photographers are thinking 50 years into the future when it comes to data storage? I know I'm not. It is something, however, that technologist Leo Notenboom is talking about in his video about long-term storage options of our data.
The video starts with Notenboom going through the various options we currently have available to us to store our data and the pros and cons of each. Hard drives, CDs, DVDs, and cloud services are all mentioned. You may be surprised to hear that traditional hard drives over the more modern solid-state drives are where Notenboom is placing his bets for the future. The reason for this is because he says we know a lot more about the longevity of these kinds of storage. He also makes a great point that these older drives are already being used in archival situations, which should mean in decades to come, they are likely to be taken into account when it comes to compatibility.
Two often overlooked areas that are discussed in the video were the actual formats of the drives themselves and the file formats used. It's all well and good having a working drive in 50 years, but if the files or the formats of those drives can't be opened, all your efforts will have been for nothing. This got me thinking about the huge Photoshop files that I have stored and how I naively presume they will always be editable going forward.
All in all, this video is well worth a watch to get you thinking about not just your data long-term, but all the technology we place our bets on as photographers. Notenboom talks about how you and future generations will appreciate the efforts made to ensure your important files can be opened in decades to come. Just imagine how disappointed your great-grandkids would be if they stumble across your life's work on a drive only to find they couldn't open it.
Where are you placing your bets in regards to long-term storage options? What do you think is going to be readable 50 years from now? We'd love to hear your thoughts in the comments below.
Lead image originally by Kowshik Roy sagor, used under Creative Commons.
Don’t they only legally have to store for 7 years though…?
The answer is tape backups which are massively expensive. They don't have to worry about that since they use our dollars.
The logical formats of the file systems will always be readable. The code is there (see FOSS). It will not disappear. If the physical connection is still feasible: I bet it is. Copying data will get easier over time as storage gets bigger and faster.
What's left is reading the files themselves. But even there, there will be software to convert and that will also become faster and faster.
Resumée: Don't worry about it.
That is, if you actively manage your storage. But many people, and essentially businesses will have an archive of some sorts that might go cold for years, and might have older technologies that are hard to replace. Thinking long terms is something people need to do.
I've had a few external hard drives go bad over the last 15 years so I keep multiple backups. Recently, I was concerned with my external SSD drives especially when transferring files from a WIN 7 PC to a WIN 10 PC. Files were repeatedly lost when being transferred. I previously used a 256 XQD card and never had any problem transferring files to and from any computer. I was advised to use the "Safely Move Hardware" utility in Windows which I will give a try and see if that works.
This is a problem with several facets. As the article mentioned, the ability to logically read the data from the media is one issue. Of greater importance is physically being able to access the media. Drive interface formats have changed, leaving old storage methods without an easy and inexpensive means of accessing data. Thus, it is necessary to occasionally migrate our data from older media to newer media. The other key is to ensure the data is stored in more than one place. Cloud storage is an excellent option as one of those storage mediums.
To me this article skips the most important thing, hard drives fail as do SSD's. It's not "if" but "when" and that's my concern. I'm worried about the next 5 to 10 years and not 50 years as a traditional hard drive has a motor and moving heads. Anything that moves can and will fail over time. SSD's have a finite life also based upon a maximum amount of data written.
What I do is use a RAID 1 drive system. With RAID, your files are actually stored on two separate hard drives. Every time you save a file to the drive, it automatically copies the file to the second drive. The RAID system monitors the health of the two drives and warns you when one drive fails or has problems. If a drive fails, the unit automatically switches to the backup drive. Then you simply replace the bad drive and the unit copies the files to the new drive. The only way you can lose a file is if someone takes a hammer to the drive enclosure. While a dual drive RAID enclosure starts at around $300 plus drives, it is cheap insurance for an image that cannot be retaken.
Unfortunately there is another easy way to lose your files without the use of a hammer. Try falling asleep with your hand on the mouse and accidentally click delete as an example. Yawn! Hmm, recycle bin looks a bit full, let's empty it.
Raid does not protect against user error or a system fault or a virus. You need some form of disconnected media with generations of data to be reasonably certain that you have your data.
The masters of photography from last century had tens and hundreds of thousands of negatives and prints. Hardcopies are the only long term solution. Your digital files will not be accessible by a new computer in 20 years; regardless of which format you currently use now. I have several hard drives that I used with Mac computers 20 years ago and I can not use any of them due to hardware incompatibility and software incompatibility issues. What do you do with an old SCSI drive in the world of USB and firewire? The new Macs don't recognize the old file format that my pre-PowerPC MAC used
Hi Brian,
Use your favourite search engine to research the solution. It is entirely possible to read old MAC files and filing systems and, believe or not, SCSI is still in use (though perhaps not with the big old connectors that were used decades ago.
Your solution will almost certainly boil down to either purchasing an adapter or paying a local Mac/PC/Whatever company to transfer the files to something your modern equipment will digest.
Apple have just about the worst history of changing connector and file systems in the IT industry as they push their customers into constant upgrade. If that has happened to you, the quick/cheap answer is to ensure you take your "old" data to your new machine as you upgrade. If the old one died on you nad you decided to replace rather than repair (highly understand in Apple world) then revert to the solutions I outline above.
Your data will be retrrievable as long as you retain a working hard disk of some sort with it on.
You suggest hard copies as an archive solution. I humbly disagree. My professional photo printer using best quality archival papers and inks *claims* that the resulting image has alif expectancy of "up to 80 years" IF placed in an archival quality box that is then stored in a temperature and humidity controlled environment.
Otherwise, all bets are off. If you enjoy looking at your images then they get exposed to light and that expected life time tumbles.
True I have some monochrome prints that are fifty years or more old now and still look "good". My memory does not serve well enough to tell you whether they are as good as the day I pulled them out of the fixer dish. I can tel you they have spent most of their lives in the dark. Anything I hung on a wall or stood on a table went yellow then brown decades ago.
If only I'd had the technology to scan them back in the day - I could have made an accurate replica in less time than it has taken me to reply.
George
I have over 50 years experience in IT. We have a saying:
"Unless data exists *at least* 3 times in 3 separate places it does not exist".
In short - if you make just one backup of your data (photos here) those images might as well not exist as at some time BOTH copies will fail at the same time (eg; consider a fire or network breach as with current ransomware attacks). Then you need the third - the one you kept off-site and detached from the network.
The IT industry (especially open source folks, as others have said) is pretty good at maintaining compatibility with old connection types and file types and systems. I can still read data I put on floppy disks at the pre-dawn of the PC. If you only have a laptop with the latest mini/micro sockets you may need an adapter - but you'll still get that backup device connected and the operating system is certain to read it with, perhaps, just a little encouragement.
Best advice is to advance with the times (I admit, I don't have anything of value only on floppy disks - that data was transferred to hard disk, SSD and USB flash long ago).
Cost puts a lot of people off and it's NOT a good idea to store data on a hard disk, leave it for 20 years then expect it to spin up on command - that's a mechanical device and you have about the same chance of it running as if you parked a car, unused, for 20 years.
HDDs offer the cheapest storage per penny but that saving is worthless if the data you backed up is irretrievable when you want it.
If you don't have access to network storage and fancy enterprise class backup I suggest this (at least this week):
1. Buy TWO fast externals SSD drives (USB 3.0 type)
2. Backup ALL your images to both drives
3. Put one drive in your house, the other in your car (or friend's place, your Mum's ... a garage if it's separate from your home and secure)
4. REGULARLY (daily is best - weekly is OK - monthly is too long) - the actual frequency depends on how often you change data - think not just about when you add new images - photo editing software (Photoshop, Lightroom, Darktable ....) all keep databases, catalogues and sidecar files on disk so those are all changing with each slider you move or effect you apply.
5. NEVER allow all three copies to be in the same place at the same time - do this: Take one external drive to wherever you keep the second and swap them ... head back home and update the freshly retrieved drive. On each backup cycle repeat this process ... exchange the two external drives at the "away" location, bring the oldest back and update it. Rinse and repeat.
Follow this and you will find it very hard to lose data OTHER than by deliberate mistake. What do I mean? Imagine you wiped you main machine photos directory and use a synchronising program (eg; rsync in Linux with the auto-delete missing files option turned on) ... then plug in a backup drive, run your backup and hey presto! All your photos are gone from that device too.
Now aren't you glad you have a third copy?
If you don't notice all your images have gone missing by the time you go fetch the third copy there is no helping you - there is no number of backups that would save you. This has been true since the dawn of the computing industry.
As for cost, external multi-TB SSDs cost a lot less than a modest photo gadget. Save that fancy flash you want till next month and buy a couple of SSDs today.
Live Long, Live Happy. If (when) USB gets replaced with something better, buy a couple new external devices and keep up with the technology.
On the question of file formats it is entirely possible to read files from (consumer machines) that ran the CP/M operating system and even old cassette tapes used by the earliest BBC Micro and gaming consoles.
It's just not extremely easy (as most would, I suspect, prefer).
Digital photos (images) exist in a bewildering array of file formats - sadly mostly PROPRIETARY - ie, the file format (layout, content etc) BELONGS to a company - often a camera manufacturer. Such a format will exist as long as (a) the owning company is around to keep it supported AND (b) as long as software exists that can read the format.
There are several open source formats that preserve much if not all of the data contained in a RAW image (eg; a CR2, ORF, RW2 ... file) including TIFF, PNG and Adobe's DNG. The first two were developed independently and are likely to be around a long time. The DNG format currently enjoys mixed support and I guess it will be around as long as Adobe is around and feels like supporting it. After which it will become another dead format. Its only (potential) saving grace is that Adobe published the format into open source so as long as a program developer feels like keeping it in step with updated operating systems I imagine someone will continue to produce "something" that can read these files and hopefully convert them to a then current format without loss of content.
Big hint: when considering long-term (archival) backup of data IT IS JUST AS IMPORTANT (for reasons just set out - among many others) to retain (ie; back up AS WELL) the operating system and application software used to create or work on the files.
In these days when a virtual machine can be "spun up" in a heartbeat it would be trivial to restore the work environment you had fifty years ago in a virtual environment and allow you to regain not only those precious RAW files but all the sidecar files, databases and catalogues that TOGETHER form an edited, FINAL image. Or you could just output your final work to PNG or TIFF which *can* retain the uncompressed quality - if not how you got there.
You need to decide today whether you or other people will still want to look at the historical methodology behind your images in a century or two. If you are on a trajectory like Richard Avedon or Ansell Adams then back up EVERYTHING. If you just like taking, making and displaying images (like me) then keep the family album in case your grandchildren get bored one rainy afternoon but leave the difficult stuff to others with grander ambitions.
Remember - people still read Chaucer and Shakespeare - though very few have any access to the originals - we are all happy with copies. Nobody has access to these writers' working papers or the content of their minds and thoughts. But their final wok lives on and is considered valuable. My point is - don't back up EVERYTHING "just in case" - concentrate on the important finished work.
One final comment on cloud backup: DO NOT DO IT !!!!
Cloud providers go out of business, change their trading terms without notice or (as happened to one very large data centre operator I use) suffer major fires that take out a few thousand servers for a week or three. Of course you can get back up and running sooner.
You just need an off-site backup and away you go.
You don't have one? Tough luck! Your data is toast (or at least highly carbonated).
Let alone three. Kept separate.
If you find yourself with 5TB of images "safely" stored with a cloud provider who emails to say you have until Friday to take your files back or lose them forever then you are in as much pickle as having never made a backup at all. I know no "cloud" storage provider who guarantees consumers multiple backed up copies of data in separate data centres. "Cloud" is not a synonym for "absolutely secure". Far from it.
In a world of digital photography, if you value your work, having a solid backup regime is more important than the fanciest of lenses. If you have negatives (I have thousands) digitise them ASAP then back up the scanned files. Even a hard disk left unused will outlast most negatives or old prints.
Most of all, if your work has value to you then you and you alone are responsible for preserving it.
Hope this helps
George
I just code it into my gene sequencer, it's cheap for 2 molecules on the 1 and 0.
I then just grow my backups into yeast cells