As your photography archive grows, so does the need to handle and protect that data. What happens if your computer doesn’t boot, or an image file won’t open? What if your home or studio gets robbed, or worse, catches fire? What if your backup drive fails, or your laptop gets stolen? These are all questions I ask myself when planning my backup strategy.
I’m a photographer, not an IT professional. Everything in this article is from my own experience since I switched to digital in 2001. I use a Mac, so if you’re on a PC you may need to do a little research to find the equivalent to some of the steps I’m going to talk about here.
Bootable Backup
Many people think that if they use Mac OS X’s Time Machine to backup their computer they’re protected if a drive doesn’t boot. But Time Machine doesn’t make bootable backups. For backups to be restored from TM, you need to have a working OS. The solution is to create a bootable drive which would allow you to hold down Option+C during boot, select the external (bootable) drive as your start-up disk, and you’re up and running. There are several pieces of software (both PC & Mac) that will allow you to create bootable backup drives, but my favorite is Carbon Copy Cloner. Trying to resolve technical issues and restore operating systems with a client standing in your studio can be a stressful situation. By creating a bootable backup, you can get back in the game quickly and replace/restore your main drive when time allows.
LaCie Rugged External Drives
Copy to second card during capture
Now that we have a solution to our bootable drive backup, we need to start thinking about protecting our images from the moment we create them. Most DSLR cameras these days have two card slots and will allow the user to setup how those slots are configured. While setting your camera to fill one card at a time so you don’t have to change cards as often may seem like a good idea, it doesn’t provide any protection of your data if something goes wrong. If you want to incorporate redundancy into your workflow, this is a good place to start. By setting up your camera to write both cards at the same time, you’re getting two copies of your data in case one card fails.
Shooting Tethered
When shooting tethered, you need to make sure your files are backed up while you’re shooting since you’re probably not writing to the memory cards. Once again, with an application like Carbon Copy Cloner, you can create “tasks” to automatically copy files, folders, or entire drives to a second location. I have modified my MacBook Pro with a DataDoubler from OtherWorldComputing allowing me to replace the internal optical drive with a second hard drive (or in my case a second SSD). I have created a task in CCC which automatically copies my “Tethered” folder from Drive A to Drive B every 30 minutes. This way if my main drive fails, I know I have a second copy on Drive B. Alternatively, if you don’t have the ability to replace your optical drive, you could just plug in an external drive and have CCC do the same thing.
OWC's Data Doubler w/Hard Drive
Copy To Second Location on Import
Most cataloging apps, like Adobe Lightroom or Capture One, allow you to define a second location to store images during import. This way when you ingest images from your memory card, or during tethered capture, you can create a backup of your images automatically. Just don’t make the second location on the same drive as the original location.
CarbonCopyCloner setup to copy "Tethered" folder
Where To Store Files?
Although they can be stored on the local drive, storing your image files on a remote drive or networked drive is likely a better option. If you have large catalogs of images, you may need a RAID system in order to create a volume large enough to accommodate all your files. By getting them off the main hard drive of your computer, it will allow your computer run faster and enable you to setup a backup system that’s accessible from any computer and expandable to fit your needs.
Redundant Array of Independent Disks (RAID)
There are several variations of RAID available, but the four most common are:
- RAID 0 = No redundancy (max size)
- RAID 1 = Mirrored (50% total size)
- RAID 5 = Spanned across multiple drives (60–80% of total capacity), 1 drive can fail
- RAID 6 = Same as RAID 5 but 2 drives can fail
I won’t dive deep into RAID types in this article, there’s plenty of information around the internet to bring you up to speed on how it all works. I’ll leave that to the IT professionals, but the biggest advantages of RAID systems are their ability to survive limited hardware failure, and their virtually unlimited size.
3–2–1 Strategy
Now that we have our data on a dedicated external drive we need to think about how protected that data is. IT professionals have long promoted a 3–2–1 backup strategy for secure, redundant backup of your data, and it certainly applies here. 3 backups of your data, on 2 types of media, 1 of those off-site.
Putting It All Together
Hang on, this is where it gets complicated. You can probably store all your files on a single external storage device, whether that’s a 2TB drive, 4TB drive, or 24TB RAID system. Look at how much you shot in the last few years on average and figure out how big of a drive you need to store several years worth of data. Maybe you can fit it all on a single drive, maybe you need a massive RAID system. Either-way, that’s the main storage for your files, so this is where you want to spend your money. But remember that RAID systems only protect against hard drive failure. They don’t necessarily protect against file corruption/deletion. In order to do that you need to have a backup of your primary files. If you're thinking RAID 1 accomplishes that, not quite. RAID 1 is an exact mirror between drive A & B. The problem is that information is instantly copied from one drive to the other. If something goes wrong on Drive A, it also goes wrong on Drive B. We need to create an opportunity to restore corrupted data from the uncorrupted copy. This is where a second drive (whether it’s a single disk or a RAID system) comes into play. By using an automated application like Carbon Copy Cloner to make daily backups of your main drive, you’ll have a second drive to restore from in the event something goes nutty on the main drive.
G-Technology G-Raid Thunderbolt
Now we have 2 copies of our data (you still with me?). We need a third to complete the first part of our 3–2–1 backup strategy. We have a couple options here. For our third copy, we need to be on a second type of media (the “2” in our 3–2–1 strategy). In Ancient Greece they used CDs and DVDs for this, but in today’s digital photography world, those methods are slow and small. Not to mention that optical media fades over years. Instead of optical media, we can use “The Cloud”. Companies like Backblaze and Crashplan allow you to store unlimited amounts of data on their servers. It’s a great way to go and covers the "2 types of media" and "1 off-site" parts of our 3-2-1 strategy. There is a downside though, it’s slow to start and slow to recover from. This is where I cheat a little on our 3–2–1 strategy. In addition to creating my third media copy on a second type of media, I also use a third drive to take off-site. Confused? Let me break it down for you:
Drive A copies to Drive B every night at midnight. This gives me a full 24 hours to recover any corrupt or deleted files. Each week I bring in Drive C from off-site and replace Drive B (taking Drive B off-site). Guess what app I use to accomplish this? CarbonCopyCloner automatically runs a task to copy everything on Drive A to Drive B or Drive C when it’s reconnected … so all I have to do is unplug Drive B, plug in Drive C and it starts to copy anything that’s changed since the last backup. Can it get any easier?
Now I have a 4–2–1 backup strategy, even better!
The Gear I Use
I use three G-RAID Thunderbolt drives from G-Technology. I also use a G-Drive Mini as my Time Machine backup (keep in mind TM likes 2x size of your main drive, my main drive is a 240GB SSD). I also have LaCie Rugged drives (various sizes) which come in both HDD and SSD versions to create copies of our tethered folder in the field. Even though we have two copies on the dual internal drives, we want to be protected in case something happens to our laptop on the way back to the studio. This drive is stored/carried separately from the laptop by my assistant. The next piece of gear we purchase, that will make this a truly bulletproof backup, is a Network Attached Storage system that can automatically sync with a duplicate NAS system off-site. This will solve the issue of something happening to the two G-Raid drives in the studio and losing everything from the past week that hasn't been copied to the off-site rotating backup. We're reviewing a few models now and will update once we've made a decision and incorporated into our backup strategy.
G-Technology G-Drive Mini
A Few Closing Thoughts
Isn’t all this expensive? Sure, but isn’t losing all your data, or even data from one job (a wedding for example) more expensive? Buy good stuff, spend the money once, spend the money now. Don’t buy cheap stuff and have it fail, you might as well not do anything. Don’t establish a backup strategy AFTER you have a problem. I don't expect everyone to run out and buy a few thousand dollars worth of RAID drives, but hopefully I've given you the incentive to start thinking about how your data is protected. Good luck, and let me know in the comments if you have any questions.
RAID 0 provides no ability to survive any hardware failure. In fact, it doubles your chance of having a hardware failure and it'll be near impossible to perform any data recovery. The only thing it's good for is to boost your IOPS.
Exactly Tam. That's why we have 3 of them backing one up to the others. We use it in RAID 0 so that we can leverage the full 8TB capacity and treat each of them as a single drive. We don't delete anything off the internal drive (tethered folder) until all the data is safely on two of the RAID 0 drives.
With any raid setup, please keep in mind that you have power failure during writing, you'll lose all data because it keeps the data in a buffer that loses its integrity if power is lost. If your raid controller borks, you'll need to find exact same hardware to rebuild the data. All raid drive backup solutions should also have battery backup along with it. These are just some good practice steps.
Not all RAID setups. Spring for a decent RAID controller with a built-in battery and it will automatically flush the buffered data in the event of a power failure.
Valuable point, thanks for filling in the hole I left. This is important and people need to consider this.
You guys probably noticed my Pure Sinewave UPS in the video. We've got everything, including our drives, running through that too. Good point I probably should have mentioned. It's a fine line between writing to mass readers and getting too technical.
Also, I'd like to point out that your nightly backup operation (and also weekly) is straight up cloning, so it has no ability to recover from a file corruption. Cloning means whatever happens to the source, happens to the destination.
I'm a CrashPlan user myself, so I use the same software to copy my stuff to a local drive. CrashPlan has the ability to use an external drive as a destination, and it uses block-level backup for both incremental and differential backups, so you have the ability to recover a file from any point in time (within your backups) even if the file is corrupt.
Of course, the drawback with block-level backup is that it really doesn't delete files that are originally deleted from the source UNTIL they're out of the retention period. This means that if, for example, you whack a giant folder of say, 500 GB, you won't gain that 500 GB until after your oldest backup is removed.
Yea, we know it's cloning, that's why we wait until midnight to do it. If something happens to that data during the day, we can still recover it from the second drive (or tethered folder on the capture drive) before it writes a clone of the corrupted data {we've had to do this before). I like the idea of using CrashPlan to write to an external drive and being able to recover from any point in time. It sounds sort of like Time Machine, but larger scale. I'll check it out . . . thanks!!!
Exceptional info....I wish I had taken this advice in my first year as a photographer, as most all of that work is gone due to a hard drive failure.
Heed this advice to save yourself the headache later.
I have my files back up to a RAID5 storage system on my computer, Crashplan (which is amazing BTW) then goes through and saves everything on that RAID system to the cloud. Fortunately, I've never had to use Crashplan for anything other than backing up, but I'm confident they'll be able to recover my files if something awful happened.
Does anyone know if ChronoSync detects (or can be set to detect) a corrupt drive before running its sync?
I recently was in exactly this business. This is what IT professionals refer to as DR/BC - Disaster Recovery/Business Continuity. What you have to ask yourself is, what do I need to keep my business running? And, what do I need, at a minimum, to recover from a complete disaster? As an example of "disaster", there's the foresightful IT guy who grabbed all the backup tapes that he could when he was exiting from one of the WTC towers on 9/11. That business continued afterward because of him. Others did not. Having off-site backup of otherwise irreplaceable data is a must, and you must regularly update those off-site backups.
Recovering from accidentally deleting files or file corruption is really at worst annoying by comparison. What can kill your business is being unable to know what projects you had going, with whom, and how much you are owed for them. Backup of your images is important, yes, but so is backup of your accounting information.
If you are running a photography business, and employ more than a couple people, you really need to consult a professional with certifications (CISSP should cover it, and so should GSEC) to see if you are doing things right. They can recommend data storage services (or provide them, but be careful), set up a backup strategy, and recommend solutions.
Now, speaking as an individual, I have lost images with disk failures. Never again, I hope. Personally, I employ multiple RAID solutions, backup onto separate RAID devices, and periodically backup onto more permanent media (yes, I use Blu-Ray DVDs for this - no, they may not last 100 years but two or three will do nicely). There are lots of tools to do this, and they all work fairly well. I've been using Acronis True Image products for several years (yes, I'm a Windows guy when I'm not using a Linux flavor) and they've been pretty good for me.
Your needs may be different. You're a professional. Consult another one. Ask around in your area. Get bids. Check with their clients. You know the drill.
I have a Drobo Mini, Drobo 5D and some Windows Storage Spaces used for all my photos, and yet still use the Cloud for backup. I use both CrashPlan and Back Blaze. Given this is your livelyhood, both CrashPlan and Back Blaze are cheap, and you could easily use both. Both give trials, IIRC, and both would cost you about $200 a year total (BackBlaze is $60 for one machine, I think Crash Plan, for the number of machines i backup (up to 10) is about $150 a year). As an IT guy first and photographer second, i know there are 3 things definate in Life... Death, Taxes and Hard Drive failure... SSDs, HDDs, RAID, Etc, all die... Have as many copies as you can, in as many different locations as you can. And, when you say loosing a week of work sucks, how much would that cost you?
Thanks for the info!
I use Backblaze myself. I love their iPhone app, makes it so easy to access photos remotely quickly right on my iPhone.
Thanks for the info, Tony. Two things to add from my experience in IT and IT Audit: (a) check for any errors in your backup app logs, and (b) test your backups periodically to ensure they work. You don't want to wait til you need to restore to find out the backup media doesn't work or got corrupted, or you didn't configure the backup app (e.g. Carbon Copy Cloner) correctly (so some precious files you presumed were included were omitted) or that your online service provider (e.g. for CrashPlan) has some peculiar steps that may cause an incomplete restore. You don't need to test after every backup (but it's not a bad idea to at least try to restore a few files/photos) but at least once a year install an empty drive, pretend you had a system failure and try to do a full restore to that blank drive. Use that full test as an opportunity to note/update your restoration steps (doesn't need to be a book - you just need enough so that when you're in the heat of a real disaster and confused by emotions, you don't need to rely on memory) and store that restoration guide with your offsite backups.
If you're in business and can't afford to faff around during a disaster, it is worth hiring an IT security professional to check your backup process (as Shane Castle noted). Don't just take advice from fellow photographers (as nice as they are) - unless they happen to also be IT security pros...
Tony, I love your backup strategies. A really good way of cloud or offsite backups is to send your work via FTP to a backup drive that is off site. For instance every night or how often you chose, software (sorry, not sure of the name its a FTP uploader) will compare your local drive to the FTP location and only upload the file and folders that have changed (I know it has other options also, and the software is only about $40). The best location for the offsite drive is your home or a friend or relatives. Then you would have to worry about toting a drive back and forth from the office and you would only lose a day if all your backups fail.
That was a lot of information to absorb! I know I'll have to revisit this article again, but this was great information!
I'm really appreciating your articles thus far. It's so nice to take a break from the latest camera gadget and "how I shot this" type of articles (not that those can't be fun either) and get down to the nitty gritty of running a photo business.
Thanks Tony!
Much appreciated Brandy
Great article Tony! Backing up our media (life) is one of the most important things you can do. I just bought the G-RAID Studio 6tb drive. Good to have for 4K media and the drive itself is set up very well with hardware RAID, FAST! Also have a Drobo and 2 4tb Hitachi Touros for on-site backups and use more portable drives for off-site backups. Spend the money on it now and save yourself later! They will fail at some point. I have used Backblaze, crash plan and carbonate. They are just so darn slow and seems like it would take a year to backup everything. Sticking with the physical off-site backups for now until there is faster bandwidth out there. Interested to hear what you find out with the NAS system and will stay tuned for the update. Thanks for the article.
Nice article Tony but I wonder why you don't use tape backups in addition to your RAID/cloud strategy? I am an IT professional who also shoots and while storing images on a RAID array or the cloud works, there is nothing like having a set of backup tapes, especially off-site tapes to cover yourself.
Thanks Robert, my assistant keep pushing for tapes, evidently theres a newer (fast) version out that's pretty solid. Guess we'll have to look into it more seriously.
Thanks for the response Tony. Is there a reason why speed would be a factor? If you can backup to more than one array, you could backup the second copy of your data to tape when the data is not being accessed and it should be pretty fast.
Not particularly, but all things being equal, If fast is an option, I'll take it. Obviously, safe first, then fast.
what conditioner do you use Tony? That mane is impressive!!!
This was obviously written as a sales pitch for LaCie and G-Tech.
While G-Tech's are amazing!!!! LaCIes are shit. I work with hard drives everyday as a Digital Image Technician for Photographers & Cinematographers alike. On the side I recover drives for clients and other Technicians. More often the not the drives I have to recover are LaCie Rugged drives. The design is rubbish, the USB 3.0 ports break all the time, the drives over heat and eventually stop working all together. As a reseller for IT & Film consumables and software I have stopped selling LaCie altogether as it cost me so much time and effort to be continuously recovering their drives after sale or replacing them.
PLEASE WHATEVER YOU DO, HEED MY WARNING!! DONT LISTEN TO THE SALES HYPE ON THE NET IT WILL END UP COSTING YOU A CLIENT OR YOUR CAREER.
Thanks for your input Wezley. Not a sales pitch for LaCie or G-Tech, just a rundown of my current setup. Sorry you've had poor experiences with LaCie. Hopefully I don't have the same issues.
I've had 4 LaCie drives fail on my now ... thank god I'm ridiculously obsessive with backups.
You guys are starting to make me question the continued use of my LaCie drives.
The PLAN, for me, is to run 4TB drives (one at a time), internally on my desktop workstation. Then, working off of 1TB 2.5 externals (one at a time) between my desktop and laptop. When the 1's fill, they get archived off location. When the 4 fills, another is added. This will protect them at another location while allowing you to have access at home, should you need to go back that far. All finalized client images will be stored in the cloud as well. Note: the 4TB is storage only. I use a SSD as a applications drive and a high speed WD HDD as the secondary working drive.
The idea in the article is nice but it's still pricey. Good if you're succeeding well but if you're still building your foundation, it's not practical.
Your right Mihael, it's pricey but has worked for me so far. Obviously I didn't start with this setup initially and worked up to it through necessity.
Sorry, didn't mean that to come across harsh.
I also agree that it's not a good idea to sacrifice quality for price in this area so I'd like to add by suggesting to everyone to hang on to your cash until after Christmas, because we all know this kind of stuff drops dramatically then.
I didn't take it that way at all Mihael. :-)
It's true that a good backup strategy is pricey but it's less pricey than being sued by your client. :p
Thanks for making this video, Tony. It was very informative. I have to tell you that you guys have a great amount of discipline that makes this work for you(which is commendable).
It's kinda tricky for us though, I work in a wedding Photo + Cinema team. We are 8 Photo guys and 8 Cinema guys. The biggest set back for us to follow this would be the volume. We'd have to scale this up 8 times at least(I'm not even going to consider the discipline involved). Nevertheless if I decide to migrate in the future, this is a very approachable workflow.
Kalyan, in your case, you could use something similar in regards to strategy, but with larger RAID volumes to handle your massive amount of data. I think the key is allowing software to create the backups for you automatically, removing the human element of error (forgetting to backup). Hopefully the video has made people at least think a little harder about how well their data is protected, regardless if they find a use for my processes.
I am an IT person and I realized that my backup strategy of USB drives had some serious flaws. I purchases a Synology 412+ as my core storage drive (RAID-5). I find that the performance loss of using a network drive opposed to the SSD on my machine is worth the security that my photos are secure. I also have a dedicated Synology 212j which I use for backups on a daily basis and weekly I push to a USB drive. I do rely on TimeMachine for to-the-minute recovery, but I have monthly image copies using SuperDuper.
In the field we take a copy to a dedicated USB drive.
Its a pretty important topic, often ignored and I appreciate your article...lots of good tips.
I'm small potatoes compared to many here (only do about 10-12 weddings a year + a handful (another 10-15) of boudoir, maternity, product ... shoots).
I have my working copy on my workstation, 2 copies on 2 external USB hard drives and I used to have another copy on a NAS but the NAS is old and out of space (and can't support larger drives) so I am retiring it (using it for my family photos and the likes ... nothing critical) and going to replace it with a new one next year.
I will also be adding a third external HDD that will be stored off site in rotation with the other 2 drives.,
The 2 internal drives on my workstation are set up as raid 1.
My contracts, invoices, model releases, mailing lists ... and all that stuff is stored in Dropbox as well as on my drives.
This has served me well over the past 5-6 years.
Nice write up. I use a combination of 2 hard drives in the laptop, time machine and 2 x lacie network storage boxes. Have a blog about it here: http://www.richie.ie/articles/photography-workflow-data-backup/
It is a well written article made for the general audience, and that is important. I've has my share of crashes in the last 15 years. Without some manner of a backup is tantamount to vying for the Darwin award.
Hey Tony, nice round up of your backup workflow. Out of curiosity how did you get CCC to make backups every half an hour? The lowest I can choose of is every full hour.
Daniel, I was waiting for someone to ask me. I simply made two tasks, one starting on the hour, the other on the half-hour.
Allright that way it makes sense. I'll stick to DropSync for onlocation backup, especially for video and copy whole cards. I can set this to seconds, minutes, hour or when a change occurs in the source folder.
If you are using a ZFS file system like in FreeNAS RAID servers then data corruption is not an issue. Your setup is fairly similar to mine. I shoot to an SD card and dump that card using Photo Mechanic to two different drives/locations, the primary and a secondary backup (which is cleared out after a few weeks). My main drive is an SSD but my photo storage is primarily on a Drobo so I can expand the space as needed and have some hardware protection.
Weekly, my Drobo and other drives are mirrored using CCC to an on-site FreeNAS RAID6 server I built. Monthly, a copy of my photo and video work is made to an external drive and brought off site and exchanged for the month-old backup copy. Worst case scenario for me is that I lose a weeks worth of work due to a significant hardware failure, or a months worth of work due to a fire/theft. As my photography is only part time, this is currently acceptable. As soon as I find a reasonable storage plan I will move to cloud backups.
File management and backups are not very glamorous, but if you dont have a good system now, its only going to get exponentially worse in the future.
I use the poor mans version of this. Basically I have a couple of hard drive docks for internal drives. and uce both time machine and carbon copy cloner for triple backup.
much cheaper to buy hard drive docks and internal drives. You can even get docks that you can fit two drives in. less cables.
Have you ever considered using Hazel to handle that tethered copy so you don't have to run on 30 minute intervals? http://www.noodlesoft.com/hazel.php And no, I don't work for them!
Hey Joseph, I use Hazel for a lot of things, in fact I have a post about it coming up soon. But CCC is automated as well, so not sure what benefit Hazel could offer in this situation. Perhaps you have something in mind?
Sure - in the video, you said that as you shoot tethered, you have CCC run every 30 minutes to make a second copy. CCC has to scan the entire folder to get the differences. Hazel should be able to do that near real time, with less an impact than the every 30 minute CCC run.
Can you elaborate on how you'd setup Hazel to handle this task?
Backblaze deletes all data, if hard drive isn't connected for over 30 days. you want to choose Crash Plan Pro. Thanks for all the tips! I'm glad I do a lot of what you do, with the same equipment!
Backup is one the biggest issue nowadays but i have been using cloudbacko software from long time and quite satisfied with it. As it has got two software one is Lite and the other is Pro . Just go and check it out their web site they have 30 days free trial period it can give backup of both photos and videos. Bulk of backup can also be done through this software so people just give it a try. http://www.cloudbacko.com/
As far as cloud backup goes, I prefer CrashPlan over Backblaze, Backblaze is fine, but CrashPlan is much more configurable. I generally like your setup, but I would definitely add cloud backup to your Studio A and/or Studio B drive for real-time or scheduled offsite backup.