"Trevor, I have been editing this wedding for the last few days and now my Lightroom catalog says it is corrupted! When I opened a new catalog and reimported the photos from my hard disk I can't find any of my previous edits I worked so hard on. I am so frustrated! What did I do wrong?" Keep this from happening to you by having Lightroom automatically save your changes, but this setting is off by default. I'll show you where to turn it on.
If this hasn't already happened to you, knock on wood because surely it will. Seems like every week I see another photographer post desperately in a Facebook group about having lost all their recent edits because of a corrupted or lost Lightroom catalog. Often this means that hours of work, maybe even days or weeks is lost.
However, this should never be the case if they just turn on one simple setting in Lightroom. Sadly, this setting is off by default. The reason the team at Adobe has it off by default is because at times, depending on your system and how many files you are working on at one time, it can slow things down a smidge. But these days our computers keep getting more and more powerful and often we will not even notice any difference in speed.
You will find the setting in the Catalog Settings of Lightroom.
MAC > Lightroom > Catalog Settings > Metadata
PC > Edit > Catalog Settings > Metadata
The important setting that I would recommend having checked is "Automatically write changes to XMP."
By having this button checked you are telling Lightroom to constantly be "saving" the changes you make to your images. If you are working on proprietary RAW files such as .CR2 or .NEF files you will see a .XMP sidecar file created alongside your image file in its folder. That XMP file contains all your changes you have made to the file in Lightroom. Which means if your Lightroom catalog gets corrupted, lost or for whatever reason is not working right all you will need to do is reimport the images into a new catalog and Lightroom will automatically read those XMP files in the same folder and apply those changes immediately to the files. Then you can wipe those tears away knowing all your hard work wasn't lost.
If you are working with DNG, JPEG, TIFF, or PNG files the XMP information will be stored directly inside your files so you will not see the accompanying XMP file in the folder, but if you were to send those files to someone and they opened them in Lightroom they would see the edited version of the image. So again, if you had to reimport the DNG files into a new Lightroom catalog because the old one got corrupted - no worries, no tears, your work is all there and up to date.
Keep in mind though that the history of your edits is not stored in the XMP file which is one of the benefits of the Lightroom catalog.
So if you don't already have that box checked in the Lightroom Catalog Settings, stop what you are doing and go turn it on now. Do some editing and see if you notice it slowing your computer down. Hopefully it does not. If so, the other option you have is keep the setting off but every few hours when you want to take a break from editing hit the keyboard shortcut, Ctrl+S to save the metadata to files. That will do the same thing, it just won't be autosaving as you work.
If this was helpful please share so we can save the world of angry and depressed photographers who have worked for days only to find out a Lightroom corrupted catalog has lost all their work. Cheers!
UPDATE: For those more interested in learning a good technique on how to back up Lightroom Catalogs, check out this article and accompanying video on Fstoppers, "Good Technique for Backing Up a Lightroom Catalog."
I think this is a great reminder about an awesome setting Lightroom has. Anything to keep our work safe is outstanding advice. I personally have several redundant backups set up for all of my work (Lightroom catalogs included), but this is just a smart practice.
Thanks Trevor!
Thanks Steven.
The software shouldnt corrupt in the first place...
But if it does YOU lost your work and YOU have to do it again and YOU have to tell your clients why they get their images later than expected.
So yes, the software should not fail. But it is you who has a problem if you just trust the software, unprepared for the case it does fail anyway.
Corrupted catalogs are very rare in my experience, but you still want to be backed up in case of disk failure or loss.
I can confirm this from my experience as well. I am using Lightroom for about 3 years now and jumped from 3 to 4 to 5 with several different catalogs. Never experienced any problems with my catalogs. But sure enough I will be glad to have backups if it ever fails somewhen in the future :)
this is also useful if you want to transport edits from one computer to another (though exporting a catalog is a better way of doing this)
Good to know Trevor. Thanks
Great advice. I started doing this about a month ago with a catalog with 27,000 photos and have seen no slow down. Let me say my catalog is on extremely fast SSD and photos are on fast thunderbolt RAID. My system is 2011 Mac Mini 16gb ram so not a Hugh power house and editing 5d MIII RAW files. Also if you edit a file in another app that wrights to the sidecar and go back into Lightroom it updates with the changes in about 1/2 second.
Import as DNG into Lightroom in the first place. Problem solved.
I love DNG when importing with Lightroom. I use Photomechanic to import (ingest) the photos so I can do multiple cards at one time and use it's amazing speed to cull through the photos. Sadly DNG is not an option to import there. But hopefully in future versions they change that.
Importing as DNG is fine, but keep in mind that you must also have the Automatically write changes into XMP option checked if you want Lightroom to write changes back to your photos (in addition to the catalog). People have been confused on this point this in the past, so I just wanted to clarify.
Another solution would be to activate the option to create a backup of your catalog every time you close Lightroom. This is the way I make sure nothing gets lost. Also I do most of the editing in Photoshop and only organize the photos in LR. So most (if not all) editing information is inside of PSD files anyway. What I do in LR is WB and maybe creating virtual copys for having multiple crop versions of images if necessary.
I understand the suggested solution may work for some people. For my way of working with LR, this wouldn't give me any big advantages but therefor slowdown my workflow and increase backup size.
Just saying that this must not be the way to go for everybody :)
Yes, this is the intended workflow, and why that setting can be safely left unchecked by default. In addition the catalog stores more data than can be written to XMP. Trevor noted that individual history steps cannot be written to XMP, but the same is true of regular collection membership, flag status, and virtual copies.
Again this still doesn't protect you if your catalog gets corrupted during your multi-hour editing session. I don't know if this happens but if it did then you never got a chance to back it up. This IS what I do BTW backup every time and then before I turn on the machine, backup again (sync actually) to external drive. But mid-session corruption, if it happens, wont be helped by that.
Yep, I agree that would be a serious problem. But as I mentioned, all I do in LR is white balance and very basic adjustments. Anything else is done in PS. So if my catalog gets corrupted, it might lose these basic adjustments in the original DNG files. But then again, the edited PSD files already include these adjustments and are not lost.
So yes, if you do multi-hour editing sessions in LR, saving to XMP might be a good way to avoid massive loss of data, time and hair ;)
I think this is also not the best choice, this impacts to your performance and already, Lightroom isn't the best optimized software. Why not backup your catalog every day? Lightroom can do this as you exit and I do this. If my Catalog corrupts, just import the backup and voila. No more issues and it won't affect to your performance. Backups people!!
So you don't save out your LR changes at all? You never go back and want to re-work an image? Or if you do you always start from scratch with the RAW file again? You never need the history of previous work?
That isn't what Apollo is saying though. If you simply back up your catalog regularly then you have all of your important work safeguarded. All of your history is stored in the catalog. You can then certainly go back and re-work a photo or start over from scratch as it is all there in the catalog.
Sorry, yes. 'History' was a poor choice of words. The History is saved in the catalogue - and only in the catalogue - not in the metadata. The actual adjustments/settings using the tools is what gets saved to the metadata.
I'm not suggesting, nor is anyone else, that backing up the catalogue is a bad idea. It's not. A sound back up strategy - and what that looks like will differ depending on the individual, but that has at least one level of redundancy - is always advisable. My LR catalogue gets backed up to a separate drive from the 4 drives that images and the master catalogue reside on, then gets backed up to an NAS configured with a RAID 5 array. Optimal? For me, yes. For others maybe not.
But in the case where the photographer isn't always working at the same computer, may want/need to do some editing outside the office/studio, where an initial edit made remotely may need to be changed upon reflection, where images are shared with others working in a different location, where there's a workflow that is not isolated to LR, to note just a few possibilities, a simple catalogue backup isn't sufficient.
Working with multiple computers, working in a multi-user environment, and so on, are all different workflows and not at all about the original intention of this blog post (using auto write to XMP to safeguard work against catalog failure/loss) or Apollo's question (doesn't backing up the catalog safeguard your work too). Though I grant the cases you raise are tangentially related to writing changes to XMP. However, each case would deserve its own blog post to discuss the whys, whens, and hows. I'm afraid it has the potential to confuse the issue at hand to bring them up here out of context.
I do agree with you about backing up the catalog though. :)
We'll have to agree to disagree, Rob. :-) I don't think it is merely a tangential relationship. I think it is a complex set of considerations that merely checking or unchecking a box or backing up the catalogue isn't going to provide the single answer to. There is no single answer. That, really, is the heart of the issue. I think you, Jeffrey and I can all agree on that.
I set my backup catalog to "every time you exit" and usually I let it unless I know I just got in to look and didn't change anything. I back that up to an external drive every time it finishes exiting from Lightroom. Now I got an extra external drive so I can have an offiste backup too but first backup is taking awhile. I've investigated but not implemented cloud backup because for the amount of stuff I have to backup it would take forever (though some let you seed that by sending them an external drive and a few hundred bucks). BUT what if I open up Lightroom, do a bunch of edits and before I can exit and save my catalog it gets corrupted? That might be when I wish I had written things out to the XMP or other format files, no? If I did one big mega hour editing session? Me, I'm paranoid enough to stop and get out periodically and backup the catalog, but I could see it happening. Fingers crossed and all that. Yes, this all takes a lot of time BTdubs, but luckily there are other things to attend to while stuff is getting backed up.
Sounds like my workflow. On laptop, it's every week but on home PC it's every time I exit.
One area I've just discovered that this can slow things down hugely is if doing a library re-org.
I recently purchased the new DAM book (which has massively helped my rapidly un-manageable way of working with libraries on different disks etc.) but I noticed that after tidying keywords etc. the fan on the laptop was abnormally (for me) loud. Eventually I traced it to having switched on this setting at some time in the past which was causing thousands of xmp files to be re-written just by making keyword changes.
For now I've turned it off again, not sure I'd bother turning it back on as I also now have very regular backups of the catalog & have it set to back-up every time I exit Lightroom. If I haven't made any changes worth saving (say, I was just looking for a file or wanted to do a quick export to JPG) then I simply tell it not to bother doing a backup on those occasions...
Seems like a solid article but do I really need to do this if I backup all my catalogs with content each evening to another drive?
I think I spent around $20 for Allway sync which simply runs each night or I run it manually after doing large chunks of work...was worth every penny.
No, you don't need to do it. If you are backing up your catalog regularly (and have a full system backup in place) then this is more of a belt-and-suspenders practice. Also, as has been pointed out, you can manually write changes to photos at any time by selecting them and pressing CMD/CTRL+S.
wow I tried this and my whole CPU melted, not only that but it erased my entire history from 1976-1994, gave me cancer and caused my unborn kids to come out with 2 heads. THANKS A LOT.
Mission accomplished!
That explains what happened when my catalog stopped working. Re-importing hasn't affected me greatly, only meaning that my files aren't ready when I come looking for an older image. Thanks for the information, I'll monitor the performance impact closely in coming shoots.
It can slow things down a LOT for those of us with huge catalogs. A "smidge" might be undetectable for someone with a 1000 images, but for someone with a hundred thousand images, it renders LR unusable (at any CPU speed). This said, I've yet to have a LR catalog corrupt, and I'm dealing with thousands of images each day. (And, in the meantime, I'm sorry I checked the box. It's going to be a long wait for those .xmp files.)
So, in support of Jeffery's comment regarding "bad advice". You should have a caveat at the top of this page: DO NOT DO THIS UNLESS YOU (a) HAVE PROBLEMS WITH CORRUPT LIGHTROOM DATABASES and (b) HAVE A RELATIVELY SMALL CATALOG.
It really shouldn't matter the size of the catalog unless you are making a change to one photo and sync'ing that change across all the images. Only then would it slow things down as it goes about writing that new data into a XMP.
"Should" or "shouldn't" is often different than reality. I've verified it on a 160,000 image catalog. If I go in, turn it on, it then attempts to create .xmp files retroactively and plays havoc with LR 5.3's performance (which LR 5 is slow anyway.) Doing it on a 120 image catalog, no big deal. You still have to wait for it to write the .xmp files, but it doesn't drive your CPU usage to the max. Thus, there is a correlation between catalog size and having to wait for LR to do it's .xmp updates.
This not NOT the most important setting and it's off for reason. Saving out to XMP uses system resources and many don't need it. If your RAW files stay within LR and as long as you back up (which you should be doing anyways) there's no real need to have this on.
If however you move back and forth between LR and bridge or need the XMP data, then it's a good option. But it's also worth noting that you can manually select and CMD+S to save the XMP occasionally as needed.
We should be backing up catalogs. But I have helped people recover from corrupted catalogs by importing the data from the corrupted catalog into a new catalog. Also worth a try if you end up in that situation. But again, just backup. Always.
So yes, valid tip. But certainly not essential. I've know it's been there since V1 and I don't think I've ever turned it on ;)
Gav - this article is about one particular feature on Lightroom. It is not a comprehensive guide about a whole Lightroom workflow. I think an article on catalog backup would be great for the future so I will definitely share mine in the near future for those that are trying to learn a system.
Thank you for mentioning the CMD+S feature. It is something I discussed in the video as well as an alternative to those experiencing any kind of slow down.
OK, the comments are ridiculous. The article, speaks from the broken heart. Lightroom uses modified version of a MySQL database for its catalog file. It seems that as of version 5, the underlying database has been updates as I've seen a significant performance and durability improvements. If your catalog gets corrupted, LR will run the most basic recovery process. If its still corrupted, you got two options - go talk to a techie near you to see if they can do their magic or go talk to a mysql db admin. MySQL has exceptionally stable database and has improved tremendously as of late. The chances of LR crashing completely are quite small now.
As for having a sidecar file, its not a bad idea however it comes with ramifications. Remember that you're editing files pretty fast, you know shortcuts and your mouse movements are done with intent, not to experiment. My guess is that LR creates a brand new XML file for after each modification, there is no other way to keep your changes safe. This entails lots of small writes to the disk. To read what had been done in the Gallery mode, LR has to open file, read, process, close 100s times over. People - HDs are bad at doing that with small files! Its irrelevant how fast your HD is or how new your computer is. Its going to be slow due to the nature of it. You only get 3 to 6 Gbps speeds. That's 374MBps to 768MBps of bandwidth. The actual read speed of HD is much less than that, plus it has to spin to correct place, so its even slower than that. Reading your raw is probably faster than that XMP.
Then you have to use up that CPU of yours to process XML. You'd be surprised how slow that is. I mean, its fast, but dog slow nonetheless. LR doesn't use any modern CPU features or your GPU at all. Its really bad, but they don't do that. So XML is slow. I wouldn't be surprised they use one core per image when reading in gallery mode.
Best is to use daily backups. Get a software that will do live copy of the file while you're working on it. Yes it blows to have corrupt catalog, but having 1000s of smaller files is worth than having one giant DB.
Its SQLite, not MySQL; its really effective, and if you know what you are doing, you may modify or read lightroom catalog from your scripts etc..
This is great advice. Having your hard work stored in two different formats is always better than one. I flip to Bridge and PS all the time from LR, and saving to xmp is the only way to your LR edits/flags/ratings. Anyone complaining about a major performance drop is exaggerating.
Great advice! Their is one thing I know about any database based application, they will break. Happens with Lightroom, the new capture one catalog, iphoto, aperture, itunes,....
Only a time machine or similar system can help you then. A normal back-up will also be corrupted. You're time machine will have a good version, but how far do you have to go back? Maybe before you made any adjustments....
Bottom line, I never trust my work to be safe in a database. Everything is organised in finder. Backed up with Carbon Copy Cloner every day and every hour with time machine.
Raws are selected with bridge and processed ACR.
If I get a corrupted xmp file I look in my back-ups and if those are all corrupted, I still have many other xmp files which hold most of the settings.(happened once, it got corrupted moving from my laptop to my server)
So after years of working like this, I lost 10 minutes due to a corrupt file.
Instead of often days you lose when the settings and selection of an entire shoot go bad.
From the perspective of someone who works as a Lightroom Help Desk support person I greatly appreciate your intent to save people from the frustration of lost work/time! I am also glad to see the vigorous discussion of catalog backup this post has sparked. :)
I personally don't use this option, but I do know of people who have benefitted from it, and people who would have benefitted from it had they turned it on. That said, the people who would have benefitted would have gotten even greater benefit if they had more regularly backed up the catalog file. I sense this is the crux of what has invigorated the folks who have taken issue with this post being titled "the most important setting in Lightroom." I would also respectfully disagree with that statement, but I do think the issue of safeguarding your work is absolutely the most important thing a Lightroom user can do, which I believe is your intention.
From my work providing support to people with this program I believe the first step in safeguarding your work is understanding exactly what the heck is going on when you use Lightroom, which means knowing that there is a catalog file that holds all of your work, knowing where that file is stored on your system, and really truly understanding the relationship between that catalog file and your photos. From this place you can take the correct steps to put redundant backup procedures into play, and understand the risk/rewards with each option. None of this is rocket science, but I believe Lightroom is easy enough to start using without knowing all (any) of this that many people don't take the time to learn it until a problem strikes. Thanks for trying to give people a preventative heads up. :)
So if I turn this on, then if I open these files in Capture One Pro 7 it would read the changes already made, right?
no.. because C1 is not LR.
but it would read EXIF, IPTC edits you made in LR.
This is SHOCKINGLY poor advise and I would tell everyone to complete ignore Trevor from this day on.
The performance lose, and damage to your harddrive for the constant writing of small changes to the file is just not worth it.
WHEN you need the XMP file, exported it as required. Otherwise just back correctly as you always should. But don't listen to Trevor.
Wish sites like this did some research before posting utter BS like this.
well the idi*t here is you..... get some clue.
your OS harddisk is writing small files all the time.
get some basic education about PC´s before you try to advice others.
well another way is to make a smart collection for images you have edited "today" and then select them all "strg+a" and save them "strg+s" at the end of the day.
not automatically and you can (in the worst case) still lose a days work.
but still better then not writing the data into the images.
and it saves system performance on slow systems.
girls.. if you are concerned about performance get an SSD.
there is absolutely no performace hit noticeable for me.
and before the noobs start yelling "SSD wear down with writes" .. get some infos about what we are speaking here.
with 100GB writes a day a SSD will still last longer then you will probably use it.
todays 120GB SSD do 100GB writes daily easily for 6-8 years. should be enough... or do you think you will use the old SSD in 8 years?
Hi, after I turned this option on, Lightroom (5.3 x64) became so f***g slow that it was redered useless. Every single operation takes ages to complete. Absolutely not a way to go. On the other hand I really is good to have the metadata stored elswhere than only in the lrcat. I wonder if going for DNGs instead of RAW files would be a good way to go in terms of image quality (should theoretically be the same as RAW), editability, and speed.
This is not good advice and it's certainly far from being the most important setting in Lightroom. Not only does it slow LR down, you can also accidently commit caption and kw changes to files that you may not want. It's always better to make decisions intentionally, not to let software do it for you. If you're doing this simply to avoid losing data due to your catalog crashing, that's a poor substitution for fixing whatever it is that's making your system so unstable.
This is also bad advice if you convert your raw to dng, which itself is terrible advice. DNG creates nothing but headaches and offers no benefit other than a small disk cost saving and the avoidance of sidecars, which are a triviality.
This is definitely bad advice when couched in such absolute terms as the "most important."
I am working with both Mac and Windows platform and saving small XMP files have been incredibly useful fro cross-platform collaboration. I don't have any idea what a poster was on about that a small XMP file can be an issue with bandwidth and file size. Its other useful feature is having several versions of an edit without increasing the file size (again, very useful in my workflow between platforms). Thank You.
Hi Trevor, I don't want to get into any debate over the best workflow and no I am not an expert on any of this. But I do want to point out that for someone who is NOT an expert and still trying to learn how Lightroom works, I found your article a little misleading. I see in the comments that you've pointed out that this is just one option for backing up your edits and not for everyone but calling it "the most important setting" and saying things like "If this hasn't already happened to you, knock on wood because surely it will." and "stop what you are doing and go turn it on now." is a bit dramatic and confusing for someone who is new to Lightroom. I first read this and thought oh no, I've been doing something seriously wrong. I had to read the comments to understand that simply backing up my catalog properly makes this scenario very unlikely and your solution should merely be a backup-backup method and not something everyone must do. I get that for you and some professionals this extra level of security is important and this is just your recommendation but I don't see how turning on this setting as a default makes sense for all Lightroom users like your article suggests. If I backup my catalog regularly and as carefully as I back up my original files I really have no need for an extra backup in XMP files. Nevermind whether it slows your computer down or not they are still extra files that need to be backed up again or uploaded to the cloud (which is a serious pain when you travel and have slow internet) and for someone like me it's an extra step I don't want or need. I totally get why you would want to do this as a professional but I just want to say please remember that not all of us are professionals and/or experts and presenting all of the options rather than giving blanket advice that probably shouldn't apply to everyone would be more helpful. Thanks.