Adventures with NAS: Switching from Using an External Hard Drive to a Network-Based Workflow

Adventures with NAS: Switching from Using an External Hard Drive to a Network-Based Workflow

In my journey to separate my family life from my work life, as detailed in my last post, a change in my work environment has been key. Namely, my wife was tired of seeing my hard drive sitting on the kitchen island and I was handily kicked down to the basement. I took this opportunity to switch up my workflow from using a local external hard drive to a NAS (network-attached storage). Here are some interesting things I've discovered along the way.

First and foremost, I have to acknowledge: I've been pretty dumb about storage.

I had no real backup. All of my files were hanging out on a 6 TB LaCie Thunderbolt drive in RAID 0. Dumb. If something were to happen to that drive I would have been royally screwed. This is totally not a knock on the drive itself. It was a beast. It had super fast read and write and I had no complaints. I also had no redundancy with my data.

When I moved my work area to the basement, I didn't want to bring the same bad habits with me. I decided to do some research into NAS drives. Now, I've built many computer systems in my day, but when it comes to networking I'm a rank amateur. As such, I wanted a drive that would simply work. I didn't want to have to fool around with too many settings and I wanted backup of my shoot files to be automatic. I ended up purchasing a Synology DS215j NAS.

I settled on it because of a few factors: Price, reviews, and — uh — price. Did I mention price? I'm cheap. I also picked up two Western Digital 5 TB Red Drives that I would be setting up in the drive. I've never had WD drives fail on me so I went with what was familiar. I'm sure there are options out there that are just as good.

After popping the top on the Synology, inserting the 2 drives, and closing it up (roughly about 10 minutes of work), it was time to plug this bad boy in. I was pleasantly surprised at how simple the setup was. I hooked the NAS up to a gigabit Ethernet port on my router, went to the web address printed in the manual, followed the instructions, and I was formatted, up and running in about 20 minutes.

Now for the fun part. I had roughly 3 TB of shoots to put onto the drive. Easy! I figured I'd just mount the NAS to my Mac (no problems there), move the files over in Finder (check), go to bed (I'm good at that!) and all would be done when I woke up in the morning.

Nope.

Apparently my transfer had gotten about halfway through during the night, then the NAS reset to apply some updates without my permission. Oy. Alright, I changed the permissions on the NAS to ask before updating and away I went. Thirteen hours later, I'm in network nirvana! I can open my files in with read times only slightly slower than my local LaCie drive was giving me.

Next step: I'll just put my Lightroom catalog on the network and fire that bad boy up to check how Lightroom performs on the network.

Nope.

Apparently, unless you hack your own computer, the Lightroom catalog has to be local to the machine. No network drives allowed. You can store the files themselves anywhere you'd like, but the catalog has to be local.

I've got a flash drive coming in the mail so that I can take my Lightroom catalog with me from machine to machine with no issues. I'll update when I receive it.

In the meantime, I wanted to check the import times I'd be getting. Long story short: Lightroom is slow. Really, really slow. At first I thought maybe I had done something wrong with my install of the NAS, so I set up a little experiment.

I downloaded Capture One and did imports with 27 files from an SD card with both Lightroom and Capture One.

Capture One imported the files in half the time. Since I was using a crappy USB 2.0 reader, I thought maybe that was the issue. My shiny Lexar USB 3.0 reader is still in transit. To make sure the slow reader wasn't the issue, I copied the 27 files directly to a desktop folder and did the import to NAS from there. Again, Capture One finished in half the time.

Zooming in to the photos in Capture One was also an order of magnitude faster than Lightroom. It was taking forever for the images to render fully at 100 percent. I have a feeling Capture One will be getting a lot more love from me in the near future.

Some perks I didn't count on with having this NAS were the ease of having access to my files from anywhere. With free apps, I can pull up any files I want to my cell phone or tablet at any time. Amazing! I'm sure this is old hat for all the tech-heads, but I might have to cancel my Dropbox subscription. I could use an extra $10 a month in my pocket. Also, I can set up folders for my family photos and files and give access to the wife for certain folders.

I still have tons more to learn about working with the NAS, but I'm really enjoying the safety and simplicity of it for now. Do any of you have tips for dealing with network storage for newbies? Are there any apps for the NAS that can make my life easier? Is Lightroom this slow for everyone else or am I missing something? Let me know!

Log in or register to post comments

69 Comments

Oh god, please no. Just no. This article is a cry for help with a bunch of ‘what not to do’ points. Please take this down before you steer people onto the wrong path. Clearly you don’t know what you’re doing (which is fine, every starts somewhere) but please don’t write articles like this making it seem like these are good approaches. Fstoppers why in the hell would you publish something like this? This is seriously detrimental rather than helpful.

UPDATE:
All of the down voters and commenters are right. I should have approached this with advice rather than criticism, but I only caught this article moments before I had to fly out the door and was so agitated (and still am) that it was published on this type of site that I ended up venting to try and get a warning up in the absence of time for a proper response.

To clarify I have absolutely zero issues with someone undertaking this kind of project and blogging about it, writing an article about it, etc. Everyone has to start somewhere and as the author wrote in one of this comments sharing the experience is a great way for people to learn. My frustration with the article is this… Fstoppers has caught flack for fluff articles before, but worse than fluff this article could put readers photos and maybe even business and livelihood seriously at risk. Readers come here for advice, often from industry professionals. Those who don’t know better could easily be inclined to start following this advice and may or may not read the comments for clarifications and corrections. This, I wholeheartedly believe, is providing misinformation and a disservice to the reader base.

Now that we have that out of the way, here are some of the issues with your setup.

1) “I had no real backup. … If something were to happen to that drive I would have been royally screwed.”
Sorry to say that you’re not much better off now. I must stress something that was stressed to me many years ago. RAID IS NOT BACKUP. RAID is only designed to reduce/prevent downtime. You still have a single point of failure in several regards. The only thing that the NAS could protect you from above and beyond what your external drive did is a hardware failure of a single drive within the NAS. It will not protect you any better from file corruption, accidental deletion, user error, fire, theft, or even a total device failure, etc, etc.

2) “… the Lightroom catalog has to be local to the machine”.
Correct, and rightfully so. The read/write from the catalog has some of the biggest impacts on overall light room performance. As you edit Lightroom stores the data inside the catalog, in real-time. This includes data intensive adjustments such as the Local Adjustment Brush and others. Storing your catalog on anything other than a fast internal disk is just a bad idea from a performance perspective which is likely one of the reasons (maybe the main reason) Adobe attempts to prevent you from storing the catalog on, say, a network drive. Even if you get this working (which you can without too much trouble) you will (under most circumstances) take a performance hit.

3) “I've got a flash drive coming in the mail…”
When I first read this I assumed you wanted to store your catalog on the USB flash drive all the time, especially since you were talking about storing the catalog on a network location. Re-reading it I’m not so sure. Regardless this is also a bad idea. See point 1.

4) “Capture One imported the files in half the time.” / “Is Lightroom this slow for everyone else…”
That’s great, but it has nothing to do with the NAS. You even somewhat proved it with your additional test. This feels like an irrelevant distraction and probably doesn’t need to be in the article at all. Again an open call for help in something that should be in a forum in the first place.

Honestly it just sounds much more like you’re reaching out for help in general with this article and doing through a larger platform like Fstoppers because you have access too it rather than doing it in a user forum where it should be done or just reading the many, many, articles and discussions that already exist on the topic.

Tam Nguyen's picture

lolwut did someone piss in your breakfast?

You should probably pick a couple of the bad points in the article and tell why you think they are wrong so you don't seem like as much like a stupid person that doesn't know what he is talking about.

Yes please post some of your ideas instead, keep it positive :)

Hans Rosemond's picture

Sometimes, the best way for people to learn is to show their own process and the things they learn along the way. I could have spent time doing all of this in anonymity, but I believe that the best way to benefit from a community like Fstoppers is showing that process to the public, warts and all. When I get solutions to my questions (as evidenced by the many comments below) everyone benefits! I think that's far more productive for the community as a whole.

This is exactly what discussion forums are for. Not articles on sites that typically provide readers with details guides, how-tos, professional advice, and alike.

Chris Parent's picture

The biggest problem is that there is absolutely no backups in this system. Sure it's great to be able to access files over a network, but if something happens to that Synology, you're still out of luck just like you were with the Lacie drive. For a beginner trusting Fstoppers to point them in a good direction on workflow and storage, this article needs to include notes that you should ALWAYS have backups.

An easy addition would be to buy another duplicate Synology and have them sync over internet or such which is easy to due via their system if I remember correctly (I don't use Synology but it's a feature on several of these NAS I believe.)

Hans Rosemond's picture

The synology is in raid 1, so there is a backup. Thats half the point of the article. That being said, I definitely need to get a third backup off-site

This is WRONG, so very wrong. RAID is NOT a backup. It is for more convenient / speedy recovery of hardware failure, nothing more. It does not protect against corruption, accidental deletion, etc, etc, etc.

Hans Rosemond's picture

Dude. Define backup.

Mike Leland's picture

A backup is a totally separate version of your files.

There are different types of raid. Striping, Mirroring, Striping + Mirroring. With & without parity or distribution.

You didn't specify which raid setup you were using. With only two drives, your choices are either striping (raid 0 for speed) or mirroring (raid 1 for redundancy)

Mirroring does not give you a "backup". If you're writing to that disk and files become corrupted during the write, you're not safe.

I have a computer engineering background and worked with storage arrays in the 1990s and early 2000s.

I can tell you from experience that the purpose of having drives in a raid 1 array is to reduce the probability of downtime. It is for scenarios where any downtime for the storage system could be bad news but not catastrophic.

Raid 1 is two drives operating as one but the drives are mirrored for redundancy. If one of the drives experiences a failure; you can replace the drive with no downtime to that storage segment. The new drive will then be written to with the data from the remaining drive.

So this is basically a single drive.

A backup would be an entirely separate drive (whether that drive is two drives in a raid array or a single drive doesn't matter as much as the fact that it's a separate drive)

I personally keep a backup on site, a backup off site and have a cloud backup...

Larry Clay's picture

Welcome to the often confusing arena of "backup". First you need to define your needs for storage.

1. Fast read/write access for day to day productivity.
2. Secure local data backup.
3. A place to put data for multiple computers on your network.
4. Off site data backup.

There may be more but these are the basics.

Each of these four require different solutions and are not compatible with each other.

1. From your article I see that you use Lightroom and Capture One. I assume Photoshop as well. For day to day productivity the best solution is storage, either internal or directly attached using USB 3 or equivalent. There is no NAS solution that can match directly attached storage.

2. Backup is just that, backup. You should never combine productivity and backup on the same device. The main reason for this is to maintain the integrity of your backup. This can be either directly attached or a NAS solution.

3. This is the place where you put what I call overflow data such as documents, family photos, etc. It is a NAS solution that is available to all of the computers on your network. This should also be backed up to your NAS backup system.

4. Off site means just that. I use a cloud based solution called Crashplan but there are many good ones out there. After the initial upload of my data the program runs in the background daily and backs up anything new to the cloud. This is a very slow solution and recovery of data can literally take days but if my house burns down I can still get everything restored.

Raid 1 is just fine for your NAS. You are not running the kind of production environment that requires a more exotic raid solution.

I hope this helps. Good luck and keep it secure.

I commented back in March about RAID not being a back up solution. A back up solution protects you from data corruption or loss, RAID doesn't do both.

https://fstoppers.com/education/six-photography-mistakes-you-may-not-rea...

Most of you are right, I probably should have approached this in a more tactful way and provided some suggestions instead of just criticism, that is my typical approach. BUT, when there are so many whole I just don't know where to start. On top of that readers of this site look here for advice and in most cases the articles are written by professionals who know what they are doing. How many readers will be lead astray by this and put their photos at risk? I find it irresponsible on the part of Fstoppers. When I can find some spare moments I will write up a few notes, if they haven't already been pointed out by others in the comments.

Hans Rosemond's picture

At no point did I claim to be a professional when it comes to computer networking. The entire article is set up (and very transparently so) as a "follow along while I try to figure this out" experience. I'm sorry you don't find the article useful, but I bet there are many others out there who have learned something by the ensuing discussion - which is the entire point.

Hans Rosemond's picture

Again, Im sorry that to you it sounds like Im reaching out for help. But the thing is, you're wrong. Its a learning post. I, for one, feel encouraged when i know that others out there struggle with the same questions and issues as myself. You have some really great info in your rebuttal. I wish we didnt have to wade through all the negativity to get to it.

Tam Nguyen's picture

If you could write this up in a post format and send it to either chelsey@fstoppers.com or david@fstoppers.com, I think we could make it a post on the front page.

I'm pretty disappointed with this article too but I have to disagree on one point - RAID 1 most definitely is a form of backup. I think you even unintentionally contradict yourself by giving the definition of RAID 1 - it's two drives containing identical information, which by definition is a form of backup.

Is the poor author completely clueless? Yes.
Does he enlist help from an expert/friend/colleague/redditor? No.
Does he need lessons in computer? Definitely.
Should we bash him? No... maybe a tiny bit, but at least not in a mean way.
I have artist friends who take paths down the road to computering properly and some reach their destination but the point here is to make the difficult explanation easily digestible which unfortunately cannot be done if you want to actually do it properly. For example Tricking a program or OS into using remote storage (whether it's connected to your router or half way around the world in the cloud) is a pretty simple procedure in itself (especially on a *nix like OSX) but can't be explained without technical details and shouldn't be dismissed as "hacking"

I love the first line though "oh God no. please no. just no."

Tam Nguyen's picture

Assuming those drives you have in there are 7200 rpm, on average, they max out at around 90-120 MB/s best case scenario (megabytes per sec); that's around 960 Mbps (megabits per sec). Your average ethernet wire is 1 Gbps (gigabits per sec). I don't see how the speed of your ethernet could ever be the bottleneck. If you get LaCie or G-Tech shit, they all pack with Seagate or WD drives with the same speed anyway. You're pretty much paying for the brand.

Btw, I wouldn't call "mounting a network drive on your local computer" hacking it.

FYI:
1 byte = 8 bits
1 mega = 1,000
1 giga = 1,000 mega = 1,000,000

Gigabit ethernet is 125MB/s and that's not counting overhead for the packets so you never end up with a pure 125MB/s of the original data.

If he set the NAS up in RAID 0 like his old USB drive was configured, I could see it easily maxing out a gigabit connection.

Tam Nguyen's picture

Right, but he's not setting it up as RAID 0. I mean, if performance is your #1 concern and you still want some redundancy, RAID 10 is a safer bet. At which point, yes, I agree, a NAS is no longer adequate, unless you have a commercial switch to hook it up using CAT-7 cables.

FYI:
1 byte = 8 bits
1 kilobyte = 1,000 byte
1 megabyte = 1,000 kilobyte = 1,000,000
1 gigabyte = 1,000 megabyte = 1,000,000 kilobyte = 1,000,000,000 byte

or

kilo = 10³ byte
mega = 10^6 byte
giga = 10^9 byte

The Synology systems can easily saturate a single link, and even a dual link Gb/e. I'm using a dual link DS415 and can pull anything from 200-220mb/s across the LAN but that's on a managed switch and with the network fabric to support it.

WD Reds are more than fast enough for this solution and the WD Reds have proven to me to be the most reliable out-with the Hitachi He drives...

Jordan Parker's picture

I've been dealing with the NAS/lightroom workflow problem at work for a few weeks now.

A little background:
For the past few years I've been able to use lightroom over Iscsi. iscsi connection is a way of mounting a network drive as local, but it requires a software controller and a separate network connection (can't be part of a home or business larger network). Works great for video, audio, and photo workflows. Problem is we are trying to set up a easily accessible photos drive for multiple departments to us (print, web, communications, etc) and decided to go with a drobo (cause you know cheap compared to an actual server). Turns out, as this author discovered, lightroom literally will not work over a NAS connection (we are using for the drobo because iscsi would require a lot more infrastructure cost and user training).

Solutions I've tried:
1) Keeping my lightroom catalogues local and synced to other users via dropbox while keeping the stills on the drobo
--outcome: works great, except dozens of photos have already corrupted through this method...so no go

2) Playing with CaptureOne as an alternative.
Outcome: So far so good--and it actually may work better for our office structures and institutional needs, but I still need to do much more extensive testing before rolling it out.

I would love to hear thoughts on this NAS problem since multiple persons on campus are still using contact sheets as a means of culling and organizing photos from a digital library--this makes collaboration a real chore!

What about keeping the catalog in a mac disk image. Put the disk image on the server, double click it to open on the local desktop. Never tried this but would love to know if it works. I use catalogue sync via dropbox with image files on a NAS. So far no issues, but you have to be meticulous in making sure that all syncing has completed before opening the catalog on another machine

If you shall work in a real multiuser enviroment you need a seperate DAM software or a RAW Editor thats multiuser enabled, because you need to check oout the files when editing them ...
There are a couple of multiuser editors out there, but the DAM/RAW Editor solution will be a better way of doing this. If you use something like Daminion Server or Fotoware Fotostation in combination with Lightroom/C1 pro you will get a system that not only are multiuser, but where you also can configure Access rights for det mediafiles ...

The files itself can be stored where ever you like when using a DAM solution.

Jordan Parker's picture

Thanks Jaran! That is the direction we are heading for our video workflow, but I hadn't thought to apply it to the photo workflow. We are still a year or two away from implementing any kind of DAM on the video side, but I'll be interested to see how it works for photos. For now I've settled on Chronosync and Adobe Bridge as a workaround workflow. I was trying to push our cataloguers (don't edit, just collect for print and web) and Aperture editors towards LR for both cataloguing and editing, but Bridge will have to do for cataloguing, and I can't really do anything about the Aperture hold-outs at this point. With Chronosync I end up with two copies of every file so at least its a form of backup since I'm storing photos locally now. It also lets me continue to use LR.

What will happen if a fire breaks in the basement?... hmmm..

Hans Rosemond's picture

I'd be screwed! Off-site storage is definitely next on my list to secure my data. Any good suggestions for myself and the rest of our readers?

Quentin Decaillet's picture

Check out Amazon Cloud Drive. For 60$/year you get unlimited storage and the Synology NAS have an "app" that can sync your files and folders automatically for you.