Okay, so I know a lot of readers may disagree with me on this and have different opinions. My writing here isn't intended to cause any offense, but more to offer an opinion and start a discussion on the matter. My evidence here is based on having worked in the IT industry for over 30 years and working in the photographic industry with creatives. Hopefully, after reading this you'll come up with an understanding which may help your next (or first) computer storage purchase.
The Cheapest Way
The cheapest and most convenient (but least reliable) way to store your images is on the internal drive of the computer you are editing on. There's no need to buy any extra equipment and you might think that this is a good place to start. On import from a camera or memory card, an editing program such as Adobe's Lightroom copies it to a location on your hard drive. This is an incredibly bad way to store your images so I'm now going to set out my reasons why.
Your internal drive contains the files needed to run your operating system and the programs you use on this hard drive. An internal hard drive will slow down as it gets filled with more data and as such slow your computer's access to the files it needs. Think of the files being like books in a library. Isn't it easier to find the book you want in a smaller library than in a library with a million books? Ideally, you shouldn't be filling your computer more than half full of data.
- Access to the files. Unless you're using a remote access program such as Teamviewer or Google remote desktop or a cloud-based archive system like Dropbox or Google Drive, you aren't going to be able to access the contents of the computer whilst away from the office, home, or place of work. Getting access to these files can be problematic sometimes and you'd normally have to log on to your computer to access files on your desktop.
- Failure of the computer system. If you have a problem with the computer system. Unless it's a catastrophic hard drive failure. Getting the files back can be a pain but generally, it can be done. It might mean taking the hard drive out of the computer and inserting it in a caddy or another computer or getting the computer repaired first.
- Archiving the files. Storing the files on the internal drive doesn't really give you the option to archive the files and store them for future reference in an alternate location.
The Most Expensive Way (Long Term)
External Hard drives are cheap and convenient. Products such as the WD 2TB Elements Portable USB 3.0 External Hard Drive are cheap and portable. Even by buying rugged drives such as the LaCie 2TB Rugged Mini USB 3.0 External Hard Drive you're not completely protected.
- Small drives are easy to misplace or get stolen if it's something you're carrying around with you. Ones that aren't rugged are likely to break when dropped.
- The data can only be accessed when they are connected to a computer or plugged into a device that has the ability to share the contents. For example, some routers have the ability to plug in a USB drive and allow access to this across the network.
- Single drives can fill up quite quickly and even if you're buying larger drives. File sizes on photos increase over the years and the longer you keep the drives plugged in and working the more chance of failure. If you're going to go down this route look at smaller drives that you repurchase every year or when they fill.
- There's zero redundancy. Once a drive fails, you're looking at a hefty bill for recovery of data or worse still lost data.
My Recommended Solution
A Network Attached Storage array such as the Synology DiskStation DS1821+ 8-Bay NAS Enclosure is definitely the way to go. This particular model is the one that I personally own and has 8 bays and allows a variety of RAID configurations. A RAID (Redundant Array of Independent Disks) configuration gives the ability to spread files over multiple drives allowing for redundancy should one of the drives fail. RAID comes in 2 types and multiple configurations. The types are Hardware Raid and Software Raid. Hardware RAID uses a physical controller chip and is in general a little more reliable than Software based RAID systems. The most common RAID configurations are 1, and 5. The DS1821+ has also the advantage of including Synology's SHR (Synology Hybrid RAID). This allows you to use drives of different sizes to create your array as opposed to RAID which only allows for drives of the same size. SHR comes in 2 configurations with SHR being the same as RAID 5 and SHR2 comparable to RAID 6.
The system is incredibly easy to set up thanks to Synology's included software and it's very self-explanatory making it easy to set up your first array. You can start by making your SHR array with just 3 drives and then add and increase the size of the array as you add more drives at a later date. This is one of the reasons for buying the 8-bay drive. Expansion is just a matter of adding extra drives and then clicking into the software and allowing it to rebuild.
Speed is also a factor with this device which has 4 network ports on the back allowing you to combine all the cables together and create a data aggregation bond making the transfer of data 4 times quicker than a single cable alone. If you connect your computer to your switch or router via an ethernet cable then editing directly from the drive is entirely possible.
Another reason why this is an ideal device is the ability for you to access the contents of your share via your phone on an app that is created by Synology and available for download on the play store and the app store. In fact, they have numerous apps that allow you to do multiple things including looking at your photos remotely.
Lastly, this device allows connection via the included app Cloud Sync to multiple, online data storage sources for cloud-based offsite backup. It's easy to synchronize different folders to different cloud-based systems.
Hopefully, this guide will give you an alternative option if you're considering your backup and editing solution. Synology has a RAID calculator on their website that will give you an idea of the type of array you can create with different-sized disks. https://www.synology.com/en-uk/support/RAID_calculator
How is Network storage more safe than Cloud storage?
Personally I recommend cloud storage to most people. First, it's MUCH easier to setup (Synology is complex enough you'd better be at least a little geeky). Second, it's automatically geographically distributed, so you're protected in the event of a theft or fire. But cloud storage isn't usually fast enough if you need to, say, pull the 1TB video project you completed last year. It also gets EXPENSIVE (I have about 120TB of data on my NAS).
Hey Rogier. Cloud storage is a great option too. Remember this is just my opinion and based on my experience and what my tech company has fixed and dealt with over the last 12 months alone. You are absolutely right Tony, cloud can get expensive especially if you're retrieving data a lot of the time.
Cloud storage is a catch-all when they are attacked and data is exfiltrated. Running a NAS at home, you would have to be "interesting" to be targeted. If you follow simple security stances of not connecting your NAS to the cloud for remote management/control then there is less then a 1% chance your at home NAS will ever be targeted by attackers. If you want a recent example just look at laspass, I would never keep my password database in the cloud and that attack is why. Same goes for my critical data. I would, however, send encrypted backups to the cloud for off-prem backups. But thats about it.
Securing against data theft and inspection from your cloud provider (don't trust them either) - means double encryption has to be applied. Encrypt your own files with a key you only have. So everything stays unreadable to Apple, Micrsoft, Amazon... You want to read that article from the german C'T where they describe the horrors of cloud providers just cutting your accounts (and that includes almost all of big tech) and the ways to save your data and what's dearly to you (and that involves lawyers).
To avoid that use the encryption of the provider and use your own at file level. Don't use their automatic syncs! (because you cannot apply your own encryption).
+1 for Synology. BTRFS has data scrubbing which finds and corrects corruption, such as bitrot, which INEVITABLY destroys your old pictures without you ever noticing. I use the 10GbE adapter which gives me real-world 3-4 Gbps transfers with SSD caches and BTRFS metadata pinned.
BTRFS or ZFS are essential in any NAS. If the NAS doesn't have it, don't buy it.
I want to know whether he's using normal Ram in his Synology or ECC?
With 120TByte of storage there may be a few bits going wrong in memory too.
Without ECC-memory there's still a lot of corruption risks both with Btrfs and ZFS (in the past the creators of ZFS advised using ECC-memory).
But...you're responsible for maintaining the NAS drive and keeping it secure. You must keep up with security patches, and over time you will run out of disk space, especially with today's very large raw files. And what if there's a fire or flood that destroys your RAID array? I think a good compromise is a local RAID array for quick access and in addition backing up everything to the cloud. I'm using Dropbox because it works best for me, but there's a lot of alternatives out there. I'm fortunate to live where gigabit FiOS is available, so backups to Dropbox are surprisingly fast. I find that a year of Dropbox costs about as much as an inexpensive RAID Level 1 (mirrored disks) for 6TB.
I've heard that Synology has a way to send files to AWS, but I think that would require more IT chops to set up than say Dropbox.
NAS storage can be expensive from a capex viewpoint, but upgrading the storage doesn't have to cost a bomb though.
For example, say you had 4x 4TB drives in a 4 drive enclosure... That's 12TB storage (# of drives minus 1). Want to go from a rather economical 12TB capacity to more? 8TB drives don't cost that much. Just buy one a month. When you buy the drive, just replace an existing drive and let the NAS rebuild. After four months your NAS will allow you to expand from 12TB to 24TB with just a few clicks and slightly lower performance for a day or so (weekends are your friend). How much will 12TB of cloud cost over four months?
Edit: Security - Almost all NAS will automatically check for updates and apply them on schedule. Just check the report emails that you should enable.
Not that I recommend NAS over cloud - Different purposes for each. I use both.
Thank you for your comment Jon.
Harry Bloomberg yes AWS can connect to it. It's one of the options on the cloudsync app. I've not set one up yet as I use Backblaze B2 which is fairly easy but yes it can be done.
Strong disagree. The only way to *safely* store images is to have them in MULTIPLE locations. The 2 NAS systems I used in the past both catastrophically failed (even with RAID) causing the loss of 50K photographs. It wasn't a big deal as I also had them on (A) Individual external drives for each year (B) A larger external drive that I de-dupe all the individual external drives onto (C) An archive PC (10 years old) with RAID storage that is kind of like a NAS I guess, but I never pull images directly off it or write anything other than the archived photos to and (D) Backblaze covering the archive PC. This year my larger external 4TB drive failed - no major trauma, bought new drive and restored from the archive PC. NAS to me always seems the worst of all worlds - they are expensive, you think your photos are safe, the performance is not great and if that's the only place you are storing your treasures, you are one day in for a nasty surprise, RAID or not.
Thank you for your comment Roger. I appreciate all views and this article is meant to demonstrate my experience compared to using single drives and cloud experience. There's always going to be a situation where any type of backup has the potential to fail.
I moved my backed up drives to my studio which is a separate building on my property. My office is in my home. Over 30 years of images that also includes film.
Bonding the multiple LAN ports on a Synology won't allow you to transfer files to/from the NAS any quicker than using a single LAN port.
You're connecting to your NAS via a 1gb ethernet cable, even if you go via a router or switch. The max you can send down that cable is 1gb. Bonding doesn't magically allow you to send 2, 3 or 4gb down that 1gb cable.
Bonding the Synology connections together allows for more than 1gb of bandwith to availible for MULTIPLE connections. So for example if you bonded 3 of the LAN ports into a switch, three seperate connections to the NAS could utilise up to 1gb each. Where as if you had 3 connections accessing the NAS via a single connection to the router/switch, they'd each get one third of a gb each.
If you want faster speeds from a Synology you need to use 10gb ethernet. Newer Synology NASs have this built in, some of the older ones require an expansion card, and some don't support it at all. Using 10gb also requires a 10gb connection on your computer, so you might need a 10gb PCIE card installed if your mainboard just has 1gb.
I fiddled with a LAG capable switch once, it was so so. Personally I just found it better to just go 10g via an SFP+ switch.
You would need a 10GB switch, port on the Synology and port on the back of your computer to take advantage yes as otherwise it will bottleneck at the weakest point. It will however allow my staff to access the contents in their office whilst i'm also using it here in my office the aggregation part allows for multiple access streams at the same time.
In my opinion, the proposed security concept is not up to date:
- Insufficient protection against encryption attacks
- No backup at spatially separated locations
- Loss of performance for the working files
My solution for me as indivdual user (workgropu solution had to be differen):
- An NVME SSD for the operating system and programs.
- One NVME SSD for the working files
- A hardware RAID with large HDDs inside the workstation for file storage of edited files (Alternatively, external SSD with Thunderbolt 40GBit/s could be used here, but is more expensive than my solution).
- A NAS connected via USB 3.1 (5 GBit/s) with RAID for running backups (with appropriate settings for Windows Ransomware protection)
- Several large external HDD with a backup rhythm. Several of these are stored in other buildings.
Online availability via sync in CreativCloud.
It’s not that it’s not up to date I just haven’t touched on that depth in this article. There’s a fine line between writing an article that most readers can understand and through in so many acronyms and tech terms that readers switch off. This is a finely tuned balance.
I'm done with cloud storage, for several reasons.
NAS is cool, and fits the use case. But NAS has been targeted by ransomeware, so you have to be awfully careful.
For me, a removable drive is fine.
With regard to encrypting ransomeware, snapshots can reduce your exposure. Not foolproof, but snapshots with a well though out retention policy can help.
FWIW:
I have 2 external drives on my desk. One is a clone of the other. All my data is in these. I then have 2 more cloned drives in a fire safe.
All the software is on the laptop hardrive and in a 5th external drive managed by time machine. Over 20 years with this system I have been able to easily recover from multiple disasters (all of them user error!).
Is this an article or an ad for Synology? The safest place to store your photos/data is in the cloud. With that said, I backup my data using backblaze. I edit my photos using a direct storage system like an external drive and then backblaze will back that data up automatically. What you're recommending is to keep the backups in the home. What if there's a fire? What if someone breaks into your home and takes all of your equipment? Synology or any NAS doesn't protect your data in those cases.
You might want to ready the article again Chi Liu as we mention western digital and Lacie in there too. You'll also see that I mention that this Synology connects and allows cloud backup easily via the installed and included apps. Cloud is great too but as a standalone solution it's just not viable because of the cost of retrieving data and the speed of download aspect.
Good article, covers a spread of options for the average user.
Just for fun and for anyone out there with even a vague interest, I used to store all my files spread across 4x 4-6TB 'average' drives inside my main PC (Windows). They were backed up with Backblaze, I thought all was well. Then one day I wanted to share a video of someone no longer with us, with other people and it was corrupted. This was saddening, but what was also not ideal was that my work files were stored in a similar fashion, most edited files were just in a dropbox folder. It was then that I learnt Operating Systems don't really care about data integrity or checking.
I saw Synology as a good option initially, but I didn't appreciate being locked into a system that in the event of failure, I'd have to replace (very quickly) with the same system in order to access my files. So I looked at UnRaid, that had it's own deficiencies even though it was very simple to use, and ideal how you can add drives as you go.
I heard murmurs about TrueNAS and that's the system I ended up using, though it took a long time to understand and trust. I don't have much data, perhaps 20TB (work and personal). I got a 10G SFP+ switch to get the most speed out of automatic backups.
Of course, the cost of electric (in the UK) went up drastically, so I've scaled down the system accordingly and have a 30W system that's always on, while other systems are turned on briefly to make automatic copies.
Thanks for your comment Chris Ellwood
You're welcome Peter Morgan
8 bay OWC DAS with Backblaze works fine for me
Love OWC been using them for 20 years.
Q: How do you backup a NAS?
A: with another NAS
If you have enough data to require a NAS, you aren't backing this all up in the cloud anyway.
Also you need to seed the cloud which takes weeks or months and will eat up your data plan (unless you pay extra for unlimited data).
You NAS will backup to the Cloud but usually in a format that requires you to to replace the NAS (in the event of loss) to get the data back and you would be best to encrypt the data to and from and on the cloud, which means you can't get access to it with a phone either. You also better not lose the encryption keys (don't store them on the NAS).
Single hard drives are large enough now that you can store all your data on a NAS and back up sets of it to single hard drives and store these off site. It a lot cheaper than cloud storage.
QNAP is superior to Synology in many ways but ZFS is probably the best reason. Rebuilding arrays takes minutes instead of days.
Yea Hyperbackup will allow Synology to Synology across the internet.
But it will be slow as shite :)
Only dependant on your internet speed I have a 1Gb line in Cardiff and it works perfectly. Granted there are parts of the world where internet speeds are not great at all and so this part of the solution is one that is not suitable for everyone
Backblaze is $7 per month. How are hard drives cheaper than that ?
$7 every month just to back up your digital files? And you think that is inexpensive? Dang, that sounds like a ton of money to me. I can get a used 3TB hard drive for $40 or $50 bucks off of Craigslist or eBay and that lasts me for years and years and years. Or worst comes to worst and I buy a brand new external hard drive from B&H when they have one at 50% off on one of their Deal of the Day email specials.
There is no need or advantage to back up every file, as only the real "keepers" are worth saving, so 3TB should last me for the rest of my life. I mean, honestly, only about 2 or 3 frames out of every 100 frames that I shoot are worth saving, so no need for more than 3TB, ever. If you back up more than that then I suspect that you may be saving files that don't really need to be saved, unless you're a pro who shoots for hire or something.
Buying second hand drives from Craigslist or Ebay is not advisable at all. You don't know whether they have failed and have bad sectors, can never guarantee the reliability. In a business solution (which is what we're talking about here) its just not reliable enough taking a chance with second hand hardware.
Time is money.
I've used to use B2 and it is a PITA to seed and took roughly 2 months to get 4TB into their cloud.
This ate up each month of Xfinity's data bandwidth, so I had to pay for unlimited data.
Then there is the time to get it back again.
B2 have a hard drive loan cloud -> on premise seeder, but I think it is limited in the size that they support.
Then again, if you are not putting all your files in the cloud, then you should probably use a NAS + cloud anyway.
I just prefer using 16TB drives in a fast attach dock connected to the NAS over USB to take a snapshot.
I'm currently using iDrive and found their speeds really good. Uploading 500GB took under 10 hours - And that's on only 100mbit up.
Sorry to hear that there's still ISPs with data caps...
Well when we look at the track record of encryptor-malware on Qnap and Synology - those numbers tell me something. Both have issues, but one more than the other.
The less connections are allowed the better it is. Put the axe into the features. Try to firewall. Update often.
One question:
Who has here uPNP turned on, on their Nas-boxes/routers?
(i'm saying since Windows98 came out this feature was pure poison, a dream for malware creators).
.
Any type of storage that involves the internet is a HORRIBLE way to store files, for me.
Why?
Because I sometimes spend weeks and weeks, or even months, with no access to the internet at all. Some people think of the internet as something that is always available, but those people do not go off to remote locations for weeks at a time. Those people do not camp in wilderness with no cell signal for weeks at a time. They are homebodies who always sleep in a town or a city or a suburb and have a bed to sleep in and electricity to charge things up with. They do not understand reality because they live entirely in the false world that man has created. They do not live in nature for long periods of time. Bah humbug to them! ha ha!
The other reason cloud storage or similar storage is horrible is because it depends on you being able to pay a bill every month or once every year. What if you just don't get online for 3 or 4 months, so you don't pay your bill? What if you get arrested for something and stuck in jail for a while, with no way to access your accounts on the internet? What if the IRS seizes your bank accounts so that automatic payments do not go through when the bill is due?
If any of the aforementioned things happen, and your account lapses for a long time for nonpayment, then you are SOL and the company you were paying to store all your data eventually deletes it and you are SCREWED.
For simple little people who always live in comfort and safety and stability, maybe the internet storage thing can work. But how do you know you will always be able to pay that bill? How do you know you will have income next year, or the year after, or the year after that? Financial collapse is just around the corner for many of us who do not have careers or regular employment, so relying on a paid service to store our data for us is foolishness.
I recommend external hard drives. I store my image files on my computer's main drive, as well as on two big external hard drives. And I store the very best images on a little 256GB flash drive, so if everything else fails, at least I will still have the most special images stored. That way, if I am out of internet service unexpectedly, for months at a time, I still have my backup copies. If I have no income or if the government seizes my bank accounts again, I will still have my backups. If I travel to another country and get held in jail for breaking their laws, I will still have my backups. I can be out of touch with the world for months or even years, and still have my backups.
.
Absolutely agree cloud storage alone is just not a viable solution. The best solution is onsite, off-site and cloud. This is just a comparison to just using single drives alone
Don't agree. I have a total now of about 17TB of data in photo and video. It has taken weeks to all upload to Backblaze, but it's there now and available. Tom R, I also spend weeks each year in remote locations, but use Starlink to back up when I can, or then just back up portable SSDs when back home. Not prepared to run the risk of losing everything if my house burned, so cloud based backup it is
First off, you compared hardware to software RAID while talking about Synology. Synology does not support hardware HBA's and is always a software raid solution with either Geom(RAID) or SHR under the hood. If you wanted to touch on this you should have also gone down the path of TrueNAS/FreeNAS with ZFS vs hardware HBA's so the readers have correct information. Synology is still the better turn key solution as the NAS Distros require building out the hardware, but whitebox NAS builds can be cheaper per drive bay then Synology.
SMB3.1 multi-pathing would be the correct way to discuss scaling the network for file access. You would still need multiple 1G links on each workstation(Can be USB->1G NICs) and a decent switch to support it, but Round-Robin path hunting is how you scale SMB access to the NAS, not LACP/LinkAGG. Link-AGG allows for more sessions into the NAS so not any one user can flood out one path, but it does not scale without round-robin. then the optional 10G would be a good talking point for scaling out throughput to the application stack (Synolgoy Drive, Synology Photos,...etc) . 10G networking, MicroTik has the best price per SFP+ port still today.
But RAID is not a backup and you still need to build out a backup strategy. USB connected drives are the best local backup solution, you can swap drives in/out if you use the right USB Sata trays, and you can encrypt your backups and version them through hyper backup. Then if you are an E3 user you have 1TB of MS Drive storage included in the sub, you can setup Cloud Sync to OneDrive by creating a folder on your NAS, then use HyperBackup to backup your NAS to that folder so Cloud Sync pushes from the folder to your cloud storage. Of course Synology offers C2 storage at about 80USD/1TB/Year if you do not have an E3 Sub or other cloud storage options.
Nice article, but you are underselling this quite a bit.
Thanks for your comments. I’ve not used truenas or free nas so I wouldn’t be speaking from experience if I wrote about those. I mentioned software and hardware raid as part of the explanation of what raid forms are available. Raid/SHR gives disk redundancy and the backup I use comes in the form of the cloud backup attached with the cloud sync app to Backblaze B2. USB local drives are not a great solution in my experience. Your single point of failure is literally the single drive itself. MS Cloud on a subscription basis is not a great option. There’s just not enough space available for the value for money. Yes Synology do offer backup, however I use Backblaze as they work incredibly well for me and this is incredibly cost efficient in the UK for what it is.
But a NAS + USB storage for backup is viable and works without Internet access.
I used to use NAS + B2 but now I use NAS + multiple 16TB drives in a quick eject dock that get rotated out and stored offsite.
A backup to disc doesn't have the disc as a single point of failure when you have a NAS in front of it.
Highly unlikely that a ZFS RAID 10 array will go down in the first place.
I prefer the USB drive instead of cloud for the Disaster Recovery use case.
Recovering 4TB of files from B2 would be a stressful time-consuming experience :(
Weird article and seems like a sponsored post, imho. Google drive is WAY cheaper and accomplishes the same thing. Am I wrong?
Thanks for your comment Zac S. Google drive is a cloud based system. This is an onsite solution with redundancy and links to a proper business backup such as Backblaze B2. You can't compare cloud to onsite as the two are completely different types of backup. You're right however that google drive is way cheaper. However most often the cheapest way isn't the best way. This isn't a sponsored post either, just my opinion as this is what I use and i also mention Lacie and Western Digital in there too.
Curious. I save my photos as the original raw file. Which I can access anywhere. Anytime with an internet connection. I couldn’t do the math but personally, I’d trust the google service I pay for over the sdds I triple back the originals. I’m just curious why you don’t trust/rely on cloud based storage over physical storage. I do triple. Hard drive. Separate ssd and cloud drive.
Zac S. you are not right... you are not even wrong... may god have mercy on your soul lol