As a wedding photographer, the ability to upload multiple cards at one time has always been intriguing for me. The problem has always been that the price for these multi-card readers have always been a little steep in price. But with this DIY enclosure, it seems to be a little easier and cheaper than I thought.
In the video, we are shown how to create a single box that can house a fast hard drive system that is capable of being used for current working projects. The benefit here is that the system also has three SD card readers that can be uploaded at the same time. Because the hard drives are set to RAID 0 and everything is USB 3, the upload speeds should be pretty impressive.
What I like about the walkthrough of the system is that after seeing it, it’s easy to start thinking about other ways to tailor this box to your specific needs. For example, you can have a larger box that incorporates your backup drive. If/when I build one, I will also have four card slots instead of three because after a wedding I almost always have four cards to upload. All this plus the ability to customize the look of the enclosure for around $100 is pretty hard to beat.
For a list of parts, or to see more from the creator of this video, check out DSLR Video Shooter.
Is this something you might give a try? Do you currently have a multi-card reader? What do you use?
This is pretty rad, Thanks for sharing this.
Thanks for this idea, Jason! Perfect project for when I have some time.
Jason, your description of RAID 0 is incorrect. When you create a RAID 0 array you are creating what is called a stripe set or one big hard disk. With a mirror (RAID 1) you have two copies of the data, with a stripe set you only have one copy because the data is striped over all of the drives. Regardless of the number of drives the stripe set has, if one of them fails you lose the entire set. That is why you need to backup your data on a regular basis.
You can make it faster by using bigger stripes, more drives (to a point) and rather than USB connect the hard disks to an eSATA controller card where the drives have dedicated bandwidth rather than shared bandwidth between the card readers and the hard disks.
I'm not sure where my description was incorrect? In my third paragraph where I talk about a backup drive I was talking to the fact that when building this, you could incorporate another drive into the box to use as a backup. I wasn't saying that the raid 0 acted as a backup.
At 11:15 the video shows a file being copied to both drives and splits it. What actually happens is that the data is striped across the the drives in the RAID 0 device based on the size of the stripe when the volume was created.
Also the speed is not doubled, depending on the OS and the volume manager handles data transfers across a RAID device, you may see a 30 to 50 % improvement in data transfer rate. Software RAID is expensive in terms of performance.
ah ok got ya. I'm not the creator of the video, just sharing it. But I get what you are saying now.
Hey Robert, you're right. I didn't do a very good job explaining that. Was tying to keep it simple and I think I botched it a little. Thanks for the explanation!
I might be mistaken but by using the same USB III hub to run raid 0 and all those cards would create a bottleneck between the processing unit and the storage. That's why drives generally get their own sata connection.
Exactly!! You want better performance, you put those drives on dedicated ports.
It might create a bit of a bottleneck when copying data from multiple cards especially if they were being copied to the disks plugged into the same hub, but not much of one. The sum of the read/write speeds of bus powered HDDs and card readers is not going to be much more than what USB 3.0 can handle. It's also barely over $100.
Instructions from Apple on how to create a mirror (RAID 1), stripe set (RAID 0) and a concatenated RAID device. Note where the instructions specify storage block size, this is where you determine how big your stripe is, generally the bigger the stripe the better.
https://support.apple.com/kb/PH20579?locale=en_US
We have a desktop external hard drive plugged to the PC and and portable hard external hard drive for backup. The transfer speed is slow but we learned to live with it. We only do one wedding a week so it's a one time per week thing. The desktop external drive stays connected to the PC and those files are the ones we use for post-processing. The portable hard drive goes into our car which is not parked inside the house. This backup system has worked perfectly for us so far. I wonder where other photographers physically keep their backups.
Cassandra and Marc,
I use a combination of a 3 TB 7,200 RPM internal drive, two USB drives and burning to 50 GB Blu Ray to manage my images. This does not include the 1 TB drive the OS and applications are installed on (Windows 7).
The 3 TB drive is formatted with a 64 kb block size, the largest I can use in Windows and this speeds up my data transfers considerably. It only takes me a few minutes to download 32 GB of data from one of my cards (I haven't timed it). I also format my CF cards with a 64 kb block size to improve the speed of data read and written to them.
Using USB is a compromise because it is a shared medium as opposed to SATA which is dedicated. And the speed of the device is dependent on how many devices are connected and which one is the slowest controls how fast all devices work. If you have a USB 3.0 hub and plug a USB 2.0 drive in, your effective bus speed for all devices is the maximum that USB 2.0 supports.
I do not know your computer but what I have done is I installed a separate USB 3.0 PCIe card into my machine just for my external drives. This improves data transfers by putting the drives on different USB bus than the one used for my CF card reader. Whenever in doubt, use a separate physical device connected to its own controller.
I am a UNIX system administrator with 15 years of experience and a photographer.
When they say it is slow, they don't state what type of connection they are using, how old their system is or their respective drives. I can't agree with you more. But people will go WAY out of their way to save a couple of bucks and chance data loss.
As with so many posts on FS I wonder why? You have Mission Critical Data and you're willing to chance it's loss with a cobbled together, half assed dufus box. This is nearly as dumb as the guy trying to tell everyone on SLR Lounge how to buy the cheapest memory cards to save $10!! It's your data, do you really want to chance it? Buy the best you can afford, but don't get stupid about it. BTW, a single 3.5" 7200 rpm drive will be nearly as fast as the two 2.5 in a RAID 0. This is meant to be a reality check.
Michael,
When I read articles on this and other photo sites where the subject is computers I cringe at most of what I read. What people don't understand about storage and backups is:
1. Mass storage is not a backup. It is either storage or a copy of data. It doesn't matter what RAID level is used or how many copies you have, drives fail, data and volumes get corrupted.
2. A backup as far as I am concerned is tape or other media that is not online and can be used to archive imagery. Although I cannot afford tape, I use Blu Ray so that my data is safe and I can put it in a safety deposit box so if my home burns down, I only lose recent data.
Tape isn't cheap but it is effective and has been used for decades. You can have multiple tapes in multiple locations so that your data is safe. This is how the "big boys" do it.
Pretty much the same here, with two slight differences: I carry a portable drive that has nearly all my data, using CCC to keep it synced. And in place of standard Bluray I use M Disc Bluray, rated to ~ 200 years of storage life, also in safety deposit. Will we have anything to read them in even 30 years? That is the digital catch 22.
Michael, there is always that Catch-22. We have the same problem at work where we had to replace our tape libraries and had to make sure they could read our old tapes. This is something we will never escape.
I personally wouldn't consider Blue-Ray/DVD/CD disks as true archive either.
Some audio CD's I wrote back in late 90's had degraded a fair bit a few years ago. Granted - back then I bought cheap disks - now stick to Verbatim.
I use Verbatim after I had a spectacular failure using Memorex media.
I don't use standard BluRay. I use M Disc BluRay, they're archival. See here: http://www.mdisc.com
Thanks Michael!! I'll have to check them out the next time I buy blanks!!
This is also an awful idea. Where do you guys get your information. If you really want to archive it go with a data storage company that can guarantee your data or get some rock solid SSD drives and those them where ever.
Vicky, really!!?? SSD's for archiving data? That truly is an awful idea in the same light as using a stripe set with no redundancy is a bad idea.
Yes you can pay a company and "put it in the cloud", I have heard that before as well. Better hope that company stays solvent in the long term and doesn't hold your work for ransom when they are about to go belly up, or worse.
I do IT for a living and the best option if you are looking for long term storage where you can have multiple copies is tape. Its not cheap but it has been effective for decades.
Using Blu Ray has the same advantages as tape without the cost of a drive at around $2,300.00 for an external LTO-6 drive and around a $120.00 per tape and the cost of a Symantec BackupExec license. I can make multiple Blu Ray discs of key data that I cannot afford to lose and put those discs in multiple locations so that if I face my worst nightmare, you still have your data. Look up disaster recovery, part of my responsibilities is attempting to make sure that the place I work at can function in the case of a full loss of a site.
The vast majority of people I have worked for have never lived up to their disaster recovery documentation and the majority of the people on most photo websites don't either. To do disaster recovery right is expensive, really expensive which is why a lot of people and companies comply with the spirit and not the letter of the law.
Ironic that I'm sending in a Sandisk SSD for replacement. Nice to have a 10 year warranty, but archival? Absolutely Not!
Somewhere down the road, I see an enclosure fabricated with a 3D printer. It will be designed to give better access to the internal drives in the event they fail or when they fill up. Then really far down the road, you have someone design and build a logic board with a ROM that automatically creates folders and copies the contents of the CF & SD cards to the internal drives with an appropriate timestamp.
Hey Jason! Caleb Pike here, would you mind linking to my original article? Thanks for sharing my video!
you got it! done.
Thanks Jason!
Nice video - yeah would be nice to get some more techy build videos please.
As Robert Escue says - this is software RAID which has some more overheads etc compared to true hardware raid.
If I were to build this for speed - I'd consider SSD. For archiving/size - I'd stick to HDD
Chase Jarvis has a nice download/backup workflow Vid.
Recorded 2010 so tech may have changed, and some parts might be costly - but worth a watch.
http://blog.chasejarvis.com/blog/2010/06/workflow-and-backup-for-photo-v...
Very cool project! I think it actually looks really cool... I'm inspired to try it myself.
Just a note about woods: the distinction between hardwood and softwood doesn't actually have anything to do with how hard the wood is. Poplar is, indeed, a hardwood. Hardwoods are those that come from deciduous trees, softwoods come from coniferous trees (pine, fir, cedar, etc). As a general rule, softwoods actually are pretty soft, but hardwoods can either be soft (maple, alder, etc) or hard (oak, walnut, ebony, etc).
Caleb, great production values on the video and cool idea.
But expanding inside my own computer case is one reason I built my own PC; 9 internal drive bays and 4 external ones for card readers and other expansion. Everything is tucked away.
This is pretty terrible. If you want to have the slowest RAID 0 setup ever do this setup. Those HGST drives must won't be using there full performance especially on a usb 3.0 RAID0. Get a dedicated setup for your raid where you are using the sata connection to transfer information across the drives. I know it is a negative post, but this is a terrible idea and should never be recommended.
Vicky, I disagree. If the setup works for what Jason intended to do, why not. Is it perfect, no not by any means. I see it as a starting point considering he using materials he already has lying around. Now maybe in a perfect world and unlimited funds we can create something better.
Anyone run a RAID setup from USB 3.0 have benchmarks? I'm curious to hear real world data rates on a USB port for a RAID 0 setup.
Paul, getting benchmark results could be cumbersome based on the OS and the tool used.