We all have a half-dozen or so USB thumb drives laying around. You know, the ones you bought to replace the ones you thought you lost, but then found a week later? Maybe they were free at a trade show? I had my fair share and decided to do something about it: I created my new favorite and totally portable live-work "SSD." Considering I just needed one, $16 accessory, it all seemed pretty reasonable to me.
While I did use my spare drives for the test, I later found a deal on some 128GB PNY USB 3.0 thumb drives that I couldn't pass up for a proof of concept. The results were great, so that's what I'm going to discuss, here.
While many know that USB keys aren't great for fast transfers of a bunch of tiny files (i.e. a folder full of a few thousand images), they are fantastic for larger files like movies. Still, if you RAID them together, the speed slowly doubles as you add more drives (in theory -- read on for caveats). And eventually, it doesn't matter what kind of file you want to transfer, you'll have the speeds you want in no time. But how do you RAID USB keys?
Before we go into "How," here's why you might want to:
1.) Your unused USB keys are a waste of space and a constant reminder of waste
2.) You've always wanted a fast hard drive to work from on-the-go, but don't want to spend hundreds on SSDs
3.) The speed and capacity benchmarks per dollar spent are pretty amazing: $176 for 512GB that get over 250MB/s read and write speeds -- and that's if you buy your USB drives new (as opposed to using some you have laying around)
How to do this yourself in just a few minutes:
The first step is to decide how many USB keys you want to use. I recommend 2 or 4, but in any case, the process is the same. You'll want them to be the same size and speed to get the most out of them. As I said before, I saw a deal on some fast and ever-affordable 128GB USB 3.0 PNY USB keys (I bought two 2-packs at $80 each, or $40 per key for four total). This allowed me to get 512GB of storage for what ended up being just under $160. If you use your own USB keys, you just need to worry about the next step.
The build quality on this is surprisingly good -- the all-metal enclosure feels quite sturdy.Second, you'll need to connect them to your computer all at the same time. I have 2 USB 3.0 ports on my Retina MacBook Pro, so I got an external USB 3.0 hub to expand the number of available ports. While many hubs will want to be plugged directly into a wall, this one allows you to attach an additional power source to your other USB port. While this means both of my ports will be used (one for data from the hub, and the other for power), I still have four ports on the hub, which means I've still doubled my USB 3.0 port count in a still-portable package.
Now for the real magic:
I'm on a Mac, for starters, and Mac users have an awesome tool called Disk Utility, within which we can do what's called a "software RAID." You'll notice I reformatted my USB drives to Mac OS Extended (Journaled), but that's just because I'll only be using it with my Mac. The choice is up to you... but MS-DOS (FAT) will be necessary to use this with your PC.
Each of the 4 USB drives show in the left sidebar once you've connected them to your computer.After selecting the RAID tab on Disk Utility, we can use the Mac's software to virtually RAID two or more hard drives:
This can work for mirroring hard drives (for backup) as well, but I'm selecting "striped" so that the hard drives (USB keys, in this case) can be seen as a RAID 0 configuration. This means that data will be split between both USB keys for twice the speed (theoretically).
Once you select your options at the top, you can drag in all of the disks that you want to use in your RAID system (don't accidentally grab the wrong one if you have others currently connected that you DO NOT want to use, like my separate Promise Thunderbolt R4 setup that I definitely don't want to touch):
Click "Create" at the bottom right, and confirm your selection (continuing past this point will erase your hard drives and create your RAID set, so DO NOT have the wrong drive selected):
And that's it! You now have your RAID setup. Don't be daunted by the remaining and showing four drives. Just ignore them and pay attention to the new, larger volume ready for use. This is the only one that will show up on your desktop, too:
The nice part about this is that the computer will recognize the RAID configuration no matter which port you have your USB keys plugged into. You don't need to have them in the same spot every time -- they just need to both be in use to work.
Naturally, there are a few caveats to all of this:
1.) The speeds tend to top out after going from 2 to 4 USB keys. You can try to get an 8-port hub, but it'll probably need even more power, and you won't see the same significant gains.
2.) Using similar and fast USB keys also helps, naturally. But you can still increase your speeds to over 100MB/s quite easily with standard USB keys.
3.) If one drive fails, you've lost everything. This is always an issue with striped systems. But as long as you keep this in use purely for your live work, you should be okay. Just back it up every now and then. The good news is that a replacement "drive" will be easy to find and rather affordable. Reformat, start over, and enjoy your system as you did before.
4.) Speeds aren't the most reliable. While you can count on fairly quick speeds, they do vary greatly depending on the types of files you're transferring.
Below are some specifications based on my tests with the 4 PNY keys in single-drive, dual-drive striped, and quad-drive striped setups:
Single USB Key:
Transfer time for a 42.4GB folder (almost 16,000 individual files): 6 minutes, 27 seconds OR 109MB/s
Transfer time for a 7.84GB DVD Rip: 58 seconds OR 135MB/s
Blackmagic Disk Speed Test for single USB-keyDual-USB Key RAID Setup:
Transfer time for a 42.4GB folder (almost 16,000 individual files): 4 minutes, 2 seconds OR 175MB/s
Transfer time for a 7.84GB DVD Rip: 38 seconds OR 206MB/s
Blackmagic Disk Speed Test for dual-drive, RAID 0 setupQuad-USB Key RAID Setup:
Transfer time for a 42.4GB folder (almost 16,000 individual files): 3 minutes, 37 seconds OR 195MB/s
Transfer time for a 7.84GB DVD Rip: 32 seconds OR 245MB/s
Blackmagic Disk Speed Test for four-drive, RAID 0 setupDo you have your own DIY storage solution? Share it with us or let us know how this solution worked for you in the comments!
These USB drives are considerably much less reliable than a real SSD, are much slower, and with 4 in a RAID 0 configuration like that you only need 1 to fail and you lose all your data. In other words, don't do this.
He does state that possibility of failure, and I imagine he doesn't advocate using this for actual long-term storage, but just as a work drive system.
All of this complexity and size for something that performs similarly (or worse) than just buying a 2.5" hard drive, which I suspect could be had in similar capacity for the same price.
A 2.5" external SSD would generally cost more than the particular one Adam put together. Bare 2.5" 512GB SSDs can be found for, let's say, $200-225.
2.5" HDDs, of course, can be much cheaper. But they're usually going to get you up to the low 100s in MB/s. A mainstream 3.5" HDD could maybe perform close to his numbers, but still probably lower unless also RAIDed, and then you have all that noise, heat, and energy usage.
Wait... the cost of buying 2 of those USB drives plus the hub is equivalent to buy an SSD for that same size. SSD is much more reliable and cheap nowadays, almost a buck a gig. WHY?
Actually, there's ever-present deals for SSDs that bring the $/GB down to about $0.50/GB.
Well I thought this was an awesome solution/suggestion, but of course there are always going to be people who feel the need to talk shit. Of course you could go buy a SSD, but this is a workaround, especially if you have a bunch of extra thumb drives sitting around.
I swear, for a website that delivers news, tips, tricks, and techniques, ALL FOR FREE, there sure are a lot of miserable fucking people here who have to naysay everything. If you don't like or don't agree with some techniques posted on here, keep it to yourselves. No one cares to read all your fucking negative thoughts/comments/criticisms. Find some happiness in this world before you die.
They are the same people that don't actually go out and shoot stuff but rather complain about things on the internet. Which is fine as it leaves more work for the rest of us :P
The problem is, this is actually a bad idea; a waste of time and money that's eventually going to crash.
And by your logic, we shouldn't complain about FOX News or reality television because, you know, it's FREE!
Check your anger somewhere else, dude.
Hah!
Anthony, I think that commenting on articles generally invites a healthy debate. People are opinionated, and like to be heard. It also allows for the editors to receive reader feedback, plus, the number of comments is a good measure of article's impact and popularity.
I wouldn't get too wound up on negative comments. You can just as easily avoid scrolling down into the comments section if you find them this upsetting.
It's a fun idea and thanks for going through the trouble of trying it out and writing it up.
What's the real-world use case for this? I'm trying to figure out who needs very large, relatively fast storage space for which data security is not a big concern? For videographers' temporary working files? I don't edit video but I thought that the connection speed to original source files doesn't have a big ramification on, say, Final Cut Pro performance, which is why so many professional users have been fine storing their projects on drives connected with Firewire 800.
Have you tried RAIDing slow USB sticks in an array? E.g. sticks that, solo, are only capable of ~20MB/s throughput? I see a lot of these and I have a hard time believing RAID 0 would make a ~5X increase in speed. It's surprising how many sticks, even ones that are USB 3.0, are that slow. The ones you are using are pretty fast, solo.
Anyway...interesting exercise and a fun idea. In the end, even if I found a similar deal on large, fast, matched USB sticks, I'd rather just shell out the extra $50 to have a single 2.5" SSD to avoid having 4X the risk of failure, to have a neater physical solution and, of course, for that solution to be potentially a couple times or more faster.
It's just not worth the savings to risk losing even an hours' worth of work.
I agree... This was more of a proof-of-concept if you have these laying around anyway. What's an additional $16 when you have some spare USB sticks in a drawer?
I posted some real-world speeds as I transferred a 42GB folder and a DVD rip... They were fairly true to the Blackmagic tests. With slower drives, you can't get THAT fast, naturally. But it's a good way to speed up your live work for photo OR video work. If you're working in FCP, there are still constantly little things you want to render (in my experience) that will benefit from an increase in speed. And for photos, your previews will load faster in Lightroom.
Naturally, I still wouldn't recommend buying USB sticks to RAID them as a solution. I'd go the full, actual SSD route (or I'd get the seagate 4TB backup fast that's stupid as a backup drive, but awesome as a live-work drive for the price -- and I do have and use that one).
It's a shame that Seagate 4TB Backup Plus Fast doesn't allow one to switch it from RAID 0 to RAID 1 so that one can either use it as fast external storage or highly portable, high-capacity backup.
And, yeah, the speed is pretty stunning for 2.5" HDDs. If this is possible using striped 2.5" HDD, then one could probably build up a very fast, very cheap portable swap drive using smaller and cheaper drives and put them into a 2.5" dual bay RAID enclosure.
Oh, and I can see how, maybe very unsecure storage could be used by a videographer as a temp work drive (if they don't value their work time highly), but there's no way I'd use it in conjunction with Lightroom in any fashion -- as storage for my photos, of course, but also not as storage for my Lightroom catalog.
My Lightroom catalog sits on an SSD drive of its own in my desktop and backups of the catalog are stored on the same drive and mirrored on another drive. Lightroom catalogs are small enough that a 60GB SSD is far more than sufficient and they're so cheap now that there would be no point in trying to save on that cost.
As for the lightroom catalogue and images... I keep my catalogue on the local SSD I have... But I do put all my images I need on the go on the backup fast... Could I lose them? Sure... But I have it backed up three times over anyway, so I don't care too much... If I put something new on there for any reason, I always keep it in at least one other location until I can get it backed up. Seems to do the trick...
If you're synching the Seagate with other drives regularly enough to satisfy your need for security, then that sounds like a nice way to carry around your archive. If one doesn't need to actually carry such a large amount of photos, though, I'd strongly recommend to a person that they should use a portable enclosure that's capable of RAID 1 and run it that way.
If it's an issue of speed, unless you are shooting massive volumes of images, you can probably keep the image files of photos you're working on in any particular session on your internal drive, and then transfer over to the archive drive when done with edits.
But everyone has their own varying tolerance for risk to data, so to each their own.
Cool hack, man!
This is such a bad idea :P
RAID 0 is not ideal, nor is USB for filetransfer, nor is a simple flashdrive as storage.
Most importantly - Thumbsticks are not meant for use as a harddrive. The high ammount of read/write operations will wear it down in no time.
Can you rig that sucker up as a RAID 0 array? That would be fun!
This is a brilliant hack. But I have concern on realibility of thumb drive on raid. Do they have write leveling?
The topic is wrong... it must be like "how to ged rid of your data in a very efficient way?". Sorry it looks cool but dont try this at home, the USB sticks are not build to perform such write/read operations in that way.
a complete bullshit article from a guy who does know nothing about flashdrive technology.
this is completely unreliable... i would not even use it for porn stuff.
neat experiment. I don't really care if it is reliable or not.
A modern twist on...
http://www.wired.com/2009/05/five-disk-floppy-raid-4mb-of-blistering-fas...
This article is great if you take it as intended - you have existing USB thumb drives laying around and want to get "some" use out of them.
If you were actually planning on buying new drives to do anything other than a proof of concept, please don't. You'll be wasting your money in the long run. There are several better, smoother and safer options than this if you want speed for a live drive situation.
We are a group of volunteers and starting a new scheme in our community. Your site offered us with valuable info to work on. You’ve done an impressive job and our whole community will be grateful to you.