Some time ago I had read an article about how someone made a RAID array out of some unused USB flash drives in Windows 7. I found it very interesting and recently realized I had a few USB flash drives I could use to attempt the same thing. After reading this article, and it hinting that using it on a "PC" (Windows) would be possible, I decided to try it out for myself...
Tip for Downloaders
A friend of mine and I were catching up on our lives over a phone call and we segued onto the topic of downloading. After a brief discussion he mentioned that his friend had told him of a good idea, which is really just good practice. If you are downloading small files (not gigs upon gigs, and even if so), it was smart to use a USB flash drive as the designated area for downloading. On browsers and download managers you can tell each where to write a saved file to. By default, these programs usually download to your C drive, or whatever drive your system is on.
You see, every time you write a file, it puts a bit of stress on your OS drive. Even if you have partitioned your one drive into two, and are downloading to the second partition that does not contain the system, it is ultimately still putting stress on the OS drive because it is truly one drive. While a file or two really does nothing, if you download often, you are essentially draining the life of your drive bit-by-bit.
Instead, you should be adjusting your downloads to go to a USB flash drive. USB flash drives are generally inexpensive, so if they die it's easy to replace. You can use an external drive, but those are normally a bit more pricey, so a USB flash drive is ideal for your downloading means.
This does not mean that your drive will die anytime soon, as most of us don't employ this trick. However, it will help you keep that drive alive that much longer by doing so.
A Bit of Background
Ever since that phone call I realized how right my friend was (actually his friend). So, I made sure to use a USB flash drive for any and all downloads. I initially used a Toshiba 32GB USB flash drive that cost around $30-$40 at the time of purchase, which worked just fine. Once in a while I would run out of space before cleaning it out and decided to upgrade when I could. I then bought a generic 64GB USB flash drive that also worked great, and cost about $10 (due to a promotion). I soon began out of room again, which led me to a generic $10 128GB USB flash drive from China. It worked good to start...
I soon saw issues with it. While my 64GB would sometimes cause a slow down to the point of 30-60 seconds before opening folders inside it, that was survivable. However, the 128GB seemed to always work good after a format. But when I'm doing too many things with it at one time, it will slow down to minutes upon minutes! Testing it showed that it was working as well as could be expected. Not wanting to give up on my 128GB, I decided to try to put some of these USB drives into an array.
Note: All of my USB flash drives are USB 2.0.
The Mac Article
In the second article I read it explains how the author bought 4 PNY 128GB USB 3.0 flash drives and put them into a RAID using OS X. He did not have enough USB 3.0 ports available, so he bought a USB 3.0 hub to attach them all. Using Mac's Disk Utility, he chose RAID and placed each PNY drive into the selection area. He then created a RAID and unleashed some real speed! After doing some tests, he found that the optimal amount of USB flash drives (for speed) are four.
Food for Thought
There are a couple of things to note about this process. The author states that to use the RAID on a PC you would have to use FAT (FAT32) in order for a Windows system to recognize it. While FAT32 is usable, if you plan to move any files onto the RAID larger than 4GB, FAT32 will be unable to do this. And while you can get a Mac to write and format to NTFS or exFAT, you do not have the option to use these formats for RAID.
The author also bought all the USB flash drives, yet, this would be counter-intuitive. The cost for all of these would equate to purchasing a decent SSD, which should be able to top the speeds he gets even if used as an external drive via USB 3.0. On top of this, a SSD would be a much better solution.
RAID Knowledge
There are many types of RAIDs, but the ones we are concerned with are RAID 0, RAID 1, and JBOD. These are the options any OS X will give you in Disk Utility.
RAID 0 is also known as striped RAID. It is all about speed. One thing many people get wrong, including myself for some time, is that RAID 0 does not require same-size disks in order to be used. However, it is common practice to buy and use identical sized drives as the RAID can only be as big as your smallest drive. The downside from RAID 0 is that if one of the drives dies, then so does the RAID, leading to a loss of data.
RAID 1 is also known as mirrored RAID. It is all about data redundancy. For a RAID 1, you do need to have identical size drives to use it. In addition to this, your drive size will only be half as much as the total amount of storage space for all drives. For example, if you have 4 100GB hard drives in a RAID 1, it will show one drive as 200GB. Likewise, if you have 2 50GB hard drives in a RAID 1, you will only have 50GB. The upside to a RAID 1 is that if one drive fails, the data on it will still be present on the other drive(s).
JBOD (Just a Bunch Of Disks) is also known as concatenated RAID. This is normally used when you have numerous drives of similar or dissimilar sizes. There is no advantage to using this type of RAID except if you want a bunch of drives to act as one drive. Maybe you have a 1GB, a 1.5GB, and 4GB drive. You need to store a 6GB video, and using JBOD would be a perfect example of how to accomplish this task.
So, just in case you're wondering how this would apply to the aforementioned article. The author is using a striped RAID for speed, so if one of those PNY drives dies, he loses all his data. If he were to use a mirrored RAID, he would not get a speed increase, and the setup would be that much more costly as he would only be able to use 256GB of that storage. He could use a JBOD, but then he would have no data redundancy or speed increase, just a large 512GB drive (that would cheaper to buy than create).
Mac on Win Pt. 1
So the requirements to do this are an OS X computer and at least two USB flash drives.
I no longer have a Mac, but I do have OS X on VMWare. This works great for me and will see any USB flash drive I put on. Unfortunately, VMWare will not recognize a USB hub, so I needed to directly connect my USB flash drives to my desktop.
As for USB flash drives, I have quite a few, but I decided to use my generic 128GB and 64GB USB flash drives. I measured them on write and read speeds beforehand and they were approximately the same.
Once I booted up my OS X and it recognized my USB flash drives, I went ahead and started experimenting. Immediately, I ran into my first problem. I had chose a FAT RAID and was trying to drop the drives from their volume names into the RAID selection area. A message would come up stating that I could not put them into a RAID because they used MBR instead of GPT (See my "USB 3.0 Hub Drive Fix..." for more information on this). I did not realize at the time that all I needed to do was to actually select the generic name (with the GB size) above the volume name and drop that into the RAID selection area to make this work.
Instead, I went ahead and formatted each USB flash drive as FAT in Disk Utility. I tried the (wrong) method of dropping the volume names again into the RAID selection area, and was met with the same error. I then decided to go back to Windows and change it from MBR to GPT.
Mac on Win Pt. 2
Back in Windows, I opened an elevated command prompt and started DiskPart (do not use this program unless you really know what you're doing). I went ahead and changed one USB flash drive from MBR to GPT and reformatted as NTFS.
I then went back into OS X to see if this would give the error message before converting the other USB flash drive. This time I could drop the volume into the RAID area. So I went back into Windows and did the same conversion for the other USB flash drive. I dropped the second volume into the RAID area and started the RAID.
Immediately I got an error. Confused, I tried again with the same results. So I decided to see if a HFS+ (journaled) RAID could be done, just to be sure OS X could at least perform that operation. It worked.
In Windows it is possible to read and write to HFS+ drives, but a RAID array is another story. If you have MacDrive 9 Pro, you can apparently read and write to RAID arrays formatted with HFS+. I downloaded the trial and disconnected the drives from OS X. MacDrive 9 Pro would acknowledge there was a RAID array present, but it would not show anything. I could see the two USB flash drives from MacDrive's format utility, and in Windows' Disk Management, but that was it.
Another program to just read HFS+ drives is TransMac. While it would recognize that both of the drives were in HFS+, it could not do more than that. It would throw out errors whenever I tried everything. I did notice that it seemed that the RAID array was connected to the 64GB USB flash drive...
Mac on Win Pt. 3
I connected the USB flash drives back to the OS X, they showed up again as a RAID. I went back into Disk Utility and then discovered I could make the FAT RAID if I used the generic names of the USB flash drives instead of the volume names.
I started the RAID and it finished successfully. It functioned perfectly on OS X, and I even threw on a file just in case there was something weird about it being empty when showing in Windows. I disconnected the drives from OS X and went back into Windows.
Again, the drives did not show. They would show in MacDrive (as just USB flash drives) and Disk Management, but not in Windows Explorer. I decided to try to see if I could do anything in Disk Management. From here, I could see the two drives, and the RAID drive it created.
I decided to delete the RAID volume and see if I could reformat it in Windows. It did delete, but it could not be reformatted. Since it was pointless to try anything further, as there was nothing else to do, I decided to delete the volumes in Disk Management...
Mac on Win Pt. 4
My mistake was trying to do anything in Disk Management. While it does come in handy, it somehow screwed up both the USB flash drives to the point that when connecting back to OS X, VMware would give errors when trying to connect them.
After a few tries (and reboots) I decided to use DiskPart. I deleted the volumes and converted both the USB flash drives back to MBR. I then was able to use Disk Management to reformat them, but the RAID volume would not disappear...
I performed another reboot in order to get the RAID volume to disappear from showing. I then used a third-party tool (MiniTool Partition Wizard) to reformat both drives as NTFS. I did this because Disk Management never seems to use all the space available, but third-party tools will allow this.
I did seem to have a residual empty volume after I restarted my computer, but I just used DiskPart to delete it and reformatted the USB flash drives again with my third-party tool. This fixed the issue.
I went ahead and assigned different letters in Disk Management and made sure they worked before stopping as it had been an all-day event. Most of this ordeal is just summarized with frustration and agitation put aside.
Win on Win Pt. 5
So the next day I decided to see what else I could try since the Mac method was not working for me. I finally found an article where someone had used VirtualBox (like VMware, but completely free) to access their USB disk directly from the host OS (known as accessing a RAW drive), which would show the USB flash drives as fixed disks instead of removable. They were able to use another Windows OS in Virtualbox and with this technique created a RAID from two USB flash drives. For them, when they disconnected the drives from VirtualBox and reconnected to their host Windows OS, the drives showed up as "foreign" disks and they were able to allegedly complete the RAID...
In VirtualBox, this is somewhat a technical and tedious process. Fortunately, in VMware, it's a bit easier, or so I thought.
I first installed a copy of Windows 8 on VMware. I then upgraded to Windows 8.1 (which was troublesome due to an issue with the Windows Store and Windows Update). I then loaded up my two USB flash drives and checked their functionality.
The USB flash drives were working, so I turned off Windows 8.1 in order to setup the access to a RAW disk. I first selected SCSI, which is recommended by VMware, and selected the correct USB flash drives (as they are only listed as numbers, and I have a ton of drives connected). I started up Windows 8.1 and waited. I got the usual message that the drives were being disconnected from the host and then got an error. The system would boot but with the regular USB flash drives as removable.
Win on Win Pt. 6
I went through the RAW disk process again, using SATA, and tried started Windows 8 again. This time the system couldn't even get to the startup screen... I took some time to peruse the Internet and
found out my first mistake. Apparently, you have to add your RAW disks,
close VMware, then re-open VMware as an administrator (right-click on VMware
and "Run as administrator"). That will allow the RAW disks to run properly.
Following this tip, I tried SCSI again, but to no avail. I recalled that sometimes IDE was needed for certain situations. So I tried IDE for the RAW disks. I used only one USB flash drive and actually got it to boot as a fixed disk! I shut down Windows 8.1 and tried the second drive, but it wouldn't boot...
I thought maybe it was a size or generic drive issue, but then realized my second mistake. The USB flash drives were already configured to automatically show up in Windows 8.1, so there was likely some conflict in Windows 8.1 during boot up. I deleted the RAW disks from VMware and booted up Windows 8.1. After ejecting both USB flash drives, I shut it off.
I made both USB flash drives RAW IDE disks and Windows 8.1 finally booted up with both drives as fixed disks! Success, but how much? I really had two options in Windows 8.1. The first was to use Disk Management and try a RAID 1, or I could use Storage Spaces and make what can be considered a RAID array. I opted for the first option as Windows 7 does not have Storage Spaces, so I doubt that it would be recognized by Windows 7 even if Storage Spaces were able to implement a RAID on these fixed disks. There might be a chance if I was running Windows Server 2012...
Win on Win Pt. 7
So I setup a normal RAID 1 from within Windows 8.1's Disk Management. It worked without issue and I now had a 64GB RAID array created by Windows. The hard part was done, but now the real test was about to begin.
I shut down Windows 8.1 and checked Windows Explorer. The two USB flash drives were shown, but they were inaccessible, and there was definitely no RAID array present. I checked Disk Manager and the two USB flash drives were there, but no third volume that would indicate a RAID was made from the two.
I had another trick up my sleeve, but I had spent some time before on it without success. Essentially, I needed to trick the USB flash drives into thinking they were fixed drives on my OS. There are plenty of tricks out there, but the ones I was able to perform didn't work. One was just impossible due to how Device Manager reacts to uninstalled drivers, the others always had me perform a reboot. Once done, the system would break and require a restore point in order to be fixed. Even if I had accomplished this trick, I would still need to convert the disks from basic to dynamic disks (read "Toshiba Canvio..." for more information).
Win or Lose Pt. 8
My final hoorah was using my two Toshiba 32GB USB flash drives. Oddly enough, they are about 100MB off from one another... However, I thought, why not try similar sizes as the original article had.
I started off with deleting their volumes from within Disk Management. No problem there. But now it was time to make the disks "active" (dynamic), and this I already had experience with so I could only presume the worst. There was no option given by Disk Management, so I tried the other two recommended options. The first was using MiniTool Partition Wizard, which should give you an option to convert from basic to dynamic disk if applicable. No such option came up. I then tried DiskPart, but messages came up telling me the USB flash drive was not an ordinary basic disk and therefore could not be converted to dynamic disk. Again, I had already tried these methods to convert the USB flash drives from basic to dynamic disks before, so this was no surprise.
My final attempt was using a special hex editor to open up and read/write to a USB flash drive. I needed to change a certain value from NTFS to a dynamic disk. I know, a bit odd since those two things aren't of the same subject matter, but from what I have read they use the same location for their values. The change was simple, but the risk was big since it could make my USB flash drives inoperable. I made the change on one USB flash drive and saved it. I ejected and reinserted my drive, which did not show up in Windows Explorer. This could be good or bad. I checked Disk Management, and saw the drive but only as a removable disk. I went into DiskPart which saw the drive twice, one as an error. I tried to change the USB flash drive back to a basic disk, which would tell me that changing it to dynamic had worked. DiskPart would not let me change it. I then reformatted the USB flash drive, took it out, and reinserted it in order to get it show in Windows Explorer again.
This brought me to the end of my journey.
So What Does This Mean?
This means that Windows cannot make a software RAID from USB flash drives. While it can be done within a virtualized environment, it will not carry the actual RAID array over to the host OS.
There are still some tricks out there for those who are willing: If you are on a Windows XP OS you have the ability to install CFAdisk as a driver to a USB. This requires going into Device Manager, finding the USB flash drive, clicking on Properties, and "updating" the driver manually to the aforementioned drivers. It should then change the USB flash drive to a fixed drive. To get back to a removable drive you need to go back to the Properties in Device Manager and uninstall the drivers. Windows XP should then reinstall the regular driver for it. However, allowing this to create a RAID is unknown to me.
Supposedly, there is a method to change FireWire devices to be seen as fixed disks. This entails a registry hack. Yet, I don't know anyone who still has FireWire ports, letalone devices that use FireWire. Again, I have no idea if this would work for a RAID array, but I feel that it could for external hard drives or SSDs.
Another, much more intricate, and pricey measure, is by using actual hard drives. Apparently you can put two hard drives (SSDs?) into a desktop system and RAID them. You can then put them in external enclosures and they will purportedly still be recognized in a RAID formation when connected to a computer. While this is better than buying an expensive hardware RAID setup, it seems like a lot of work.
What's more is that for a USB flash drive, this would be huge risk and a lot of work. You would need a USB female to SATA male adapter, which I don't exist. You could try a USB female to USB header cable, but that will only be seen as a USB connection. And if you could force a RAID, you wouldn't want them just hanging in your system, especially with all the heat accumulating. This means you need an external enclosure, which means you have a USB flash drive connected to the SATA adapter (if it were possible), that is connected to the external enclosure, which has a firewire or eSATA. I included firewire because of the trick above, but USB won't work as I have tried and failed with external drives connected via USB 3.0 (see "Toshiba Canvio..."). A RAID cannot be done over a USB connection for external drives, from what I have read. At the end of the day, even with all the right adapters, it may not work at all.
Another solution where the opportunity for a USB flash RAID may become possible is using the SanDisk Cruzer USB flash disks. Initially, when they came out, they would connect to a Windows OS as a fixed disk. This created an uproar in the user community as there was no way to eject it, causing data mishaps. SanDisk stated they had done so because it was required of Microsoft's licensing in order to officially add support for Windows 8. They have since changed their tune and made newer versions removable. If you happened to get a couple of these, you might be able to actually RAID them with as much effort it takes for actual hard drives or SSD's. To take this thought a bit further, and make it more complex, we could attempt to use an external enclosure that has inputs for a micro USB device. This creates a whole host of other problems that are unnecessary to state since it just won't be possible.
Note: It is not recommended to RAID SSD's. From the research I have seen it will at first give boosts in speed, then it will soon degrade past the point of what one SSD should be doing. In other words, a single SSD will maintain its high speeds a lot longer than if it is in RAID array.
There is still the possibility of Storage Spaces, which is included in Windows 2012 and Windows 8+. Microsoft may be setting up to do away with RAID done through Disk Management, so maybe they will allow USB RAID at some point. While I have not tested this with Storage Spaces, as I have no host OS that supports it, I do not feel the results would be any different. I do know that Storage Spaces did not even see my USB flash drives in Windows 8... And even if it could be done through virtualization, I have a feeling you still will not get it to come up on your host system.
Lastly, there was trying to use identical size USB flash drives. Now, while my USB flash drive brand was the same, the actual storage size was not (for whatever reason). One was purchased a few years before the other, so that could be a potential reason. Regardless, as simple as the article made it sound, it did not work. I even made the two USB flash drives logical drives at one point (just in case), and still came to the same unsuccessful conclusion. Now, there is a very slight chance that they need to be identical USB flash drives, but I doubt it would matter. The article is from 2010, so the only logical assumption I can come up with would be that somehow the actual USB flash drives being used matter (or it was just author writing something exciting they hadn't actually accomplished). While I can't wholeheartedly admit to that, I can neither contest it as the drives used were a Geek Squad brand, that is no longer sold; so testing with the same hardware cannot be done. One final thought on this is that maybe the system was 32-bit. I use 64-bit, but again, this is just another thing I doubt would matter.
So, you pretty much have only seven options to do this. The first is to use a Mac or Linux. It evidently works and is not hard to setup (on a Mac). As a side note, Macs can also use a special PCI Express card that has RAID integrated and USB 3.0 ports, whether it works or not I have yet to discover. The second option is to perform it in a guest (virtualized) OS of Windows and only use in that environment. The fourth is to use one of the tricks I mentioned, which it seems only Windows XP can do. The fifth option is to find and buy the SanDisk Cruzer USB flash drives that come as fixed disks, but that will become increasingly difficult as time goes on. The fifth option is to use the USB flash drives designated in the original article (if you can find them) and try your luck. The last option is to try to use a hex editor, or a program, that will change a USB flash drive to a fixed disk instead of removable.
I should mention that Super Talent is coming out with the USB 3.0 RAIDDrive, which is supposedly already in a RAID array. But this really takes the fun out of trying! And I'll also add that I have no experience with Thunderbolt, but at those speeds and prices is it even worth trying?
What Did I Use?
I used Windows 7 SP1 x64, Windows 8.1, and OS X Mountain Lion. Again, the USB flash drives I have are USB 2.0. MacDrive 9 Pro and TransMac was used, while HFSExplorer is also another free alternative for viewing HFS+ drives. I also tried the free drivers from Seagate (through Paragon) to view HFS+ on a Windows system. These drivers will force you to uninstall MacDrive first, likely since it is also a Paragon product trying to perform some similar functions.
If you are a Windows XP user there are a set of drivers from Apple (meant for Bootcamp) that many people have said allow them to see HFS+ formatted drives. You can get the official drivers from Apple here.
And...
That's really it. I have become dejected that only a Mac will allow this. While a USB flash RAID is not a widely sought solution, I personally would have some use for such a technique. There is no way to make comments on the original article for Windows 7. Yet, the author can be emailed from ZDNet as he is a contributing editor for ZDNet articles. I have asked a few questions and will hopefully get a response, but it has been a matter of days with no reply.
As for my 128GB USB flash drive, I was eventually able to mitigate some of the lag. You can read about it here at "USB Flash Drive Getting Sluggish?..."
No comments:
Post a Comment