Is a RAID array right for you?


There has been much written about storage devices and requirements for imaging professionals and there are many types.  There is internal storage, external storage, and off-site storage.  Lets briefly discuss each type.

Internal Storage:  Internal storage is the hard drives, DVD drives, SSD’s, and RAID arrays inside your computer case.  Everyone has different ideas of the perfect system but most agree a fast 10,000 RPM SATA II drive such as the Western Digital 150g or 300g Raptor to be ideal for a fast system drive.  A system drive holds your operating system and programs.

WD Rapter

An imaging computer using Photoshop and/or other imaging programs also needs a “scratch” disk for temporary files created during processing.  A small high performance drive speeds performance, but I’ve found Hitachi’s 1tb 7200rpm SATA II drives just as fast as the 150g WD Raptor so now I have three of these 1tb Hitachi drives in my case running as separate drives.  I use as a “work” and “scratch” drive, it holds my current work files.  When the project is finished the files get moved to an internal storage drive.  I use another to hold my most recent image archives, and another as a destination drive for Norton Ghost 14 to backup both my system drive, my work drives, and my recent archives.  So far this is working fine for me.

Hitachi SATA II

External Storage:  External storage are devices such USB or Firewire drives are very popular and getting less expensive for quality units.  These are also external RAID arrays of various types.  I use Lacie’s Big Disk Extreme Plus because they are both reliable and they have a triple interface, USB, Firewire 400, and Firewire 800.  These are quality very well built units with internal cooling and RAID capabilities which can be run 24/7 if desired.  I use these for backing up my archives and moving large blocks of images between workstations when I need it done faster than my gigalan network can do it.

Lacie Big DIsk Extreme Plus

Off-site Storage:  Off-site storage simply refers to any storage device not physically connected to your workstation/computer but which can still be accessed via your internal home network or via the Internet.  I use Hammer Technologies MYSHARE storage devices as both NAS (network attached storage) and as a FTP server (file transfer protocol) and they’ve been very reliable.  I back up my system drives from my workstation and laptops using Norton Ghost 14, my archives and work disks, and just about everything else.  This also makes them available to me from anywhere in the world via my personal FTP, or to upload images while on the road as a secondary storage device.

Hammer Tech MYSHARE

I’ve learned to rely on a multi-layer approach to storage, each of the above devices has their purpose and I wouldn’t want to be without either one of them.  At the end of each year I buy an appropriate size internal SATA HDD and move my years work images and catalogs to this drive and then carefully label and wrap it before storing it off-site in a fireproof safe.  ALL my archives are also safely kept on an internal RAID 50 array and this is the topic of this blog entry.

My luck with RAID arrays has been very bad and I hope this is about to change.  My main workstation is water cooled out of necessity since I live in Bangkok and it’s very hot here.  I had a 4tb RAID array driven by a Promise Technology’s EX8350 Supertrak RAID controller configured in two 1.5tb RAID 5 arrays.  My archives were kept on one, and backed up on the other.  This used to be my “backup” and this was a major mistake.

One day a $2.00 plastic fitting on my cooling fluid tank broke and wiped out 12 internal hard drives, 8 of them 500g SATA II drives I was using for both RAID 5 arrays.  It wiped me out.  3 years of travel to exotic South East Asian locations and the images I worked so hard to capture, tens of thousands of dollars in travel expenses, and three more than half finished books on the area, all gone for a $2.00 fitting.  As you can see from my multi-layer storage approach above I’ve learned my lesson. (I also replaced all plastic fittings with brass fittings)  I did manage to save about 30% of my work by buying some new 500g drives of the exact make and model, and using the new low level and power boards managed to find 3 disks intact which I ran up in a RAID critical mode and managed to save about 30%.  Thank you for small favors.

Since then my Promise EX8350 card had failed me three times, each time I lost more data, but because I now backed up on several devices in several locations data loss was kept to a minimum.  Still, I was constantly fighting to keep the RAID array working properly within the Windows environment and the cards failed after 3-12 months.  Simply put it was not a reliable method of storage which I could count on.  Promise Tech to their credit promptly replaced the controllers and offered good technical advice, but still I was never able to use the RAID arrays reliably and even if I could I found them too slow for real time use.

This last time my EX8350 Supertrack Controller went bad Promise Tech did the right thing and shipped me their new EX8650  SAS/SATA RAID controller which is their newest fastest and best RAID controller yet.  Several on-line reviews reveal this new controller to not only be very reliable, but also very fast.  It can utilize the newest 15,000 rpm SAS hard drives if you want speed and performance, or even Seagates new 1tb SAS 7200rpm drive if you just need fast storage, or standard SATA II drives for economy.  The new EX8650 will even allow you to run two arrays from the same card, dividing it’s eight ports, with one array hosting fast 15,000 rpm drives for real time processing and the other array hosting large storage drives for archiving.  You can divide the ports in any order.  This sounds ideal!

EX8650 RAID Controller

My new card will arrive next week and the lead engineer at Promise Tech USA will be getting back to me soon, hopefully with some recommendations on compatible hard drives and motherboards.  The EX8350 only required a PCIe 4x slot, but the new EX8650 requires a PCIe 8x slot so it appears I’ll be upgrading the motherboard of my workstation.  I won’t mind this relatively small expense, or even the more major expense to purchase a combination of 15,000 rpm SAS drives and 7200rpm large SAS drives, IF I can make my system perform as advertised and as they’ve done in the review.

I suspect it will take me about two weeks to order in my new motherboard and SAS drives, rebuild my workstation, and use it enough to update my experiences here in my blog.  I’m looking forward to adding another layer of storage which can “do it all” on it’s own.