May 13, 2004 edition


Storage Growth Continues Unabated
Consider DFS and X1 To Help You Manage Growth


I can clearly remember the time when my 2.1 Gigabyte Western Digital Hard drive was the envy of my college dorms. Most of my peers were still using "out-dated" 540 MB drives. These days, at any given time, I store over 200 Gb of data on a network attached external disk. In case you're wondering, digital media makes up for less than twenty percent of my space usage. In fact, I haven't even needed to transfer my digital pictures from an 8 Gb compact flash card that was given to me by a friend who works at the card manufacturer's headquarters.

This morning, I read that Hitachi has brought a 400 gigabyte drive to market for about four hundred dollars. Wow! Now this is a dream come true for someone like myself. I have always wanted to have more storage so that I could mirror data without having to worry about backups. Even besides mirroring, I prefer to keep as much data on disk as possible. In fact, even when I have the original media for software, I prefer to keep it on disk for faster access and installations.

I am sure that there are countless others like myself out there whose eyes lit up with the announcement of this disk being brought to market. The exponential growth of storage capacity within a single chassis as well as the overall explosion of storage space within corporations begs the question: What is it really that is driving the storage growth? Surely it cannot simply be cost or bragging rights. Even though it would greatly enhance my "coolness," I would be hard pressed to even contemplate purchasing an 8 Gb CF Card at its current price tag of several thousand dollars. And, I certainly wouldn't buy 10 terabytes of disk space even if it were dirt cheap. In other words, what are the industry or business drivers behind this growth? Also, what are some of the ways that we as Information Technology professionals can best present this data to our customers?

A little bit of research yielded some answers. The first and most obvious answer was E-mail. With the volume of messaging (not to mention spam) increasing at a breakneck gradient and email providers offering more and more storage, this was quite logical. In fact, industry analysts predict a 40 percent sustained yearly growth in email until the market matures and is saturated. Then, there is digital media. Digital Video Recorders, Digital Cameras and Digital Jukeboxes are churning out gigabytes upon gigabytes of movies, memories and music.

Other storage growth drivers in the corporate setting include E-Commerce and Database Applications and Business Data including everything from Microsoft Word files to engineering test result logs. Intrusion detection and security audit logs such as web server, firewall and data access logs are another source of data. One of the largest sources of data today is the health industry. Medical Imaging is creating immense storage needs. Over 2 billion medical images are captured each year including X-Rays, CAT Scans and Mammograms. And at over 8 MB of space required to store an X-Ray in digital format, that's a lot of data right there! The Health Industry Portability and Accountability Act (HIPAA) of 1996 put forth regulations that also impact the worldwide consumption of storage. As per this Act, there are specific standards for most health transactions. As more and more health providers turn to electronic transactions, the storage growth in the health sector is just beginning to take off.

The Education and Learning sector is another factor in this whole scenario. With online classrooms and e-learning courses on the increase, it is no wonder that this is a growth area too. The media sector was an obvious source as well. More and more publishing is becoming electronic and less paper oriented. In fact, I would venture to guess that you are reading this very text on a computer screen somewhere.

By the time that I was done, I couldn't think of a single business sector that was not rapidly increasing its consumption of storage. And being a geek at heart, it led me to the next most important question. What can I do to deal with this? How can we better manage all of this data?

There are several technologies out there that can allow us as system administrators to ease our customers' pain and lessen the confusion. One technology that I am particularly fond of is Microsoft's Distributed File System (DFS) which was first made available with the Windows 2000 server line of operating systems. Using DFS, A single UNC share, or a single drive mapped to a share can present data from any number of file servers or network locations. Whereas we cannot go into the inner workings of DFS for the purposes of this article, I would like to mention that it works a lot like the UNIX automount feature. A detailed overview of DFS is available at http://www.microsoft.com/windowsserver2003/
techinfo/overview/dfs.mspx

One of the most frustrating limitations for windows on the client side is that one cannot assign drive letters to more than 26 locations. If you are like me and have a floppy disk, two disk drives and a CD drive, this leaves even fewer letters available for network drives. DFS is an excellent mechanism to simplify all of this manual mapping. Using Active Directory policies, login scripts or even a simple batch file, it is very easy to map the DFS root from a users system. Anything else from there on down is only a few mouse clicks away. If you haven't ever considered DFS, I would urge you to give it some consideration.

And what about searching through the gigabytes of un-indexed email, attachments, files and other data at home? Well, you're in luck because I am going to tell you about the best search software I have seen to date. X1 is simply the best search software available in the market. I have tried Bloomba, DTSearch and several other index-based search applications but nothing breaks the "found barrier" like X1. It finds information before you are even done typing out the keywords. For more information about X1, visit www.x1.com.

There's no end in sight to increasing storage needs at both the end-user and corporate level. Storage technologies and management tools are improving but whether you're managing your personal data storage needs or the data storage needs of a large corporation, get ready to spend more time planning, administering, and troubleshooting your storage solution.

Athar A. Khan
Executive Information Technologist
atharkhan@email.com

Consolidate Storage Infrastructure | Review WANb Application Case Studies | Integrate Fibre Channel and IP SANs


Learn how to integrate Fibre Channel and IP SANs across multiple sites in this FREE white paper from McDATAŽ.

Download a FREE copy today!




Discover the benefits and challenges of integrating iSCSI, FCIP and iFCP protocols with FC SANs.
Download your FREE white paper today!




Review important requirements, deployment examples and case studies in performance and protocol tuning in this FREE white paper from McDATAŽ.
Download your FREE copy today!