By Sarah Gain
Regulatory compliance has put increasing pressure on enterprises to shield corporate data, check its integrity and safeguard network reliability. However, in order to demonstrate information control, organisations must be able to enforce and monitor policies at every point.
|~|CA-and-Salalah-BODY.jpg|~|Salalah Port Services’ Duca believes as companies move towards the paperless office ideal, the generation of data increases. What a business might save by implementing a new ERP solution, it can easily then spend on expanding its storage capacity.|~|Research by the University of California, Berkley, attempts to answer the question of how much digital data currently exists worldwide. The study defines magnetic, optical, print and film as the four types of physical media where new data is stored.
In 2002, these four media produced a total of approximately five exabytes of data. In the name of operational transparency, legislation across all verticals means companies are required to store vast quantities of information for longer periods, often in high-availability environments. Dubai Municipality is all too aware of what this means for its storage infrastructure. “All our different departments have a lot of data stored. The volume of this grows rapidly and if it is decided that one single piece of data is needed, then we must be able to produce it at once. It is a serious balancing act,” says Abdulla Ali Al Madani, CIO of Dubai Municipality.
At the same time, in the Middle East in particular, the very nature of the emerging market means the swift growth that can be witnessed in many industries is fuelling demand.
According to Alexandru Duca, CIO of Salalah Port Services, the streamlining of business practices leads to a burgeoning in storage capacity requirements. “As [increasing] number of companies decide to automate their systems, moving towards the paperless office ideal, the generation of data increases. The improved systems allow better reporting and greater productivity — all this translates into more information to store. What a business may save by implementing a new ERP (enterprise resource planning) solution, it can easily then spend on expanding its storage capacity,” he says.
It is only logical that as dependence on technology evolves, enterprise systems will become more memory-hungry, particularly as long as e-mail remains the preferred means of business communication. As Fred Moore, president of Horison Information Strategies, highlights in his paper on storage navigation, “E-mail is presently generating about 400 petabytes of digital data per year making it perhaps the most storage-intensive application.”
All this extra information translates into heavy investment on the part of enterprises. Global research firm IDC put spending on storage in the Gulf States at over US$112 million and predicts it will exceed US$400 million by 2008. “Organisations are realising the importance of their business-critical data and know they cannot afford to cut corners when it comes to securing it,” states Heini Booysen, program manager for IDC in the Middle East and Africa, continuing: “Data protection is the main driver in spending and end user demand for additional capacity will only continue to heighten.”
Booysen’s view is indicative of the dramatic change in the storage industry since the year 2000. The concept of storage itself has evolved from a non-centralised, tactical issue in most businesses to becoming the primary strategic concern at the heart of a company’s survival. The major economic correction that began in 2000 has significantly impacted the IT value chain.
Reshaped by the collapse of the dot-coms and the events of 9/11, the hardware-based storage industry of the past has given way to a new value system increasingly based on software, services and the true value of the data itself. “These things have opened the eyes of many business people. They realise now that information is their business. Without that, they have nothing and they want to do everything they can to protect it,” confirms Fawaz Bassim, IT infrastructure, operations and support manager for Kuwait-based Wataniya Telecom.
Corporate data plays a significant role in today’s competitive business environment. It allows enterprises develop business plans that can help them gain competitive edge. “For most enterprises, storage has gone from being an issue that only concerned the IT infrastructure group to an issue that concerns senior management throughout the organisation,” points out Murali Nambiar, sales manager for Veritas in the Gulf States.
A number of reasons have contributed to this escalation, including consolidation of technology resources, cost reduction and compliance with new regulatory legislation. The value of digital data varies considerably over its lifetime. While in the past businesses did not pay a great deal of attention to understanding or managing the aptly termed data lifecycle, companies are quickly realising the place where data is initially stored is unlikely to be the appropriate place for it further down the line.
“The three-fold challenge facing companies today — the sheer volume of data, the retention requirements of government regulations and the budget restrictions mean that CIOs are forced to do more for less — is leading enterprises to consider information lifecycle management (ILM) strategies,” says Jason Phippen, director of product marketing at Computer Associates (CA). ||**|||~|EMC--Mohammed-Amin-BODY.jpg|~|ILM is about the “I” in IT, the information and its management. On the other hand, virtualisation is about the “T”, the technology that enables this,” says Mohammed Amin, EMC Middle East. |~|ILM involves aligning information, data and storage management with business use, allowing data to be managed differently at different points in time based on its changing business value.
Such an approach is designed to streamline and optimise the management of information from creation to archiving and/or removal by employing intelligent use of storage management technologies such as data movers, archiving tools and hierarchical storage management (HSM). The general consensus is this type of strategy is critical for any organisation in order to avoid being overwhelmed by a growing amount of data.
Historically, data was often moved from one medium to another based primarily on its age or how frequently it was accessed. Older, less regularly used information was typically moved to low-cost, low-performance, less-accessible systems. While this idea may sound reasonable, in practice it ignores factors like duplication and does not account for key areas of growth such as e-mail or databases.
In addition, the approach may not align with compliance goals or security issues. “The traditional strategy posed problems for organisations that needed immediate access to information that is not frequently used. For example, medical records might be old or accessed infrequently, but in an emergency they must be instantly available,” Phippen points out.
The major challenge faced by enterprises, which decide to adopt an ILM approach is deciding what information to store where.
Abdulla Ali Al-Madani of Dubai Municipality believes data prioritisation has to be managed on a departmental basis. “Each of the different departments of the Municipality has different operational requirements and each needed a tailored storage solution. Categorising the different types of information is a business decision we felt was best handled by the individual department. We then did our best to align these needs with the storage strategy and balance the cost of storage with the information’s changing value over time,” he explains.
In Kuwait, the number of mobile phone call minutes per month is one of the highest in the region and, as a result, the capacity required by operator Wataniya to store call records for its customers is an ongoing challenge. With all call detail records (CDRs) having to be stored for up to five years for security reasons, and twice-yearly audits, data consolidation and unification is a must. However, telecommunications companies face a particular problem. The issues of knowing what data is critical, which records need to be retained for how long, and at what level of accessibility, is compounded by the unstable landscape of the industry.
“In our field the legislation [pertaining to] all the various documentation and data we collect could change at any time and then we might find we need immediate access to something we do not have or have to search for,” says Wataniya’s Bassim. “We have to store everything, just in case and as our customer base continues to grow this is proving to be problematic — we have to update our system on a daily basis just to keep up,” he adds.
Bassim describes the organisation’s current IT infrastructure as disparate, with services having been quickly launched without any long-term vision. He also says, however, like most businesses, the focus is now on data, storage and server consolidation. The Kuwaiti operator is currently in the process of drawing up an effective roadmap for ILM in order to gain better insight into the use of its resources. It is also hoped by taking a more centralised approach and shifting to a standardised, single capacity storage area network (SAN) the architecture will be easier to provision.
As an online company, however, Wataniya cannot afford to have any down time and has to make its migration from database to file system slowly and on a step-by-step basis. Ultimately though, this should prove to be a worthwhile move. “Overheads will be lowered by around 20% and we will be able to achieve consistency across our different services,” Bassim predicts.
Oil giant Saudi Aramco also recognises the advantages of storage networking solutions in effectively managing information flow and achieving business continuity. Scalable and robust, Salem Al Rowaihi, Saudi Aramco’s IT strategic planner, views SANs as a key factor in assuring timely and accurate information and providing an organisation with a competitive edge. “[Storage networking technologies] contribute to the organisation’s success by facilitating reliable and secure storage and retrieval of information, as well as establishing a flexible and scalable storage infrastructure. Thus maximising the value of available information and positively impacting the business,” he says.
Despite the benefits of storage networking solutions and ILM, there is no denying the complexity involved in managing such solutions. Al Rowaihi points to the difficulty of customising systems to fit the exact requirements of the business as the biggest challenge facing most organisations.
With the multitude of technologies, solutions and architectures available, and the complication brought by ongoing standards development in these areas, strategies must be carefully planned, he says. “Each approach has its advantages and disadvantages and an organisation’s choice depends on its requirements,” he emphasises. Al Rowaihi also highlights the complexity of SAN solutions, requiring specialist skills to manage effectively.
By pooling physical storage from multiple network storage devices onto a single storage device and allowing centralised management, storage virtualisation within a SAN can be a good solution for reducing complexity and speeding up administration tasks like archiving, back-up and recovery. Indeed, as well as the greater flexibility and utilisation virtualisation permits, EMC claims the most significant benefit of the technology is its ability to assist customers in building and implementing an ILM strategy.
“ILM is the foundation of a flexible infrastructure — one that can be tiered, share and provide intelligent movement of information — and that is the type of environment where virtualisation can be successfully leveraged. Another way of saying this is to say that ILM is about the “I” in IT, the information and its management. On the other hand, virtualisation is about the “T”, the technology that enables this,” says Mohammed Amin, regional manager for EMC Middle East.
||**|||~|Bassim,-Fawaz-BODY.jpg|~|The collapse of the dot-coms and the ramifications of 9/11 have opened the eyes of many business people making them realise the importance of information to their business, says Wataniya’s Bassim. |~|Veritas is also quick to promote the genuine, practical usefulness of virtualisation for providing an easy-to-use storage environment. Omar Dajani, the company’s technical manager, is adamant: “It is not just hype. I almost wish it were. All too often companies run out of space and then call us when it is too late. Despite growing understanding — organisations are certainly being made to realise they simply can’t afford to run out of space — effective data management is only now coming to be adopted. ILM and virtualisation effectively go hand in hand.”
Among the end users themselves, however, the general opinion is that virtualisation is an interesting prospect and could certainly prove useful further down the line. “We have a lot of legacy applications, so virtualisation would be good. It’s definitely a big trend at the moment, but I think more needs to be done so the technology is able to support a more heterogeneous, multi-vendor configuration,” Wataniya’s Bassim says. For Dubai Municipality, too, the prospect of virtualisation has its appeal.
“The hardware layer would be completely hidden from the application layer. This would mean we would not have to worry about what was stored where and management would be so much simpler,” Al Madani muses. “Once we have been able to categorise and classify all our data, virtualising the infrastructure appears to be the next logical step of simplification.”
The uptake of elements such as SAN, network attached storage (NAS) direct attached storage (DAS), IP storage and virtualisation schemes generally remains slower than expected. Integrating these components into a workable solution is the challenge facing enterprises that wish to move away from their aging legacy interfaces.
“With so many solutions on the market knowing what to deploy and what will be an intelligent long-term investment is a challenge. Ensuring the technology we choose will be scalable and meet the relevant best practice requirements is a priority but we also have to ensure we have the necessary skill sets to get the most out of the infrastructure,” says Salalah Port Services’ Duca.
The storage industry is assessing new storage and network interfaces, which are currently being developed and anticipated to have major impact in the future. “New technologies rarely change the world overnight,” says Horison Information Strategies’ Moore. “The biggest architectural shift ahead is likely to come in the form of the movement of most storage intelligence away from the server and onto the storage networks.”
Transferring storage applications onto the network offers vastly improved and long-overdue management relief when compared to host-based techniques, which normally require multiple management tools from a variety of vendors. This server offload enables a more centralised approach to storage management, minimising the number of administration touch points.
This type of development makes other new technologies a possibility. A new concept, that of the virtual storage area network (VSAN), is just beginning to emerge, making it possible for users to tie their existing SAN fabrics onto a central core switch and protecting the integrity of each fabric. These fabrics can then exist on the same switch and remain unaware of one another. While this kind of intelligent switch has been discussed since the early 1990s, no SAN-based solutions have been widely implemented as yet.
“Several storage management software vendors are aligning their products with major switch companies in order to make the intelligent network a reality — this would bring major improvements in storage management,” according to Veritas’ Dajani.
There are several other new and potentially beneficial offerings entering the storage industry. The new disk-to-disk-to-tape (D2D2T) appliances, which consist of disk tape, a back-up server and software on a single physical module have generated initial interest from the small and medium-sized business marketplace as a plug-and-play backup solution requiring minimal management effort. “All businesses, large or small, need to find a solution that will allow them to cope with the amount of data they produce. It is not an issue that even the smallest business can afford to ignore,” says Bassim. ||**|||~||~||~|Many vendors anticipate the path to higher-capacity tape cartridges and disk drives will quickly exceed the one terabyte capacity level and massive arrays of idle disks, known as MAID libraries and consisting of economy disks, are presenting the lowest price-point for any disk subsystem.
At the same time, such arrangements can potentially shrink the time it takes to perform data recovery. “When there is such a massive amount of data stored there has to be an efficient means of recovery. Searching blind for a file simply takes too much time away from critical business activities, and time means money. Just like searching for a physical object, if you have not paid attention to where and how you’ve stored it, finding it could be like looking for a needle in a haystack, and who knows what condition it might be in,” asserts CA’s Phippen.
Data preservation and integrity are the focus of a new era of storage lifetime management applications, with archiving becoming a science involving media life, devices, security and data migration capability to enable infinite archive data conservation for tens if not hundreds of years.
Security has become the latest storage management discipline and is quickly driving development of security appliances, surveillance technologies and encryption capabilities. “How does an organisation decide what data to delete? It can’t. At any point in time any of that information could be needed, so it is possible we might be required to store all data somewhere, securely, forever,” says Al Madani.
Selecting the right framework to manage this challenge comes down to numerous choices. Always a criterion, product costing needs to include not only software licensing and maintenance fees, but also the careful consideration of auxiliary costs such as training, deployment consulting services, product set-up, and policy creation time. “If something is worth doing, it’s worth doing well. Ensuring not only that you have the right equipment, but also people with the right skills to maximise that equipment is vital. This is why finding the right vendor to partner with is essential,” notes Duca.
With more than 17,000 storage array products on the market, plus a smaller number of host bus adapters, switches and tape libraries, organisations must carefully choose a management framework that delivers support to the entire infrastructure out of the box.
Products should adapt to a business’ manual approaches and workflows around storage administration rather than requiring the adaptation of existing best practices to the product’s approach. As Phippen explains, “Interfaces should be flexible and intuitive, designed to satisfy the individual information needs of different users.”
The design of management products and their functional integration should also be as seamless as possible, integrating via a common code and shared database. While the ideal solution may be one that can be deployed and begins to deliver business value immediately, many enterprises are starting to realise that when it comes to the lifeblood of their enterprise, a bigger outlay upfront may be worth the greater peace of mind in the future.
As Al Madani says, “Our reliance on computers will not diminish, and therefore the volume of data we produce will continue to grow exponentially. Companies have to realise that they either take steps to store their data securely and efficiently now or risk losing not just bits and bytes but their entire business.”||**||