By Eliot Beer
Volumes of digital data increasing through growth of content about individuals created by third parties
The total amount of digital data in the world exceeded 281 billion gigabytes (281 exabytes) in 2007, and will rise to 1,800 exabytes (1.8 zettabytes) by 2011, according to a new study from IDC.
IDC's findings, which were sponsored by storage vendor EMC, reveal the 2007 ‘digital universe' was 10% larger than estimated in last year's research, primarily due to the growth of digital photography and video applications.
Digital data is set to grow 60% year on year up to 2011, and this data will increasingly come from what IDC describes as individuals' ‘digital shadows': information about people generated by their activities, such as phone or bank records, web histories or surveillance footage from security cameras.
"In the updated study, we discovered that only about half of your digital footprint is related to your individual actions - taking pictures, sending emails, or making digital voice calls," said John Gantz, chief research officer and senior vice president at IDC.
"The other half is what we call the ‘digital shadow' - information about you. For the first time your digital shadow is larger than the digital information you actively create about yourself," he added.
Information on the governance of data also supports this conclusion: while 70% of data is - directly or indirectly - created by individuals, 85% of data is the responsibility of enterprises and other organisations, rather than individuals themselves.
"Business will have much more information on each of us, they'll have our personal financial details, our health information - you don't want to have that available to the masses. It's going to rest on the businesses to provide that level of security, so that people aren't lying awake at night worrying about who has access to their most personal information," said Frank Hauck, executive vice president for global marketing and customer quality at EMC.
While no major data breaches have been made public in the Middle East in recent years, data protection has been brought into sharp focus thanks to scandals such as the theft of 90 million credit card details from TJX in the US, and the British government's loss of millions of bank details after mislaying an unencrypted CD.
"I think we will see these types of incidents in the Middle East, as many enterprises do not have mature infrastructures, do not invest in disaster recovery in a big way, and some organisations are not looking at the overall design of their systems. I would be very surprised not to see such data breaches here," said Mohammed Amin, regional manager for EMC Middle East and Africa.
The report also highlighted how one action - such as sending an e-mail - can generate much larger amounts of data in the form of cached items, backed-up data and other mirrors. IDC gave the example of a single 1.1Mbyte e-mail sent to four people, which could potentially generate 51.5Mbytes of data in total.
IDC suggests 2007 will mark the last point when available storage is roughly equal to the volume of data generated - the future will see more data created than can be retained. While much of this will be transitory information such as phone conversations or broadcast TV, enterprises will still need to make decisions around what data to retain, and what to discard.
As well as the volume of data, the number of data "containers" is also set to explode, due to the growing use of RFID tags, sensors and VOIP technology. IDC predicts the number of containers will grow 50% faster than the volume of data, and will number 20 quadrillion (20 million billion) by 2011.