April 5, 2012 - Matt Jacobson
“Big Data” is more than just another buzzword floating around the technology and business press. Even though your company may not be Google, Amazon or Facebook, with our current information economy Big Data impacts your business. The three key questions for your business are:
What is Big Data?
Is Big Data just marketing hype? We now have more than 110 million tweets per day and over 30 billion pieces of content shared on Facebook per month. Add up just a day’s Twitter stream, a day of Facebook data along with the compounding impact of Moore’s law and Big Data sure feels real. Twitter started in 2006 when MySpace was the leader in social networking so the change in our information economy dynamics over the last several years is dramatic. Consider how your own company’s transaction data, website metrics, social media accounts, online advertising, vmail, email and inventory tracking data have grown in the last five years. According to IDC, 1.8 trillion gigabytes were created and replicated in 2011 – a figure that has grew by a factor of 9 in comparison to the previous five years. With these figures in mind that I can categorically state that Big Data is here and its real.
So what is Big Data? Mike Loukides says in his O’Reilly Radar post that data is Big Data is when the size of the data itself becomes part of the problem. I prefer Gartner’s definition of Big Data which highlights three key characteristics: volume, variety and velocity. These terms form a common theme in the Big Data approaches of many leading vendors.
Clearly, only a relatively small portion of all of the Internet’s data is relevant to your business, but the sheer scale of the Web makes even a small percentage translate to massive amounts of data. The volume threshold for Big Data is dependent on your organization. Fortune 100 companies have the resources to deal with a much greater volume, velocity and variety of data than a company with only $500 million dollars in revenue. Big Data best practices are currently being established for people, process and technology, but companies need to first understand their goals and objectives before any implementation. This will be covered in my next blog post, “Why Do You Care?”