"What is Big Data?"
My guess is that you have heard the term big data. Data that is so big it is supposed to change everything. The big data that predicted a teenager’s pregnancy before her father found out. But, as a business person, how is this going to affect you? Do you even need to care?
With all the information and tools on the market, deciding where to start is a daunting task. Let’s keep it simple and start with the realization that you, yes you, are not a data scientist and have no idea what a ‘Yottabyte is, nor how to administer let alone run a 2000-node Hadoop cluster.
You are a business manager, an IT manager, looking to find solutions to help you solve your day-to-day business challenges. At the end of the day, you simply want to improve you and your team’s productivity.
For you, big data is not about yadi-yada-yotta bytes. (For your reference, a Yottabyte is a million times a million times of a Terabyte.) For you, big data is a way of thinking. A way of thinking that will lead to better decision making and productivity gains.
The Big Data way of thinking can be broken down into a 4-level maturity model:
Level 1: Collect
Your journey into big data thinking starts with the demand for good data. Perhaps your data is lying around the office on pieces of paper, or in emails. Worse, you are failing to collect vital data in a meaningful way. If you do not yet have a quick and systematic way to collect the data you need, simply start with what you are already doing on paper and expand from there.
Level 2: Integrate
Once you are able to collect good data quickly and easily, the next step is to get the most out of that data. Create dashboards and reports, integrated with your backend systems. Use the data to manage your business; gauge customer satisfaction, manage inventory. Good data allows you to react to the changing business landscape in real-time.
Level 3: Centralize
You will start thinking about things like data standards, data validation, and data retention, and to accomplish those things, the centralized data repository is a must. Eventually you will create a data policy across your organization that addresses such things as data collection, data curation, data validation, data security, and finally data retention. Now, you are thinking big.
Level 4: Strategize
With your data neatly consolidated, you can now take your imagination beyond the single project | data source ‘point-to-point’ reporting mentality. As you probably have noticed by now, reporting and or dashboard tools are much more effective if you feed in ‘refined’ data streams. Indicators and trends will become evident across business departments, projects, and the organization as a whole, allowing you to make more informed decisions; helping you strategize for the future based on multi-dimensional data. Congratulations, you are now running a data-driven organization.
Is that it?
Hardly. As you find yourself sitting in Level 4, you will want to go back to Level 3 and see if you can consolidate even more of your data streams. “Wonder if I can incorporate weather information? Demographic information?” Big data thinking continues...