Big data or genuine insight?
So "Big Data" is obviously the phrase du jour (or du week, month, year, decade?). I'm all for having access to lots of data. Properly managed and interpreted, Big Data can lead to better decisions, more satisfied customers, more successful businesses, maybe even a "better" society. Show me the benefits, and let's roll.
But ... don't confuse Big Data with genuine insight. What am I talking about? Well, I'm talking about stuff like understanding customer needs and preferences, assessing the worth of a new technology or new-product idea, optimizing "customer experience" (sorry about that), deciding whether a contemplated M&A deal or partnering arrangement makes any sense, and the like.
Despite what some may demand, the preferred output of such activities is insight, not some database. I acknowledge that the insight often must be supported by data in some form, but never forget that the truly valuable output is genuine insight.
So, what does it take to stimulate genuine insight? Well, here are some general observations based upon my own experience. (The tactical details will depend upon the context of your inquiry, but you get the idea.)
- Understand the field of inquiry. Ideally, be - or become - an expert.
- Identify and communicate with the right people. And, reach out to them using the right communication vehicle(s).
- Don't obsess about large numbers of information sources and statistical significance. More than 20-30 samples are almost never required, and statistical significance is a pipe dream.
- Exploit the Web and social media with care. Be sure you understand who crafted the message and for what purpose. Exercise due diligence.
When that a-hah moment arrives, you'll know you are (almost) done.