One of the huge values of the big data era is the ability to mine and analyze massive amounts of information to draw very sophisticated conclusions. That’s a wonderful advance of science that continues to prove its value and I am among the many who are thrilled to see how far we can push this technology to better our lives. However, there’s a key missing element to today’s big data analytic conversation – trust.
I was at a breakfast this morning of industry technology executives, many of whom are leaders in their field. It’s always inspiring when this group convenes to hear titans of industry share their insight and experiences. Inevitably, once the off-stage conversations begin, the environment is electric as ideas and revelations are exchanged.
However, what struck me most about today was a question from the audience that, unintentionally, revealed a problem that transcended the core topic of the day. The audience member was curious as to why certain behavioral changes among a particular group of practitioners proved challenging in the face of overwhelming evidence that became available as a result of big data analytics. The speaker receiving the question, while brilliant in many regards, shook his head in wonder not able to provide a definitive response. Although, he provided a nod in the direction of trust when he referenced nagging conflicts of interest among publishers of professional journals.
I think that’s it. We have a trust problem. People are skeptical; hypersensitive to the discovery of a personal agenda or selfish moti
vation. Everybody is working an angle, so to speak. Well, aren’t they? Therein lies the hidden challenge of big data. You can come up with the most exquisite analysis on any given topic, and if you, the producer (writer, author, publisher, manager, executive, etc.) are not viewed as a trusted source, then what have we achieved? Despite the perfect application of technology, its outcome falls on deaf ears. As a result, despite all evidence to the contrary, we don’t change. We may find the information interesting, but it fails to move us.
Therefore, if you plan to consume big data technology to perform analytics and deliver outcomes, it is well worth the investment to develop your reputation as an unbiased, trusted source by those you intend to influence or all could be for naught.
This article was originally published on my LinkedIn profile under the heading "The Hidden Challenge of Big Data" on April 9th, 2015.