I think we’re on the verge of a dramatic change in the technology architecture for analytics, and I plan to write several posts about it. If you think about it, we’ve had virtually the same technology environment for 20 years or so. It involves such features as:
• The separation of analytics software from transaction processing software;
• The creation and use of data warehouses to feed reporting and analysis processes;
• The use of multi-purpose analytical toolkits, a.k.a. “statistical packages;”
• Premise-based (as opposed to online service-based) analytical applications;
• Applications and tools that are generic with regard to the industry
These attributes have led to a technology environment I will call “the analytical sandbox”—a term that some organizations actually use. It signifies a separate world in which an analyst can play to his heart’s content; all the data and analytical methods that he or she could ever need are made available, and any question can be answered, any decision supported.
The problem, however, is that unless you are a very skilled analyst, there is too much sand (the data) and too many toys (the analytical tools) in the sandbox. Even the most capable analysts are probably familiar with only a few tools. And large volumes of data in enterprise-level warehouses make it difficult to find what you need for any particular analysis.
The problem for non-specialist businesspeople is even worse. We talk about “drill-down” and query processing and do-it-yourself analytics, but let’s face facts. Most people don’t have the skill to use sophisticated analytical tools, and they are even less able than analysts to identify and prepare the data they need to assist them in making a decision.
It’s as if we pointed to a big sandbox and said to a group of very young children, “Go play!” But the sand is too deep, the toys too unfamiliar. As a result, we seriously undermine the ability of businesspeople to make better decisions with analysis and data.
In the next few posts, I’ll write about a different analytical technology environment that is now emerging. It may take a while for it to vanquish the current paradigm, but I’m confident that it’s coming.