* The chaos theory of the CMDB and analytics
I got a fair amount of correspondence to the first newsletter in which I wrote about analytics and the configuration management database. All of the feedback was positive and largely echoed my belief that without some form of analytics, an investment in the IT Infrastructure Library’s CMDB is not likely to bring as many dividends as most might hope. In a worst-case scenario, an underutilized CMDB might become an inert mass of data with huge upkeep and little real value.
Just to ground all of you who didn’t read the first column – the ITIL’s CMDB is really a process enabler with huge architectural dimensions and issues. Ideally it is a trusted, multi-dimensional and current view of inventory, configuration, topological, service, organizational, business and policy-related information to support a whole host of management disciplines, from change and configuration, to service assurance, to asset management, etc.
Today, I’m going to address the question of “intelligence” or “analytics” built into the CMDB vs. the analytics that harvest the CMDB. For starters – it’s clear that auto-discovery analytical engines that capture physical network topologies, configuration details, and application service-to-component mapping are often complex intelligent systems in themselves. There are huge research and development investments in this arena, which spans everything from SNMP polling, to agent-based inventory – typical in the systems environment, and flow-based awareness through packet and protocol analysis that capture application flows over the network. These “analytics” would come at the front end of the system – as a form of highly intelligent data gathering and relationship building.
Then the CMDB itself, with its data schema is another foundation for capturing intelligence. While probably no one would equate a data schema with an analytic engine on a one-to-one basis, there are many architects who seem to believe that once the models are built accurately, most of the work is done. Intelligent actions simply flow from the models and the policies around them with a logic and precision that even Plato might envy in his world of ideas. And its appeal to IT buyers is also enormous – bringing with it a comforting sense of order.
So for those of you who are of this determinist mindset – who might ask me that if the CMDB is to become such a well modeled view of an IT service ecosystem as it changes, do all the other hard-line analytic investments we talked about the other week remain necessary? (By the way, these ranged from pattern matching, to correlative algorithms, to neural networking, to OLAP, to chaos theory, as just a few examples.) Can’t most of the diagnostic power be captured within the modeling of interdependencies within the CMDB itself?
Well, in my opinion – no, or at least not for the foreseeable future. One analogy for the CMDB vis-a-vis the analytics layer that harvests it is to contrast an accurate model of the earth and all its existing topologies as they change (rivers overflow, etc.) with the capabilities to do weather prediction and climactic modeling (the role of analytics, often done by supercomputers for meteorology). The analogy underscores the growing complexity of distributed networked environments.
One might push back and say that the variables are hugely smaller in a totally manufactured “high tech” ecosystem vs. the still unfathomable ecosystems of nature. While this is fair enough, in my view the analogy still holds as vast uncertainties persist. Even within operations, for instance, the many “hands” implementing actions and changes across interdependent organizations and teams are never likely to be totally controlled. (Or in other words – human error will always continue at some level.) Perhaps even more significantly, the interactions of human consumers of services with those services and with each other is beyond any hardwired predictability – and this trend will be mirrored in highly distributed Web services. Finally, we simply don’t know enough about all the interdependencies within the infrastructure itself to understand what a change to a server in Detroit might somehow do to the performance of a router in Berlin.
This level of transcendent sensibility to what might seem even imperceptible changes is most dramatically captured in chaos theory’s “Butterfly Effect” – in which a butterfly flapping its wings in Tokyo may, through a series of minimal chain reactions across the atmosphere – actually impact the weather in Texas. As far-fetched as it may seem, it is beginning to open many doors in understanding ecology, genetics and even something as basic as the speed of water crossing that imperceptible threshold when it begins to move a mill wheel. Within IT, chaos theory has helped to discover server latencies when the focus had been WAN performance, or unearthed less-than-conspicuous security issues, and captured performance degradation directly caused by enterprise management tool sets.
Of course chaos theory is just one of many examples to show that capturing and diagnosing the unknown is at least as important as modeling the known. The role of analytics as suggested here – as a way of building dynamic insights and control over the enabling foundation of the CMDB – is one of growth and complexity on a scale equivalent to the CMDB, itself. If this is true, then in many respects the CMDB is only the beginning. So welcome to the dawn of next-generation management. (Or maybe it’s still dusk.)
I do welcome more of your thoughts, insights, and perspectives on this vast and puzzling but critical topic.
