Beware of Iterative Fatigue, a Hazard of Agile Analytics
A fair amount has been said and written of late on “agile BI,” or the use of agile—evolutionary, with frequent, small, and iterative outputs—methods to produce Business Intelligence outputs, specifically reports. It is difficult to disagree with this concept, since BI was originally supposed to be largely about businesspeople producing their own queries and reports—presumably using agile methods. Certainly there is little doubt that most people don’t know exactly what information they want in what format until they actually see it. The fact that there are few objections to agile BI has meant that a number of vendors now claim they produce “agile BI solutions,” though it seems to me that you could employ almost any BI tool in an agile fashion.
More importantly, I’d like to propose extending the agile BI concept to incorporate “agile analytics.” Again, most executives and organizations don’t know what analyses they want until they see them. Therefore, analysts should strive to produce a series of quick-and-dirty analyses until it’s clear that the analysis employs the right methods on the right data using the right display formats.
Of course, agile analytics may take a bit longer than agile reporting. Getting data in shape to be analyzed is still going to take some time. In many cases it may make sense to do the analysis on a subset of data if doing so would accelerate the process. There may also be more iterations with agile analytics, because the number of ways one can analyze data is limitless, and after that you still have to figure out how you want to report the data.
Agile analytics can also play a valuable teaching function for the business partners of analysts. Quantitative analysis is an inherently iterative activity anyway, and formalizing the notion of agile analytics makes that iteration more apparent to the customer. What happens when we exclude these outliers? What if we lag this variable by a year? Is there an interaction effect going on between these two variables? A good analyst is always asking him or herself these types of questions, and (within reason) the partner might as well realize and learn from it.
Taken too far, agile analytics might make the business customers of analytics feel that it’s a never-ending exercise, and that you can just keep manipulating the data until you get what you want. So you have to be a little careful in deciding just how much analytical ambiguity to which you will expose your internal customers.
Next time I’ll give some examples of this emerging phenomenon, and I’ll explain that it’s really about “agile decision-making,” not just agile analytics.
