Big Data visualization: Finding the needle in a haystack

How Big Data was able to help stop an E. coli outbreak, and what we can learn from that example.

big data numbers
Credit: Thinkstock

It seems you can't go anywhere these days without hearing about Big Data. It's a broad term used to describe the challenge of working with massive complicated data sets. But the problem with looking at data from the point of view of volume or size is that it misses some of the particular benefits of finding those little bits of valuable information.

Some might describe finding bite-sized pieces of information similar to that of finding a needle in a haystack, but recent advancements in data visualization design and predictive analytics have made this much easier to do. Today, various data visualization tools can be used to not only assemble all of the available information and data inputs, but also output a complete illustrated vision. The use of these sorts of visuals can help key stakeholders see and act upon the information presented in a much more timely fashion.

042115 big data graph infographic reuven cohen Reuven Cohen

A great example of this is a 2012 multistate outbreak of E. coli. The use of various analytics led public health and agriculture officials to identify the source of the outburst and a means of responding to it in record time.

After gathering medical records – and the ages, locations, and recent dietary patterns of patients who have been treated – visualization helped bring to light that the majority of people suffering from E. coli illnesses had eaten romaine lettuce that was sold primarily through the same grocery chain across different states.

042115 reuven cohen image 2 Reuven Cohen

Content source: Centers for Disease Control and Prevention

From there, public health and agriculture officials were able to identify the farm, or farms, where the lettuce sold at those grocery stores was produced, and ensure that farm workers, distributors, and grocery staff are educated on the proper produce handling and hygiene recommendations.

042115 reuven image 3 Reuven Cohen

Content source: Centers for Disease Control and Prevention

Now, don't misunderstand me, there is certainly value to a holistic view of information as well. This is particularly true when it comes to the visualization and understanding of data you're working with. Regardless of the scope of your viewpoint, we should always attempt to comprehend the sum of the parts as intimately interconnected to the whole of the entire data set. Yet the challenges still remain for many corporate users of data and analytics. Getting the data is easy, using it and understanding it hard.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: 10 new UI features coming to Windows 10