This column is available in a weekly newsletter called IT Best Practices. Click here to subscribe.
The old saying goes, "Necessity is the mother of invention." The tech veterans behind the startup Kentik (formerly CloudHelix) saw a need for a simple, large scale network visibility and analytics solution, so they invented one. Their SaaS solution, Kentik Detect, is already gaining traction with network operators from ISPs, Web companies like Box and Yelp, cloud-delivered security companies like OpenDNS, and enterprises with extensive customer-facing applications.
Kentik just emerged from stealth mode at the end of June. Over the past few years, Kentik technologists talked to dozens of network operators who represent companies ranging from small ISPs to the largest carriers and enterprise organizations. The conversations had a common theme: there is a dearth of ready-to-use, affordable tools that can tell them about the efficiency, availability and security of their infrastructure.
Kentik found the operators were having to build their own monitoring tools using a patchwork of open source products. Or, they had to buy appliance-based solutions that cost almost as much as the infrastructure they were meant to monitor. Many of these folks lamented that they had the bare minimum in the way of tools to do their jobs. Kentik saw an opportunity to fill this void.
Kentik Detect is a cloud-based service that allows customers to upload their flow, SNMP and BGP data into a single system at multi-terabit scale. Kentik built its own column-stored database that can operate at Web scale to hold all that information. The company uses Big Data applications to perform analytics on that massive amount of metadata.
Customers log in to a portal to find out about the efficiency of the network, the availability of resources, and whether there are security issues to be concerned about. They can also use SQL to query the database outside of the portal, and the solution can send alerts to notify a customer of active attacks or other conditions based on user-specified criteria.
The vendor cites several popular use cases for its solution. One of the primary use cases is to detect active DDoS attacks within minutes—long before the typical DDoS remediation tools kick in. The vendor paints this scenario: You have an application performance problem. You look at the network and your SNMP tools and see things are full. But why? What's going on? Is it an attack? Or a misconfiguration? Is someone running a backup? Kentik Detect is said to let you see known traffic patterns of people flooding you with packets or attacking your application. This is all visible within a minute or so of when it begins, and you can take immediate steps to begin remediation.
Another big use case is for traffic efficiency planning and connectivity. Network operators can use Kentik's insight to determine where to add capacity. They can see where traffic is going and where it is going at the next hop. They can determine who to connect to in order to reduce costs, or to make sure the APIs that render a website are not held up. A simple SNMP view doesn't provide enough detail to see what the Kentik tool allows customer to see, the company says.
Performance monitoring is a third common use case, with a focus on what's happening externally. Kentik takes the routing and the BGP data and unifies it with the flow metadata. Kentik can then show the network layer performance problems causing application problems and group them by the path they are taking. Similar to what SD WAN vendors do for inside a WAN, Kentik does externally to the WAN to determine if there are better paths to take. Today the vendor simply provides visibility, and the control side of this use case will come later.
And finally, security is a fourth common use case. Kentik Detect provides a single place to converge and store the raw data for at least 90 days, and longer at a customer's request. This allows a customer to go back in time to do forensic analysis. For example, a company might want to look at historical data to see what was going on before a compromise took place. Kentik says it has SOC customers that are using historical data for this purpose.
Another security example is to take the direct feed data and identify from the netflow what traffic on a network is going to suspicious places. The customer can learn if a device is connected to a botnet and then go back through historical data to look for how the device became compromised.
Kentik says it has quite a few customers that are building their own applications on top of this data platform. The vendor is working with customers to take some of those custom applications and build them into the commercial solution.
The Kentik Detect solution is all automated. Kentik takes in a customer's data, performs the analysis, and makes the intelligence available through a portal within minutes. There are built-in alerts, such as the warnings for a DDoS attack, and Kentik works with each customer to tune the parameters to ensure they aren't getting many false positives. Kentik has a professional services team to work with individual customers on their specific infrastructure issues.
Real-time network visibility is essential to ensure resilience of any large scale network. Now it's available as a service via public SaaS or on-premise private SaaS.