- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
Network World - If there's gold in log files, Splunk, Inc's Splunk Enterprise will help you to find it. Splunk bridges the gap between simple log management and security information and event management (SIEM) products from vendors such as ArcSight, RSA, Q1 Labs, and Symantec.
Splunk lets you gather log data from systems and devices, and run queries on that data to find issues and debug problems. Splunk's capabilities also include reporting and alerting, pushing it every-so-slightly into the world of SIEM.
What separates out Splunk from the world of Syslog servers and SIEM tools is Splunk Apps, a library of nearly 200 add-ons that make Splunk smarter about particular types of log information, change its look-and-feel, or add new types of analysis.
We found Splunk to be a powerful, if complicated, tool. Moving from a simple syslog server or event viewer with a "search" command to Splunk is not a one-day operation. But Splunk pays off in the power that it brings, and the information it can pull out of your logs. Network managers who look at their logs every day, and those who wish they could get useful performance, capacity, and security information out of the gigabytes of log data stacking up should take a close look at Splunk.
There's a free version of Splunk for small and midsized deployments, so if your log files don't add up to 500MB each day, Splunk can be yours for the cost of the server you run it on. Some features, such as alerting, role-based access control, and distributed searching are not available in the free version; you also can't run premium applications on top of the free version.
But Splunk is designed to scale up, way, way, up. With distributed search databases, role-based access control, and the ability to eat terabytes of log data each day, Splunk is aimed at the large enterprise.
Splunk wants to be fed everything, including system, web, security, and every other type of log or performance data you can find. We didn't want to go quite that large, so we tested using Splunk on our own small data center, using live data.
Getting data into Splunk follows the same paths as any log management solution. We set up Splunk on a Linux system (Windows and other Unix flavors are also supported), a simple matter of an RPM installation, and had it listen for data sent to it with Syslog, probably the most common way to get your log data off systems and into an analysis tool.
For Windows systems, Splunk provides their "universal forwarder," an application that will pull Windows WMI data and forward it off to a Splunk server. The Universal Forwarder can also monitor file systems for changes and forward data from remote systems back to a central Splunk installation. We only used it to pull Windows event log information.
Splunk isn't too particular about where and how it gets data, with options for scripting and other network input sources.
Our initial contact with Splunk's input system, however, gave us a pretty good feel for Splunk's operational style. Splunk is not a do-it-yourself piece of open source software, but it also doesn't have the smooth polish we have seen from other commercial products. Splunk has an internal complexity that the Splunk team is happy to share with everyone through an extensive on-line documentation system.