Log and event management appliances improve compliance, security, operations

Log and event management is now a requirement for organizations that need to monitor security and IT policy enforcement, document compliance, and achieve IT operations excellence without increasing head count. However, current approaches to log and security event management force customers to purchase and integrate two or more products for each discipline. This approach is complex, costly, and difficult to deploy and manage for enterprises with large data centers, distributed operations and/or branch offices.

In a typical organization, millions of logs are generated by every system, application and device on the network every day. According to the SANS Institute, logs represent up to 25% of the total data created in a typical enterprise.

While most logs are not important or meaningful, a small percentage are extremely valuable. They contain insights and warnings about the health of the network, security issues, compliance violations and operational problems.

To unlock the value of logs, a new class of appliance has emerged that combines universal log-data collection, analysis, event management, automated report distribution and incident response. They employ a building-block approach that allows organizations to start with a single appliance then add more devices as the number of log sources and volumes grow. A single management console makes expansion seamless.

Diagram of how log and event management works

These new log- and event-management appliances perform the following continuous cycle of functions:

* Log collection: Log sources can include servers, applications, databases, firewalls, switches, routers, point of sale (POS) systems and more. Anything connected to the network is likely generating logs. Logs can be delivered to the appliance via standard network-logging protocols such as Syslog and Netflow. They can be pulled from Windows hosts (event logs) and any database compliant with Open Database Connectivity. Logs also can be collected by agents from remote sites and flat-file sources (that is, Web server logs) and forwarded to the appliance.

* Log management: Since log formats are as varied as the log sources themselves, once logs are collected they must be normalized. Log normalization includes classifying logs so they can be correlated, stored, reported on and managed. Normalization is a key step in transforming logs from raw data to valuable information. During the normalization process, the appliance also automatically synchronizes the time stamps of all log entries to single ‘normal time’ for reporting and analysis purposes.

* Archival and restoration: Many organizations must retain log data for specific periods to meet regulatory requirements. Integrated log- and event-management appliances completely automate the process of archiving and restoring log data. Based on policy settings, the appliance automatically archives log data and generates bookkeeping information such as where and when the log data originated. Archive files are cryptography signed and compressed, providing tamper-proof, cost-effective long-term storage. They can be easily restored via intuitive wizard-based tools that verify the archive files have not been modified since originally created.

* Log analysis: Once collected and normalized, logs should be assigned a common name and classified under the appropriate category, such as security, operations or audit/compliance. Logs having the most immediate operational relevance should be appropriately designated. The latter are typically critical security events, audit failures, warnings and errors. Most systems include predefined events that can be customized, or allow new events to be created to meet the unique requirements of an organization.

* Event management: The importance of an event varies by organization and by log source or the system in question (that is, the value of the asset). For instance, a system reboot is unimportant on a user workstation, but when it occurs on an ERP server with a 99.999% uptime requirement, it’s critical. The appliance should support risk-based prioritization. One way to do that is to assign a priority from 1 to 100 based on the type of event; likelihood the event is a false alarm; threat rating of the host causing the event (for example, remote attacker); and risk rating of the application, system or device on which the event occurred.Risk-based prioritization helps ensure the most important events are identified and forwarded to the appropriate individual(s) for rapid response. It should be possible to send alerts via e-mail, SMS, page, SNMP, etc. A customizable “Personal Dashboard” interface is usually available to allow users to quickly assess problems and drill down to individual log and/or event data for root cause analysis.

* Flexible reporting: Log- and event-management appliances should also offer robust reporting capabilities, including prebuilt reports for specific regulatory requirements (such as the Sarbenes-Oxley Act and the Payment Card Industry Data Security Standard) as well as customizable reports that can be tailored to meet specific analysis and reporting needs.

For organizations looking to automate security information management, cut regulatory compliance audit and reporting costs, and proactively control operations for better service levels, the new breed of all-in-one log- and event-management appliances are a compelling choice.

Petersen is CTO of LogRhythm. He can be reached at chris.petersen@logrhythm.com.

Learn more about this topic

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2008 IDG Communications, Inc.

SD-WAN buyers guide: Key questions to ask vendors (and yourself)