Log and event management appliances improve compliance, security, operations

Log and event management is now a requirement for organizations that need to monitor security and IT policy enforcement, document compliance, and achieve IT operations excellence without increasing head count. However, current approaches to log and security event management force customers to purchase and integrate two or more products for each discipline. This approach is complex, costly, and difficult to deploy and manage for enterprises with large data centers, distributed operations and/or branch offices.

In a typical organization, millions of logs are generated by every system, application and device on the network every day. According to the SANS Institute, logs represent up to 25% of the total data created in a typical enterprise.

While most logs are not important or meaningful, a small percentage are extremely valuable. They contain insights and warnings about the health of the network, security issues, compliance violations and operational problems.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

Log and event management is now a requirement for organizations that need to monitor security and IT policy enforcement, document compliance, and achieve IT operations excellence without increasing head count. However, current approaches to log and security event management force customers to purchase and integrate two or more products for each discipline. This approach is complex, costly, and difficult to deploy and manage for enterprises with large data centers, distributed operations and/or branch offices.

In a typical organization, millions of logs are generated by every system, application and device on the network every day. According to the SANS Institute, logs represent up to 25% of the total data created in a typical enterprise.

While most logs are not important or meaningful, a small percentage are extremely valuable. They contain insights and warnings about the health of the network, security issues, compliance violations and operational problems.

To unlock the value of logs, a new class of appliance has emerged that combines universal log-data collection, analysis, event management, automated report distribution and incident response. They employ a building-block approach that allows organizations to start with a single appliance then add more devices as the number of log sources and volumes grow. A single management console makes expansion seamless.

Click to see: Diagram of how log and event management works

These new log- and event-management appliances perform the following continuous cycle of functions:

* Log collection: Log sources can include servers, applications, databases, firewalls, switches, routers, point of sale (POS) systems and more. Anything connected to the network is likely generating logs. Logs can be delivered to the appliance via standard network-logging protocols such as Syslog and Netflow. They can be pulled from Windows hosts (event logs) and any database compliant with Open Database Connectivity. Logs also can be collected by agents from remote sites and flat-file sources (that is, Web server logs) and forwarded to the appliance.

* Log management: Since log formats are as varied as the log sources themselves, once logs are collected they must be normalized. Log normalization includes classifying logs so they can be correlated, stored, reported on and managed. Normalization is a key step in transforming logs from raw data to valuable information. During the normalization process, the appliance also automatically synchronizes the time stamps of all log entries to single ‘normal time’ for reporting and analysis purposes.

* Archival and restoration: Many organizations must retain log data for specific periods to meet regulatory requirements. Integrated log- and event-management appliances completely automate the process of archiving and restoring log data. Based on policy settings, the appliance automatically archives log data and generates bookkeeping information such as where and when the log data originated. Archive files are cryptography signed and compressed, providing tamper-proof, cost-effective long-term storage. They can be easily restored via intuitive wizard-based tools that verify the archive files have not been modified since originally created.