Log management has become an essential enterprise service yet to paraphrase Rodney Dangerfield, it gets no respect. Why? Log management is as geeky an IT service as there is as it hums along and collects, processes, and stores cryptic Syslog and Windows Log files. Insert yawn here.
Boring stuff -- yes but increasingly essential. Correlation and analysis of log events helps users detect security events and suspicious behavior. Analysis and reporting on log data is the heart of regulatory compliance. Logs also provide a view into the entire IT infrastructure. Analysis here can help CIOs tune their IT infrastructure and plan for the future.
Let's face it, log management is a foundational technology these days. Users who understand this and build an enterprise-class log management infrastructure will be better prepared for future security, IT, and business challenges.
Log management will eventually evolve into something much bigger. I envision an IT data warehouse in the future that collects log data but also network flows, CMDB records, asset information, vulnerability scans, SNMP events, etc. Users will load this into the IT equivalent of a Teradata warehouse and run all types of queries and reports.
As far as vendors go, I see good things from ArcSight, HP, LogLogic, LogRhythm, and Q1 Labs. Any one of these guys could emerge as the market leader for this future IT data warehouse vision.