Historically, log data has been viewed by IT professionals as a valuable asset in the areas of security information and event management. And while there is no denying the benefits of log data for security teams, I suggest that organizations also consider logs as an important source for managing the performance of their infrastructures.
By definition, logs are a record of all user transactions, customer and machine behavior, security threats, fraudulent activity and more. Applications, systems, and network devices produce enormous volumes of unstructured log data. And it's this unstructured data that presents a challenge to properly categorize and mine for intelligence. But when a performance-based log analytics platform can collect and analyze unstructured log data, that data becomes a valuable resource for you to better predict, detect, troubleshoot and resolve network and data center issues.
According to Jim Frey, Vice President of Research at Enterprise Management Associates (EMA), organizations should ensure that log analytics is a key component of their overall performance monitoring strategy. To this point, research from EMA has found that there is strong and growing interest in leveraging log data across multiple infrastructure troubleshooting and operations management uses cases.
However, it's not if – but how – you incorporate log analytics into your performance monitoring process that produces the greatest results.
Many organizations today leverage log search solutions, but the reality is that it takes a lot of time, effort, and education on your part to get value from log data. For instance, you're required to manually search log data after an event takes place – this often requires knowledge of a complex and vendor-specific query language. Essentially, you have the tools to help put out the fires, but wouldn't you rather detect the smoke beforehand?
Another issue with log search solutions is that you must manually compile log reports and then correlate performance metrics to that log data – another time-intensive effort.
Based on the numerous challenges inherent with traditional log search solutions, I suggest organizations look for a performance-based log analytics platform that provides, with a single click, the ability to pivot from real-time performance metrics (such as SNMP or an IP SLA test) to the related log records, and without the time-consuming search and manual correlation typically associated with log tools. Your success with log analytics should be measured by the extent to which you can automate the extraction of actionable insight from logs at the point of ingestion. Your ability to guarantee the performance of your infrastructure depends on a more proactive approach than what we've seen from many log "analytics" tools on the market today.
Vess Bakalov is Senior Vice President, CTO and Co-Founder of SevOne.