About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
CLOUD 2024
Short paper
Decoding Logs for Automatic Metric Identification
Abstract
Automated Log Analysis tasks such as root cause analysis, fault prediction, etc. play a pivotal role in maintaining the overall application health. These tasks employ log parsers that extract the dynamic (variable) and constant (template) parts of a log line. However, our observations indicate that not all of them carry equal significance. Hence, there is a need to prioritize which templates/variables to use for log analysis. In this paper, we introduce LogMId, a Logs-based Metric Identification method, which is designed to extract critical IT metrics from logs. Through LogMId, we aim to enhance monitoring, observability tools and in turn Site Reliability Engineers to mine better insights from log data. We showcase the effectiveness of LogMId on a popular log analysis task of anomaly detection. Our experiments indicate that integrating previously used benchmark tools with LogMId features leads to improved results. Additionally, LogMId demonstrates effectiveness even with a smaller amount of training data, emphasising its utility.