Scalable Log Analysis with LLMs: A Case Study with IT Software Support

IT environments typically have logging mechanisms to monitor system health and detect issues. However, the huge volume of logs generated makes manual inspection impractical, highlighting the importance of automated log analysis in IT Software Support. In this paper, we propose leveraging Large Language Models (LLMs) for log data processing and issue diagnosis, enabling the generation of automated insights and summaries. We further present a methodology for efficiently running LLMs on CPUs to process massive log volumes in minimal time without compromising output quality. We share the insights and lessons learned from a production deployment - in production since March 2024 - scaled across 70 software products, processing over 2000 tickets for issue diagnosis, achieving a time savings of 350+ man hours and an estimated $15,000 per month in manpower costs compared to the traditional log analysis practices.