Transformer Models with Explainability for IT Telemetry and Business Events

Abstract

Temporal event data are commonly encountered in software applications across a wide range of domains, from IT telemetry and system logs to business process automation. Temporal event data in software applications carry information in various forms: both structured and unstructured, and with both regular and irregular occurrence frequency, making the representation of temporal event data challenging. Further challenges come from the diversity in terms of the scale and volume of events that need to be summarized by the representation. We propose a general and unified approach to handle temporal event data for the purpose of learning a predictive transformer model. Our approach subsumes many existing techniques and is relevant across diverse application domains. Further, our approach can be used with many transformer architectures from the original transformer to recent, more complex architectures. Through a simple extension of the transformer training procedure, we enable the model to provide explanations of its predicted events. Empirically, we show that the proposed approach achieves state-of-the-art performance on software-related tasks coming from a wide variety of application domains.