Identifying malicious activities from system execution traces

Abstract

Every day, massive amounts of system events from software agents deployed at endpoint devices across the world are received by the IBM Trusteer security group. The software associated with each event is verified with respect to third-party malware inspection services such as VirusTotal. Unfortunately, many events are associated with software that is unrecognized by inspection services. As a result, it is impossible to manually investigate and react to all of them. Traditional quantitative analysis is nearly useless because benign anomalies and attacks are indiscernible. We developed a system that continuously and automatically processes streaming data to help identify suspicious activity. The data comprises low-level traces of process activity. Each streamed activity is augmented with a signature that heuristically biases the degree of suspicion associated with the activity. The system then flags activities that are unknown to inspection services and likely to be malicious. It extracts behavioral and statistical information from the events, builds a predictive model based on supervised learning, and ranks the events suspected of being malicious. We tested the system using VirusTotal on three months of historical data. The results showed we were able to predict more than two thirds of the malicious events unknown at that time, with less than a 2% false positive rate.