Automated security debugging using program structural constraints

Abstract

Understanding security bugs in a vulnerable program is a non-trivial task, even if the target program is known to be vulnerable. Though there exist debugging tools that facilitate the vulnerability analysis and debugging process, human developers still need to manually trace the program execution most of the times. This makes security debugging a difficult and tiresome task even for experienced programmers. In this paper, we present the development of a novel security debugging tool called CBones (SeeBones, where bones is an analogy of program structures). CBones is intended to fully automate the analysis of a class of security vulnerabilities in C programs, the exploits of which would compromise the integrity of program structures satisfied by all legitimate binaries compiled from C source code. In other words, CBones automatically discovers how unknown vulnerabilities in C programs are exploited based on program structural constraints. Unlike the previous approaches, CBones can automatically identify exploit points of unknown security bugs without requiring a training phase, source code access (analysis or instrumentation), or additional hardware supports. To validate the effectiveness of this approach, we evaluate CBones with 12 real-world applications that contain a wide range of vulnerabilities. Our results show that CBones can discover all security bugs with no false alarms, pinpoint the corrupting instructions, and provide information to facilitate the understanding of how an attack exploits a security bug. © 2007 IEEE.