WinHeap explorer: Efficient and transparent heap-based bug detection in machine code

Abstract

Despite all the efforts of the research community, buffer overflows remain one of the most dangerous bugs for modern IT systems. The problem is compounded by the fact that there are many developers who do not follow the basic rules of a secure software development lifecycle, supplying proprietary vulnerable products. To address this problem, the industry has proposed a number of techniques that perform analysis at the binary level. While most of them focus on problems that lead a program to crash or exception, there is a large class of more complex bugs that it is not possible to detect using only this criterion.In this paper we propose WinHeap Explorer, a highperformance solution for heap based bug detection in machine code using an original approach called light-weight dynamic binary instrumentation. The light-weight instrumentation is based on preliminary static analysis of code paths to highlight potentially erroneous parts, due to which we are able to decrease the overhead. Moreover, WinHeap Explorer does not change any memory allocation mechanisms, which preserves transparency of the tool towards the operating system. Our experiments have proven the ability of WinHeap Explorer to detect heapbased bugs as well as decrease runtime overhead for widely-used complex applications ranging between 24.2%-71.1% along with the same level of memory overhead in comparison with existing solutions. WinHeap Explorer is distributed under BSD license and available at [1].