FPGA-based computing in computer vision

Abstract

Algorithms in computer vision are characterized by (i) complex and repetitive operations; (ii) large amount of data and (iii) a variety of data interaction (e.g., point operations, neighborhood operations, global operations). Based on the computation and communication complexity, vision algorithms have been characterized into three categories: (i) low-level, (ii) intermediate-level and (iii) high-level. In this paper, we describe the usage of custom computing approach to meet the computation and communication needs of computer vision algorithms. By customizing hardware architecture for every application at the instruction level, the optimal grain size needed for the problem at hand and the instruction granularity can be matched. Field Programmable Gate Array (FPGA)-based precessing elements (PEs) are being used to provide this facility. Using programmable communication resources, the diverse communication requirements can be met. A vision system needs to integrate hardware for the three levels. A custom computing approach alleviates the problem of achieving optimal granularity for different stages as the same hardware gets reconfigured at a software level for different levels of the application. We demonstrate the advantages of our approach using Splash 2 - a Xilinx 4010-based custom computer.