About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
GOMACTech 2023
Conference paper
Millimeter-wave and IR Multimodal Sensing System Enabling AI-based Event Recognition with Enhanced Privacy
Abstract
Technologies for sensing and monitoring human-related activities and vehicles/objects are an important and increasing part of urban management and safety. It is desirable to enable these systems to operate without visible-domain cameras for enhanced privacy and robustness to varying illumination and weather conditions. In response to these requirements, we introduce a multimodal sensing system that comprises 60-GHz phased array TX and RX modules, a passive IR camera, a COTS-based FMCW radar sub-system, a mixed-signal data acquisition module, and an FPGA. The FPGA’s key role is to accurately control radar waveform generation, phased array beam steering, and IR camera data acquisition events, producing a synchronized high-speed stream of 3D radar (>3 Kfps) and IR (9 fps) data. The prototype system and an associated multi-modal DNN are evaluated in two use cases (1) concealed object detection and (2) head gesture classification.