BL-DKF: A Robust IoT Sensor Data Anomaly Detection Method

As the deployment of Internet-of-Things (IoT) devices in intelligent systems from smart health to smart city expands, ensuring sensor data reliability is critical for accurate decision-making. Individual sensor data suffer data drift due to unwanted noises, faults and false data injections during deployment, which paves the way for exploiting inter-sensor data correlation to improve anomaly detection. Studies suggest that sensor readings are correlated in many applications. Existing methods employ data correlation among sensors in pre-processing steps but do not integrate it into the bedrock of the Kalman Filter (KF) state transitions. Such integration will provide a way to detect anomalies in dynamic and complex applications. This paper proposes a novel, two-stage anomaly detection system named BL-DKF to tackle these challenges. First, an innovative 2D dynamic KF (DKF) is introduced that incorporates inter-sensor data correlation for adaptive estimation through noise and fault/false data injections. DKF dynamically adjusts its process covariance matrix by analyzing correlation deviations, reducing its influence when the anomalies cause correlation drops to emphasize internal predictions. The estimates produced by DKF are fed to a compact Bidirectional LSTM (BL) model in the second stage, which analyzes anomalous patterns. We evaluate BL-DKF on two real-world sensor datasets, including the data collected from our IoT lab testbed. Results demonstrate that BL-DKF robustly detects nonlinear and physical anomaly injections, outperforming existing methods with an average of 9.36% detection accuracy improvement across four experiments. The proposed system presents an analytical solution to embed inter-sensor correlations into KF, advancing its effectiveness in many application domains.