Robust Dynamic Multi-Modal Data Fusion: A Model Uncertainty Perspective

This letter is concerned with multi-modal data fusion (MMDF) under unexpected modality failures in nonlinear non-Gaussian dynamic processes. An efficient framework to tackle this problem is proposed. In particular, a notion termed modality “<italic>usefulness</italic>,” which takes a value of 1 or 0, is used for indicating whether the observation of this modality is useful or not. For <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> modalities involved, <inline-formula><tex-math notation="LaTeX">$2^n$</tex-math></inline-formula> combinations of their “<italic>usefulness</italic>” values exist. Each combination defines one hypothetical model of the true data generative process. Then the problem of concern is formalized as a task of nonlinear non-Gaussian state filtering under model uncertainty, which is addressed by a dynamic model averaging (DMA) based particle filter (PF) algorithm. This DMA algorithm employs <inline-formula><tex-math notation="LaTeX">$2^n$</tex-math></inline-formula> models, while all models share the same state-transition function and a unique set of particle values. That makes its computational complexity only slightly larger than a single model based PF algorithm, especially for scenarios in which <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> is small. Experimental results show that the proposed solution outperforms remarkably state-of-the-art methods. Code and data are available at <uri>https://github.com/robinlau1981/fusion</uri>.

[1]  M. Kumar,et al.  A Method for Judicious Fusion of Inconsistent Multiple Sensor Data , 2007, IEEE Sensors Journal.

[2]  Bin Liu,et al.  A Survey on Trust Modeling from a Bayesian Perspective , 2018, Wirel. Pers. Commun..

[3]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[4]  Mónica F. Bugallo,et al.  Sequential Monte Carlo methods under model uncertainty , 2016, 2016 IEEE Statistical Signal Processing Workshop (SSP).

[5]  W. Marsden.  I and J , 2012 .

[6]  W. Hager,et al.  and s , 2019, Shallow Water Hydraulics.

[8]  R. Sarpong,et al.  Bio-inspired synthesis of xishacorenes A, B, and C, and a new congener from fuscol† †Electronic supplementary information (ESI) available. See DOI: 10.1039/c9sc02572c , 2019, Chemical science.

[9]  Fuad E. Alsaadi,et al.  Particle Filtering for Nonlinear/Non-Gaussian Systems With Energy Harvesting Sensors Subject to Randomly Occurring Sensor Saturations , 2021, IEEE Trans. Signal Process..

[10]  Yunpeng Li,et al.  Particle Filtering With Invertible Particle Flow , 2016, IEEE Transactions on Signal Processing.

[11]  Wen-an Zhang,et al.  Robust Information Fusion Estimator for Multiple Delay-Tolerant Sensors With Different Failure Rates , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[12]  Bin Liu.  Robust Particle Filtering via Bayesian Nonparametric Outlier Modeling (Poster) , 2019, 2019 22th International Conference on Information Fusion (FUSION).

[13]  Martin Brown,et al.  Fusion of soft information using TBM , 2010, 2010 13th International Conference on Information Fusion.

[14]  Subhash Challa,et al.  Empirical analysis of generalised uncertainty measures with dempster shafer fusion , 2007, 2007 10th International Conference on Information Fusion.

[15]  Bin Liu,et al.  A particle filter using SVD based sampling Kalman filter to obtain the proposal distribution , 2008, 2008 IEEE Conference on Cybernetics and Intelligent Systems.

[16]  Bin Liu,et al.  State Space Model based Trust Evaluation over Wireless Sensor Networks: An Iterative Particle Filter Approach , 2016, ArXiv.

[17]  Simon Maskell,et al.  Fusion of data from sources with different levels of trust , 2010, 2010 13th International Conference on Information Fusion.

[18]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[19]  Junjie Chen,et al.  Toward reliable data analysis for Internet of Things by Bayesian dynamic modeling and computation , 2015, 2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP).

[20]  Audun Jøsang,et al.  Interpretation and fusion of hyper opinions in subjective logic , 2012, 2012 15th International Conference on Information Fusion.

[21]  Bin Liu,et al.  Dynamic Ensemble Modeling Approach to Nonstationary Neural Decoding in Brain-Computer Interfaces , 2019, NeurIPS.

[22]  Bin Liu,et al.  Online Fault-Tolerant Dynamic Event Region Detection in Sensor Networks via Trust Model , 2016, 2017 IEEE Wireless Communications and Networking Conference (WCNC).

[23]  Bin Liu,et al.  Instantaneous Frequency Tracking under Model Uncertainty via Dynamic Model Averaging and Particle Filtering , 2011, IEEE Transactions on Wireless Communications.

[24]  D. Madigan,et al.  Bayesian Model Averaging for Linear Regression Models , 1997 .

[25]  D.P. Garg,et al.  A generalized approach for inconsistency detection in data fusion from multiple sensors , 2006, 2006 American Control Conference.

[26]  Simon Maskell,et al.  The No-U-Turn Sampler as a Proposal Distribution in a Sequential Monte Carlo Sampler with a Near-Optimal L-Kernel , 2021 .

[27]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[28]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[29]  Edward P. Herbst,et al.  Tempered Particle Filtering , 2017, Journal of Econometrics.

[30]  Petar M. Djuric,et al.  Particle Filtering Under General Regime Switching , 2020, 2020 28th European Signal Processing Conference (EUSIPCO).

[31]  Bin Liu.  Data-Driven Model Set Design for Model Averaged Particle Filter , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  Bin Liu.  ILAPF: Incremental Learning Assisted Particle Filtering , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Nando de Freitas,et al.  The Unscented Particle Filter , 2000, NIPS.

[34]  Sibel Adali,et al.  A Survey on Trust Modeling , 2015, ACM Comput. Surv..

[35]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[36]  Bin Liu,et al.  Robust particle filter by dynamic averaging of multiple noise models , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).