Alert button

Multimodal Neural Processes for Uncertainty Estimation

Apr 04, 2023
Myong Chol Jung, He Zhao, Joanna Dipnall, Belinda Gabbe, Lan Du

Figure 1 for Multimodal Neural Processes for Uncertainty Estimation
Figure 2 for Multimodal Neural Processes for Uncertainty Estimation
Figure 3 for Multimodal Neural Processes for Uncertainty Estimation
Figure 4 for Multimodal Neural Processes for Uncertainty Estimation

Share this with someone who'll enjoy it:

Neural processes (NPs) have brought the representation power of parametric deep neural networks and the reliable uncertainty estimation of non-parametric Gaussian processes together. Although recent development of NPs has shown success in both regression and classification, how to adapt NPs to multimodal data has not be carefully studied. For the first time, we propose a new model of NP family for multimodal uncertainty estimation, namely Multimodal Neural Processes. In a holistic and principled way, we develop a dynamic context memory updated by the classification error, a multimodal Bayesian aggregation mechanism to aggregate multimodal representations, and a new attention mechanism for calibrated predictions. In extensive empirical evaluation, our method achieves the state-of-the-art multimodal uncertainty estimation performance, showing its appealing ability of being robust against noisy samples and reliable in out-of-domain detection.

View paper onarxiv icon

Share this with someone who'll enjoy it: