Neural Belief Reasoner

This paper proposes a new generative model called neural belief reasoner (NBR). It differs from previous models in that it specifies a belief function rather than a probability distribution. Its implementation consists of neural networks, fuzzy-set operations and belief-function operations, and query-answering, sample-generation and training algorithms are presented. This paper studies NBR in two tasks. The first is a synthetic unsupervised-learning task, which demonstrates NBR's ability to perform multi-hop reasoning, reasoning with uncertainty and reasoning about conflicting information. The second is supervised learning: a robust MNIST classifier. Without any adversarial training, this classifier exceeds the state of the art in adversarial robustness as measured by the L2 metric, and at the same time maintains 99% accuracy on natural images. A proof is presented that, as capacity increases, NBR classifiers can asymptotically approach the best possible robustness.