Deep neural networks (DNNs) are vulnerable against adversarial examples, i.e., examples that are carefully crafted to fool the DNNs while being indistinguishable from the natural images to humans. In this paper, we propose a new framework to detect adversarial examples motivated by the observations that random components can improve the smoothness of predictors and make it easier to simulate output distribution of deep neural network. With these observations, we propose a novel Bayesian adversarial example detector, short for BATector, to improve the performance of adversarial example detection. In specific, we study the distributional difference of hidden layer output between natural and adversarial examples, and propose to use the randomness of Bayesian neural network (BNN) to simulate hidden layer output distribution and leverage the distribution dispersion to detect adversarial examples. The advantage of BNN is that the output is stochastic while neural network without random components do not have such characteristics. Empirical results on several benchmark datasets against popular attacks show that, the proposed BATector outperforms the state-of-the-art detectors in adversarial example detection.