Deep neural networks have gained increased attention in machine learning, but they are limited by the fact that many such regression and classification models do not capture prediction uncertainty. Though this might be acceptable for certain non-critical applications, it is not so for oil and gas industry applications where business and economic consequences of wrong or even sub-optimal decision is quite high. In this work I discuss the application of deep neural networks as a framework for approximate Bayesian inference in oilfield reservoir simulation study. Surrogate models with different neural network architecture are proposed to speed up compute- and labor-intensive simulation workflow. Regularization tools such as dropout and batch normalization, variational autoencoder for regression, and probabilistic distribution layers are used to quantify prediction uncertainty. Monte-Carlo dropout approach is further applied to estimate uncertainty given by standard deviation values for the predictions. Probabilistic distribution layers are used to compare its efficacy in capturing the model prediction uncertainty with respect to deterministic neural layers. Deep ensemble approach is also used to train multiple surrogates which capture uncertainty. Among different models tested, VAE based regression model with multivariate-normal latent features works best for prediction uncertainty assessment. Compute time required by surrogate model for prediction is a small fraction of that for full-physics reservoir simulator. Prediction uncertainty information can be used in various simulation workflows to decide when to use surrogate model and when to further explore the solution space using reservoir simulator, thus reducing total computational cost.