We present a Bayesian probabilistic model to estimate the brain white matter atlas from high angular resolution diffusion imaging (HARDI) data. This model incorporates a shape prior of the white matter anatomy and the likelihood of individual observed HARDI datasets. We first assume that the atlas is generated from a known hyperatlas through a flow of diffeomorphisms and its shape prior can be constructed based on the framework of large deformation diffeomorphic metric mapping (LDDMM). LDDMM characterizes a nonlinear diffeomorphic shape space in a linear space of initial momentum uniquely determining diffeomorphic geodesic flows from the hyperatlas. Therefore, the shape prior of the HARDI atlas can be modeled using a centered Gaussian random field (GRF) model of the initial momentum. In order to construct the likelihood of observed HARDI datasets, it is necessary to study the diffeomorphic transformation of individual observations relative to the atlas and the probabilistic distribution of orientation distribution functions (ODFs). To this end, we construct the likelihood related to the transformation using the same construction as discussed for the shape prior of the atlas. The probabilistic distribution of ODFs is then constructed based on the ODF Riemannian manifold. We assume that the observed ODFs are generated by an exponential map of random tangent vectors at the deformed atlas ODF. Hence, the likelihood of the ODFs can be modeled using a GRF of their tangent vectors in the ODF Riemannian manifold. We solve for the maximum a posteriori using the Expectation-Maximization algorithm and derive the corresponding update equations. Finally, we illustrate the HARDI atlas constructed based on a Chinese aging cohort of 94 adults and compare it with that generated by averaging the coefficients of spherical harmonics of the ODF across subjects.