In this paper, we propose a novel large deformation diffeomorphic registration algorithm to align high angular resolution diffusion images (HARDI) characterized by orientation distribution functions (ODFs). Our proposed algorithm seeks an optimal diffeomorphism of large deformation between two ODF fields in a spatial volume domain and at the same time, locally reorients an ODF in a manner such that it remains consistent with the surrounding anatomical structure. To this end, we first review the Riemannian manifold of ODFs. We then define the reorientation of an ODF when an affine transformation is applied and subsequently, define the diffeomorphic group action to be applied on the ODF based on this reorientation. We incorporate the Riemannian metric of ODFs for quantifying the similarity of two HARDI images into a variational problem defined under the large deformation diffeomorphic metric mapping (LDDMM) framework. We finally derive the gradient of the cost function in both Riemannian spaces of diffeomorphisms and the ODFs, and present its numerical implementation. Both synthetic and real brain HARDI data are used to illustrate the performance of our registration algorithm.