Interacting agent and particle systems are extensively used to model complex phenomena in science and engineering. We consider the problem of learning interaction kernels in these dynamical systems constrained to evolve on Riemannian manifolds from given trajectory data. Our approach generalizes the theory and algorithms in [1] introduced in the Euclidean setting. The models we consider are based on interaction kernels depending on pairwise Riemannian distances between agents, with agents interacting locally along the direction of the shortest geodesic connecting them. We show that our estimators converge at a rate that is independent of the dimension of the manifold, and derive bounds on the trajectory estimation error, on the manifold, between the observed and estimated dynamics. We demonstrate highly accurate performance of the learning algorithm on three classical first order interacting systems, Opinion Dynamics, Lennard-Jones Dynamics, and a Predator-Swarm system, with each system constrained on two prototypical manifolds, the $2$-dimensional sphere and the Poincar\'e disk model of hyperbolic space. [1] F. Lu, M. Zhong, S. Tang, M. Maggioni, Nonparametric Inference of Interaction Laws in Systems of Agents from Trajectory Data, PNAS, 116 (2019), pp. 14424 - 14433.