In this paper, we studied the federated stochastic bilevel optimization problem. In particular, we developed two momentum-based algorithms for optimizing this kind of problem. In addition, we established the convergence rate of these two algorithms, providing their sample and communication complexities. To the best of our knowledge, this is the first work achieving such favorable theoretical results.