Algorithmic recourse aims to recommend actionable changes to a factual's attributes that flip an unfavorable model decision while remaining realistic and feasible. We formulate recourse as a Constrained Maximum A-Posteriori (MAP) inference problem under the accepted-class data distribution seeking counterfactuals with high likelihood while respecting other recourse constraints. We present PAR, an amortized approximate inference procedure that generates highly likely recourses efficiently. Recourse likelihood is estimated directly using tractable probabilistic models that admit exact likelihood evaluation and efficient gradient propagation that is useful during training. The recourse generator is trained with the objective of maximizing the likelihood under the accepted-class distribution while minimizing the likelihood under the denied-class distribution and other losses that encode recourse constraints. Furthermore, PAR includes a neighborhood-based conditioning mechanism to promote recourse generation that is customized to a factual. We validate PAR on widely used algorithmic recourse datasets and demonstrate its efficiency in generating recourses that are valid, similar to the factual, sparse, and highly plausible, yielding superior performance over existing state-of-the-art approaches.