We consider a novel approach to formulate the Cram\'er-Rao Lower Bound (CRLB) for the rigid body localization (RBL) problem, which allows us to assess the fundamental accuracy limits on the estimation of the translation and rotation of a rigid body with respect to a known reference. To that end, we adopt an information-centric construction of the Fisher information matrix (FIM), which allows to capture the contribution of each measurement towards the FIM, both in terms of input measurement types, as well as of their error distributions. Taking advantage of this approach, we derive a generic framework for the CRLB formulation, which is applicable to any type of rigid body localization scenario, extending the conventional FIM formulation suitable for point targets to the case of a rigid body whose location include both translation vector and the rotation matrix (or alternative the rotation angles), with respect to a reference. Closed-form expressions for all CRLBs are given, including the bound incorporating an orthonormality constraint onto the rotation matrix. Numerical results illustrate that the derived expression correctly lower-bounds the errors of estimated localization parameters obtained via various related state-of-the-art (SotA) estimators, revealing their accuracies and suggesting that SotA RBL algorithms can still be improved.