The manifold of unitary and symmetric matrices: characterization, Riemannian optimization and application to BD-RIS design

This paper proposes and analyzes Riemannian optimization algorithms on the manifold of unitary and symmetric matrices, denoted ${\cal {U}}_s$, which naturally models the scattering matrices of passive and reciprocal devices such as beyond-diagonal reconfigurable intelligent surfaces (BD-RISs). Despite its relevance, the geometry of ${\cal {U}}_s$ has remained largely unexplored, and existing BD-RIS optimization methods either ignore the symmetry constraint or rely on costly Takagi-based parameterizations. We first provide a rigorous geometric characterization of ${\cal {U}}_s$, deriving its tangent space, a simple retraction, and closed-form expressions for geodesics. Building on these results, we develop two Riemannian manifold optimization (MO) algorithms tailored to ${\cal {U}}_s$: a line-search (LS) based scheme and a phase-optimization (PO) update along geodesics. We then apply the proposed framework to BD-RIS-assisted multiple-input multiple-output (MIMO) links, addressing sum-gain maximization, rate maximization, and minimum mean-square error problems, where they outperform existing approaches. Furthermore, we show that when the number of BD-RIS elements exceeds the total number of antennas, the optimal scattering matrix is low-rank, which motivates and enables efficient low-rank variants of the proposed algorithms.