Convergence is a crucial issue in iterative algorithms. Damping is commonly employed to ensure the convergence of iterative algorithms. The conventional ways of damping are scalar-wise, and either heuristic or empirical. Recently, an analytically optimized vector damping was proposed for memory message-passing (iterative) algorithms. As a result, it yields a special class of covariance matrices called L-banded matrices. In this letter, we show these matrices have a number of elegant properties arising from their L-banded structure. In particular, compact analytic expressions for the determinant, the inverse and the LDL decomposition of L-banded matrices are derived. In addition, necessary and sufficient conditions for such a matrix to be definite, as well as some other properties, are given.