Linear-Quadratic-Gaussian (LQG) control is concerned with the design of an optimal controller and estimator for linear Gaussian systems with imperfect state information. Standard LQG control assumes the set of sensor measurements to be fed to the estimator to be given. However, in many problems in networked systems and robotics, one is interested in designing a suitable set of sensors for LQG control. In this paper, we introduce the LQG control and sensing co-design problem, where one has to jointly design a suitable sensing, estimation, and control policy. We consider two dual instances of the co-design problem: the sensing-constrained LQG control problem, where the design maximizes the control performance subject to sensing constraints, and the minimum-sensing LQG control, where the design minimizes the amount of sensing subject to performance constraints. We focus on the case in which the sensing design has to be selected among a finite set of possible sensing modalities, where each modality is associated with a different cost. While we observe that the computation of the optimal sensing design is intractable in general, we present the first scalable LQG co-design algorithms to compute near-optimal policies with provable sub-optimality guarantees. To this end, (i) we show that a separation principle holds, which partially decouples the design of sensing, estimation, and control; (ii) we frame LQG co-design as the optimization of (approximately) supermodular set functions; (iii) we develop novel algorithms to solve the resulting optimization problems; (iv) we prove original results on the performance of these algorithms and establish connections between their sub-optimality gap and control-theoretic quantities. We conclude the paper by discussing two practical applications of the co-design problem, namely, sensing-constrained formation control and resource-constrained robot navigation.