Future, "NextG" cellular networks will be natively cloud-based and built upon programmable, virtualized, and disaggregated architectures. The separation of control functions from the hardware fabric and the introduction of standardized control interfaces will enable the definition of custom closed-control loops, which will ultimately enable embedded intelligence and real-time analytics, thus effectively realizing the vision of autonomous and self-optimizing networks. This article explores the NextG disaggregated architecture proposed by the O-RAN Alliance. Within this architectural context, it discusses potential, challenges, and limitations of data-driven optimization approaches to network control over different timescales. It also provides the first large-scale demonstration of the integration of O-RAN-compliant software components with an open-source full-stack softwarized cellular network. Experiments conducted on Colosseum, the world's largest wireless network emulator, demonstrate closed-loop integration of real-time analytics and control through deep reinforcement learning agents. We also demonstrate for the first time Radio Access Network (RAN) control through xApps running on the near real-time RAN Intelligent Controller (RIC), to optimize the scheduling policies of co-existing network slices, leveraging O-RAN open interfaces to collect data at the edge of the network.