What will the future of UAV cellular communications be? In this tutorial article, we address such a compelling yet difficult question by embarking on a journey from 5G to 6G and sharing a large number of realistic case studies supported by original results. We start by overviewing the status quo on UAV communications from an industrial standpoint, providing fresh updates from the 3GPP and detailing new 5G NR features in support of aerial devices. We then show the potential and the limitations of such features. In particular, we demonstrate how sub-6 GHz massive MIMO can successfully tackle cell selection and interference challenges, we showcase encouraging mmWave coverage evaluations in both urban and suburban/rural settings, and we examine the peculiarities of direct device-to-device communications in the sky. Moving on, we sneak a peek at next-generation UAV communications, listing some of the use cases envisioned for the 2030s. We identify the most promising 6G enablers for UAV communication, those expected to take the performance and reliability to the next level. For each of these disruptive new paradigms (non-terrestrial networks, cell-free architectures, artificial intelligence, reconfigurable intelligent surfaces, and THz communications), we gauge the prospective benefits for UAVs and discuss the main technological hurdles that stand in the way. All along, we distil our numerous findings into essential takeaways, and we identify key open problems worthy of further study.