Contemporary network architectures are pushing computing tasks from the cloud towards the network edge, leveraging the increased processing capabilities of edge devices to meet rising user demands. Of particular importance are machine learning (ML) tasks, which are becoming ubiquitous in networked applications ranging from content recommendation systems to intelligent vehicular communications. Federated learning has emerged recently as a technique for training ML models by leveraging processing capabilities across the nodes that collect the data. There are several challenges with employing federated learning at the edge, however, due to the significant heterogeneity in compute and communication capabilities that exist across devices. To address this, we advocate a new learning paradigm called {fog learning which will intelligently distribute ML model training across the fog, the continuum of nodes from edge devices to cloud servers. Fog learning is inherently a multi-stage learning framework that breaks down the aggregations of heterogeneous local models across several layers and can leverage data offloading within each layer. Its hybrid learning paradigm transforms star network topologies used for parameter transfers in federated learning to more distributed topologies. We also discuss several open research directions for fog learning.