The idea behind procedural content generation (PCG) in games is to create content automatically, using algorithms, instead of relying on user-designed content. While PCG approaches have traditionally focused on creating content for video games, they are now being applied to all kinds of virtual environments, thereby enabling training of machine learning systems that are significantly more general. For example, PCG's ability to generate never-ending streams of new levels has allowed DeepMind's Capture the Flag agent to reach beyond human-level-performance. Additionally, PCG-inspired methods such as domain randomization enabled OpenAI's robot arm to learn to manipulate objects with unprecedented dexterity. Level generation in 2D arcade games has also illuminated some shortcomings of standard deep RL methods, suggesting potential ways to train more general policies. This Review looks at key aspect of PCG approaches, including its ability to (1) enable new video games (such as No Man's Sky), (2) create open-ended learning environments, (3) combat overfitting in supervised and reinforcement learning tasks, and (4) create better benchmarks that could ultimately spur the development of better learning algorithms. We hope this article can introduce the broader machine learning community to PCG, which we believe will be a critical tool in creating a more general machine intelligence.