POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging

Fine-tuning models on edge devices like mobile phones would enable privacy-preserving personalization over sensitive data. However, edge training has historically been limited to relatively small models with simple architectures because training is both memory and energy intensive. We present POET, an algorithm to enable training large neural networks on memory-scarce battery-operated edge devices. POET jointly optimizes the integrated search search spaces of rematerialization and paging, two algorithms to reduce the memory consumption of backpropagation. Given a memory budget and a run-time constraint, we formulate a mixed-integer linear program (MILP) for energy-optimal training. Our approach enables training significantly larger models on embedded devices while reducing energy consumption while not modifying mathematical correctness of backpropagation. We demonstrate that it is possible to fine-tune both ResNet-18 and BERT within the memory constraints of a Cortex-M class embedded device while outperforming current edge training methods in energy efficiency. POET is an open-source project available at https://github.com/ShishirPatil/poet

New Directions: Wireless Robotic Materials

We describe opportunities and challenges with wireless robotic materials. Robotic materials are multi-functional composites that tightly integrate sensing, actuation, computation and communication to create smart composites that can sense their environment and change their physical properties in an arbitrary programmable manner. Computation and communication in such materials are based on miniature, possibly wireless, devices that are scattered in the material and interface with sensors and actuators inside the material. Whereas routing and processing of information within the material build upon results from the field of sensor networks, robotic materials are pushing the limits of sensor networks in both size (down to the order of microns) and numbers of devices (up to the order of millions). In order to solve the algorithmic and systems challenges of such an approach, which will involve not only computer scientists, but also roboticists, chemists and material scientists, the community requires a common platform - much like the "Mote" that bootstrapped the widespread adoption of the field of sensor networks - that is small, provides ample of computation, is equipped with basic networking functionalities, and preferably can be powered wirelessly.