We present a novel approach to the cloth simulation problem in human-centric scenarios through deep learning. Computer graphics approaches rely on Physically Based Simulations (PBS) to animate clothes. These are general solutions that, given a sufficiently fine-grained discretization of space and time, can achieve highly realistic results. However, they are computationally expensive and any scene modification prompts the need of re-simulation. We propose using deep learning, formulated as an implicit PBS, to learn accurate cloth deformations in a constrained scenario: dressed humans. By using deep models, we can obtain high-resolution garments that can be efficiently deployed in real-time. Furthermore, we show it is possible to train these models in an amount of time comparable to a PBS of a few fixed sequences. To the best of our knowledge, we are the first to propose a neural simulator for cloth. Other deep-based approaches for cloth dynamics learn the distribution of huge volumes of simulated data. Therefore, these approaches require a great investment of computational resources for data gathering. Alternatively, data can be gathered through expensive 4D scans in constrained scenarios. With our proposed methodology, we completely skip the data gathering part while obtaining appealing results.