* Some of the results have appeared in AISTATS 2018. This is an
extended version with additional results, in particular, an adaptive
synchronicity strategy called AdaSync. arXiv admin note: substantial text
overlap with arXiv:1803.01113 Access Paper or Ask Questions
* arXiv admin note: text overlap with arXiv:1808.05904 A special case
of the model studied in this paper is presented in arXiv:1808.05904 Access Paper or Ask Questions
