We present a class of methods for federated learning, which we call Fed+, pronounced FedPlus. The class of methods encompasses and unifies a number of recent algorithms proposed for federated learning and permits easily defining many new algorithms. The principal advantage of this class of methods is to better accommodate the real-world characteristics found in federated learning training, such as the lack of IID data across the parties in the federation. We demonstrate the use and benefits of this class of algorithms on standard benchmark datasets and a challenging real-world problem where catastrophic failure has a serious impact, namely in financial portfolio management.