Unbalanced Optimal Transport using Integral Probability Metric Regularization

Unbalanced Optimal Transport (UOT) is the generalization of classical optimal transport to un-normalized measures. Existing formulations employ KL (or, in general, $\phi$-divergence) based regularization for handling the mismatch in the total mass of the given measures. Motivated by the known advantages of Integral Probability Metrics (IPMs) over $\phi$-divergences, this paper proposes novel formulations for UOT that employ IPMs for regularization. Under mild conditions, we show that the proposed formulations lift the ground metric to metrics over measures. More interestingly, the induced metric turns out to be another IPM whose generating set is the intersection of that of the IPM employed and the set of 1-Lipschitz functions under the ground metric. We further generalize these metrics to obtain analogues of p-Wasserstein metrics for the unbalanced case. When the regularizing IPM is chosen as the MMD, the proposed UOT as well as the corresponding Barycenter formulations, turn out to be those of minimizing a convex quadratic subject to non-negativity constraints and hence can be solved very efficiently. We also discuss connections with formulations for robust optimal transport in the case of noisy marginals.

Statistical Optimal Transport posed as Learning Kernel Embedding

This work takes the novel approach of posing the statistical Optimal Transport (OT) problem as that of learning the transport plan's kernel mean embedding. The key advantage is that the estimates for the embeddings of the marginals can now be employed directly, leading to a dimension-free sample complexity for the proposed transport plan and transport map estimators. Also, because of the implicit smoothing in the kernel embeddings, the proposed estimators can perform out-of-sample estimation. Interestingly, the proposed formulation employs an MMD based regularization to avoid overfitting, which is complementary to existing $\phi$-divergence (entropy) based regularization techniques. An appropriate representer theorem is presented that leads to a fully kernelized formulation and hence the same formulation can be used to perform continuous/semi-discrete/discrete OT in any non-standard domain (as long as universal kernels in those domains are known). Finally, an ADMM based algorithm is presented for solving the kernelized formulation efficiently. Empirical results show that the proposed estimator outperforms discrete OT based estimator in terms of transport map accuracy.

A Convex Feature Learning Formulation for Latent Task Structure Discovery

This paper considers the multi-task learning problem and in the setting where some relevant features could be shared across few related tasks. Most of the existing methods assume the extent to which the given tasks are related or share a common feature space to be known apriori. In real-world applications however, it is desirable to automatically discover the groups of related tasks that share a feature space. In this paper we aim at searching the exponentially large space of all possible groups of tasks that may share a feature space. The main contribution is a convex formulation that employs a graph-based regularizer and simultaneously discovers few groups of related tasks, having close-by task parameters, as well as the feature space shared within each group. The regularizer encodes an important structure among the groups of tasks leading to an efficient algorithm for solving it: if there is no feature space under which a group of tasks has close-by task parameters, then there does not exist such a feature space for any of its supersets. An efficient active set algorithm that exploits this simplification and performs a clever search in the exponentially large space is presented. The algorithm is guaranteed to solve the proposed formulation (within some precision) in a time polynomial in the number of groups of related tasks discovered. Empirical results on benchmark datasets show that the proposed formulation achieves good generalization and outperforms state-of-the-art multi-task learning algorithms in some cases.