Federated Learning for distribution skewed data using sample weights

One of the most challenging issues in federated learning is that the data is often not independent and identically distributed (nonIID). Clients are expected to contribute the same type of data and drawn from one global distribution. However, data are often collected in different ways from different resources. Thus, the data distributions among clients might be different from the underlying global distribution. This creates a weight divergence issue and reduces federated learning performance. This work focuses on improving federated learning performance for skewed data distribution across clients. The main idea is to adjust the client distribution closer to the global distribution using sample weights. Thus, the machine learning model converges faster with higher accuracy. We start from the fundamental concept of empirical risk minimization and theoretically derive a solution for adjusting the distribution skewness using sample weights. To determine sample weights, we implicitly exchange density information by leveraging a neural network-based density estimation model, MADE. The clients data distribution can then be adjusted without exposing their raw data. Our experiment results on three real-world datasets show that the proposed method not only improves federated learning accuracy but also significantly reduces communication costs compared to the other experimental methods.

AMPose: Alternatively Mixed Global-Local Attention Model for 3D Human Pose Estimation

The graph convolutional network has been applied to 3D human pose estimation. In addition, the pure transformer model recently show the promising result in the video-based method. However, the single-frame method still need to model the physically connected relations among joints because the feature representation transformed only by the global attention has the lack of the relationships of human skeleton. We propose a novel architecture to combine the physically connected and global relations among joints in human. We evaluate our method on Human3.6m and compare with the state-of-the-art models. Our model show superior result over all other models. Our model has better generalization ability by cross-dataset comparison on MPI-INF-3DHP.