Horizontal Federated Learning and Secure Distributed Training for Recommendation System with Intel SGX

With the advent of big data era and the development of artificial intelligence and other technologies, data security and privacy protection have become more important. Recommendation systems have many applications in our society, but the model construction of recommendation systems is often inseparable from users' data. Especially for deep learning-based recommendation systems, due to the complexity of the model and the characteristics of deep learning itself, its training process not only requires long training time and abundant computational resources but also needs to use a large amount of user data, which poses a considerable challenge in terms of data security and privacy protection. How to train a distributed recommendation system while ensuring data security has become an urgent problem to be solved. In this paper, we implement two schemes, Horizontal Federated Learning and Secure Distributed Training, based on Intel SGX(Software Guard Extensions), an implementation of a trusted execution environment, and TensorFlow framework, to achieve secure, distributed recommendation system-based learning schemes in different scenarios. We experiment on the classical Deep Learning Recommendation Model (DLRM), which is a neural network-based machine learning model designed for personalization and recommendation, and the results show that our implementation introduces approximately no loss in model performance. The training speed is within acceptable limits.

A Systematic Analysis for State-of-the-Art 3D Lung Nodule Proposals Generation

Lung nodule proposals generation is the primary step of lung nodule detection and has received much attention in recent years . In this paper, we first construct a model of 3-dimension Convolutional Neural Network (3D CNN) to generate lung nodule proposals, which can achieve the state-of-the-art performance. Then, we analyze a series of key problems concerning the training performance and efficiency. Firstly, we train the 3D CNN model with data in different resolutions and find out that models trained by high resolution input data achieve better lung nodule proposals generation performances especially for nodules in too small sizes, while consumes much more memory at the same time. Then, we analyze the memory consumptions on different platforms and the experimental results indicate that CPU architecture can provide us with larger memory and enables us to explore more possibilities of 3D applications. We implement the 3D CNN model on CPU platform and propose an Intel Extended-Caffe framework which supports many highly-efficient 3D computations, which is opened source at https://github.com/extendedcaffe/extended-caffe.