A Roadmap for Big Model

With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the construction of BMs and the BM application in many fields. At present, there is a lack of research work that sorts out the overall progress of BMs and guides the follow-up research. In this paper, we cover not only the BM technologies themselves but also the prerequisites for BM training and applications with BMs, dividing the BM review into four parts: Resource, Models, Key Technologies and Application. We introduce 16 specific BM-related topics in those four parts, they are Data, Knowledge, Computing System, Parallel Training System, Language Model, Vision Model, Multi-modal Model, Theory&Interpretability, Commonsense Reasoning, Reliability&Security, Governance, Evaluation, Machine Translation, Text Generation, Dialogue and Protein Research. In each topic, we summarize clearly the current studies and propose some future research directions. At the end of this paper, we conclude the further development of BMs in a more general view.

Vertical Federated Learning: Challenges, Methodologies and Experiments

Recently, federated learning (FL) has emerged as a promising distributed machine learning (ML) technology, owing to the advancing computational and sensing capacities of end-user devices, however with the increasing concerns on users' privacy. As a special architecture in FL, vertical FL (VFL) is capable of constructing a hyper ML model by embracing sub-models from different clients. These sub-models are trained locally by vertically partitioned data with distinct attributes. Therefore, the design of VFL is fundamentally different from that of conventional FL, raising new and unique research issues. In this paper, we aim to discuss key challenges in VFL with effective solutions, and conduct experiments on real-life datasets to shed light on these issues. Specifically, we first propose a general framework on VFL, and highlight the key differences between VFL and conventional FL. Then, we discuss research challenges rooted in VFL systems under four aspects, i.e., security and privacy risks, expensive computation and communication costs, possible structural damage caused by model splitting, and system heterogeneity. Afterwards, we develop solutions to addressing the aforementioned challenges, and conduct extensive experiments to showcase the effectiveness of our proposed solutions.

A Wearable ECG Monitor for Deep Learning Based Real-Time Cardiovascular Disease Detection

Cardiovascular disease has become one of the most significant threats endangering human life and health. Recently, Electrocardiogram (ECG) monitoring has been transformed into remote cardiac monitoring by Holter surveillance. However, the widely used Holter can bring a great deal of discomfort and inconvenience to the individuals who carry them. We developed a new wireless ECG patch in this work and applied a deep learning framework based on the Convolutional Neural Network (CNN) and Long Short-term Memory (LSTM) models. However, we find that the models using the existing techniques are not able to differentiate two main heartbeat types (Supraventricular premature beat and Atrial fibrillation) in our newly obtained dataset, resulting in low accuracy of 58.0 %. We proposed a semi-supervised method to process the badly labelled data samples with using the confidence-level-based training. The experiment results conclude that the proposed method can approach an average accuracy of 90.2 %, i.e., 5.4 % higher than the accuracy of conventional ECG classification methods.

Negative-ResNet: Noisy Ambulatory Electrocardiogram Signal Classification Scheme

With recently successful applications of deep learning in computer vision and general signal processing, deep learning has shown many unique advantages in medical signal processing. However, data labelling quality has become one of the most significant issues for AI applications, especially when it requires domain knowledge (e.g. medical image labelling). In addition, there might be noisy labels in practical datasets, which might impair the training process of neural networks. In this work, we propose a semi-supervised algorithm for training data samples with noisy labels by performing selected Positive Learning (PL) and Negative Learning (NL). To verify the effectiveness of the proposed scheme, we designed a portable ECG patch -- iRealCare -- and applied the algorithm on a real-life dataset. Our experimental results show that we can achieve an accuracy of 91.0 %, which is 6.2 % higher than a normal training process with ResNet. There are 65 patients in our dataset and we randomly picked 2 patients to perform validation.

Path Planning for the Dynamic UAV-Aided Wireless Systems using Monte Carlo Tree Search

For UAV-aided wireless systems, online path planning attracts much attention recently. To better adapt to the real-time dynamic environment, we, for the first time, propose a Monte Carlo Tree Search (MCTS)-based path planning scheme. In details, we consider a single UAV acts as a mobile server to provide computation tasks offloading services for a set of mobile users on the ground, where the movement of ground users follows a Random Way Point model. Our model aims at maximizing the average throughput under energy consumption and user fairness constraints, and the proposed timesaving MCTS algorithm can further improve the performance. Simulation results show that the proposed algorithm achieves a larger average throughput and a faster convergence performance compared with the baseline algorithms of Q-learning and Deep Q-Network.

Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks

Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements. In this work, we present a systematical reproduction of 12 recent HGNNs by using their official codes, datasets, settings, and hyperparameters, revealing surprising findings about the progress of HGNNs. We find that the simple homogeneous GNNs, e.g., GCN and GAT, are largely underestimated due to improper settings. GAT with proper inputs can generally match or outperform all existing HGNNs across various scenarios. To facilitate robust and reproducible HGNN research, we construct the Heterogeneous Graph Benchmark (HGB), consisting of 11 diverse datasets with three tasks. HGB standardizes the process of heterogeneous graph data splits, feature processing, and performance evaluation. Finally, we introduce a simple but very strong baseline Simple-HGN--which significantly outperforms all previous models on HGB--to accelerate the advancement of HGNNs in the future.

Federated Learning for Smart Healthcare: A Survey

Recent advances in communication technologies and Internet-of-Medical-Things have transformed smart healthcare enabled by artificial intelligence (AI). Traditionally, AI techniques require centralized data collection and processing that may be infeasible in realistic healthcare scenarios due to the high scalability of modern healthcare networks and growing data privacy concerns. Federated Learning (FL), as an emerging distributed collaborative AI paradigm, is particularly attractive for smart healthcare, by coordinating multiple clients (e.g., hospitals) to perform AI training without sharing raw data. Accordingly, we provide a comprehensive survey on the use of FL in smart healthcare. First, we present the recent advances in FL, the motivations, and the requirements of using FL in smart healthcare. The recent FL designs for smart healthcare are then discussed, ranging from resource-aware FL, secure and privacy-aware FL to incentive FL and personalized FL. Subsequently, we provide a state-of-the-art review on the emerging applications of FL in key healthcare domains, including health data management, remote health monitoring, medical imaging, and COVID-19 detection. Several recent FL-based smart healthcare projects are analyzed, and the key lessons learned from the survey are also highlighted. Finally, we discuss interesting research challenges and possible directions for future FL research in smart healthcare.

Federated Learning for COVID-19 Detection with Generative Adversarial Networks in Edge Cloud Computing

COVID-19 has spread rapidly across the globe and become a deadly pandemic. Recently, many artificial intelligence-based approaches have been used for COVID-19 detection, but they often require public data sharing with cloud datacentres and thus remain privacy concerns. This paper proposes a new federated learning scheme, called FedGAN, to generate realistic COVID-19 images for facilitating privacy-enhanced COVID-19 detection with generative adversarial networks (GANs) in edge cloud computing. Particularly, we first propose a GAN where a discriminator and a generator based on convolutional neural networks (CNNs) at each edge-based medical institution alternatively are trained to mimic the real COVID-19 data distribution. Then, we propose a new federated learning solution which allows local GANs to collaborate and exchange learned parameters with a cloud server, aiming to enrich the global GAN model for generating realistic COVID-19 images without the need for sharing actual data. To enhance the privacy in federated COVID-19 data analytics, we integrate a differential privacy solution at each hospital institution. Moreover, we propose a new blockchain-based FedGAN framework for secure COVID-19 data analytics, by decentralizing the FL process with a new mining solution for low running latency. Simulations results demonstrate the superiority of our approach for COVID-19 detection over the state-of-the-art schemes.

BEdgeHealth: A Decentralized Architecture for Edge-based IoMT Networks Using Blockchain

The healthcare industry has witnessed significant transformations in e-health services by using mobile edge computing (MEC) and blockchain to facilitate healthcare operations. Many MEC-blockchain-based schemes have been proposed, but some critical technical challenges still remain, such as low quality of services (QoS), data privacy and system security vulnerabilities. In this paper, we propose a new decentralized health architecture, called BEdgeHealth that integrates MEC and blockchain for data offloading and data sharing in distributed hospital networks. First, a data offloading scheme is proposed where mobile devices can offload health data to a nearby MEC server for efficient computation with privacy awareness. Moreover, we design a data sharing scheme which enables data exchanges among healthcare users by leveraging blockchain and interplanetary file system. Particularly, a smart contract-based authentication mechanism is integrated with MEC to perform decentralized user access verification at the network edge without requiring any central authority. The real-world experiment results and evaluations demonstrate the effectiveness of the proposed BEdgeHealth architecture in terms of improved QoS with data privacy and security guarantees, compared to the existing schemes.

Cooperative Task Offloading and Block Mining in Blockchain-based Edge Computing with Multi-agent Deep Reinforcement Learning

The convergence of mobile edge computing (MEC) and blockchain is transforming the current computing services in mobile networks, by offering task offloading solutions with security enhancement empowered by blockchain mining. Nevertheless, these important enabling technologies have been studied separately in most existing works. This article proposes a novel cooperative task offloading and block mining (TOBM) scheme for a blockchain-based MEC system where each edge device not only handles data tasks but also deals with block mining for improving the system utility. To address the latency issues caused by the blockchain operation in MEC, we develop a new Proof-of-Reputation consensus mechanism based on a lightweight block verification strategy. A multi-objective function is then formulated to maximize the system utility of the blockchain-based MEC system, by jointly optimizing offloading decision, channel selection, transmit power allocation, and computational resource allocation. We propose a novel distributed deep reinforcement learning-based approach by using a multi-agent deep deterministic policy gradient algorithm. We then develop a game-theoretic solution to model the offloading and mining competition among edge devices as a potential game, and prove the existence of a pure Nash equilibrium. Simulation results demonstrate the significant system utility improvements of our proposed scheme over baseline approaches.