UniPreCIS : A data pre-processing solution for collocated services on shared IoT

Next-generation smart city applications, attributed by the power of Internet of Things (IoT) and Cyber-Physical Systems (CPS), significantly rely on the quality of sensing data. With an exponential increase in intelligent applications for urban development and enterprises offering sensing-as-aservice these days, it is imperative to provision for a shared sensing infrastructure for better utilization of resources. However, a shared sensing infrastructure that leverages low-cost sensing devices for a cost effective solution, still remains an unexplored territory. A significant research effort is still needed to make edge based data shaping solutions, more reliable, feature-rich and costeffective while addressing the associated challenges in sharing the sensing infrastructure among multiple collocated services with diverse Quality of Service (QoS) requirements. Towards this, we propose a novel edge based data pre-processing solution, named UniPreCIS that accounts for the inherent characteristics of lowcost ambient sensors and the exhibited measurement dynamics with respect to application-specific QoS. UniPreCIS aims to identify and select quality data sources by performing sensor ranking and selection followed by multimodal data pre-processing in order to meet heterogeneous application QoS and at the same time reducing the resource consumption footprint for the resource constrained network edge. As observed, the processing time and memory utilization has been reduced in the proposed approach while achieving upto 90% accuracy which is arguably significant as compared to state-of-the-art techniques for sensing. The effectiveness of UniPreCIS has been evaluated on a testbed for a specific use case of indoor occupancy estimation that proves its effectiveness.

Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data

We propose Compressed Vertical Federated Learning (C-VFL) for communication-efficient training on vertically partitioned data. In C-VFL, a server and multiple parties collaboratively train a model on their respective features utilizing several local iterations and sharing compressed intermediate results periodically. Our work provides the first theoretical analysis of the effect message compression has on distributed training over vertically partitioned data. We prove convergence of non-convex objectives at a rate of $O(\frac{1}{\sqrt{T}})$ when the compression error is bounded over the course of training. We provide specific requirements for convergence with common compression techniques, such as quantization and top-$k$ sparsification. Finally, we experimentally show compression can reduce communication by over $90\%$ without a significant decrease in accuracy over VFL without compression.

Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning

We consider federated learning in tiered communication networks. Our network model consists of a set of silos, each holding a vertical partition of the data. Each silo contains a hub and a set of clients, with the silo's vertical data shard partitioned horizontally across its clients. We propose Tiered Decentralized Coordinate Descent (TDCD), a communication-efficient decentralized training algorithm for such two-tiered networks. To reduce communication overhead, the clients in each silo perform multiple local gradient steps before sharing updates with their hub. Each hub adjusts its coordinates by averaging its workers' updates, and then hubs exchange intermediate updates with one another. We present a theoretical analysis of our algorithm and show the dependence of the convergence rate on the number of vertical partitions, the number of local updates, and the number of clients in each hub. We further validate our approach empirically via simulation-based experiments using a variety of datasets and objectives.

Multi-Tier Federated Learning for Vertically Partitioned Data

We consider decentralized model training in tiered communication networks. Our network model consists of a set of silos, each holding a vertical partition of the data. Each silo contains a hub and a set of clients, with the silo's vertical data shard partitioned horizontally across its clients. We propose Tiered Decentralized Coordinate Descent (TDCD), a communication-efficient decentralized training algorithm for such two-tiered networks. To reduce communication overhead, the clients in each silo perform multiple local gradient steps before sharing updates with their hub. Each hub adjusts its coordinates by averaging its workers' updates, and then hubs exchange intermediate updates with one another. We present a theoretical analysis of our algorithm and show the dependence of the convergence rate on the number of vertical partitions, the number of local updates, and the number of clients in each hub. We further validate our approach empirically via simulation-based experiments using a variety of datasets and both convex and non-convex objectives.

Multi-Level Local SGD for Heterogeneous Hierarchical Networks

We propose Multi-Level Local SGD, a distributed gradient method for learning a smooth, non-convex objective in a heterogeneous multi-level network. Our network model consists of a set of disjoint sub-networks, with a single hub and multiple worker nodes; further, worker nodes may have different operating rates. The hubs exchange information with one another via a connected, but not necessarily complete communication network. In our algorithm, sub-networks execute a distributed SGD algorithm, using a hub-and-spoke paradigm, and the hubs periodically average their models with neighboring hubs. We first provide a unified mathematical framework that describes the Multi-Level Local SGD algorithm. We then present a theoretical analysis of the algorithm; our analysis shows the dependence of the convergence error on the worker node heterogeneity, hub network topology, and the number of local, sub-network, and global iterations. We back up our theoretical results via simulation-based experiments using both convex and non-convex objectives.

Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devices

Federated Learning enables training of a general model through edge devices without sending raw data to the cloud. Hence, this approach is attractive for digital health applications, where data is sourced through edge devices and users care about privacy. Here, we report on the feasibility to train deep neural networks on the Raspberry Pi4s as edge devices. A CNN, a LSTM and a MLP were successfully trained on the MNIST data-set. Further, federated learning is demonstrated experimentally on IID and non-IID samples in a parametric study, to benchmark the model convergence. The weight updates from the workers are shared with the cloud to train the general model through federated learning. With the CNN and the non-IID samples a test-accuracy of up to 85% could be achieved within a training time of 2 minutes, while exchanging less than $10$ MB data per device. In addition, we discuss federated learning from an use-case standpoint, elaborating on privacy risks and labeling requirements for the application of emotion detection from sound. Based on the experimental findings, we discuss possible research directions to improve model and system performance. Finally, we provide best practices for a practitioner, considering the implementation of federated learning.