Top-k Multi-Armed Bandit Learning for Content Dissemination in Swarms of Micro-UAVs

In communication-deprived disaster scenarios, this paper introduces a Micro-Unmanned Aerial Vehicle (UAV)- enhanced content management system. In the absence of cellular infrastructure, this system deploys a hybrid network of stationary and mobile UAVs to offer vital content access to isolated communities. Static anchor UAVs equipped with both vertical and lateral links cater to local users, while agile micro-ferrying UAVs, equipped with lateral links and greater mobility, reach users in various communities. The primary goal is to devise an adaptive content dissemination system that dynamically learns caching policies to maximize content accessibility. The paper proposes a decentralized Top-k Multi-Armed Bandit (Top-k MAB) learning approach for UAV caching decisions, accommodating geotemporal disparities in content popularity and diverse content demands. The proposed mechanism involves a Selective Caching Algorithm that algorithmically reduces redundant copies of the contents by leveraging the shared information between the UAVs. It is demonstrated that Top-k MAB learning, along with selective caching algorithm, can improve system performance while making the learning process adaptive. The paper does functional verification and performance evaluation of the proposed caching framework under a wide range of network size, swarm of micro-ferrying UAVs, and heterogeneous popularity distributions.

Unsupervised Speaker Diarization in Distributed IoT Networks Using Federated Learning

This paper presents a computationally efficient and distributed speaker diarization framework for networked IoT-style audio devices. The work proposes a Federated Learning model which can identify the participants in a conversation without the requirement of a large audio database for training. An unsupervised online update mechanism is proposed for the Federated Learning model which depends on cosine similarity of speaker embeddings. Moreover, the proposed diarization system solves the problem of speaker change detection via. unsupervised segmentation techniques using Hotelling's t-squared Statistic and Bayesian Information Criterion. In this new approach, speaker change detection is biased around detected quasi-silences, which reduces the severity of the trade-off between the missed detection and false detection rates. Additionally, the computational overhead due to frame-by-frame identification of speakers is reduced via. unsupervised clustering of speech segments. The results demonstrate the effectiveness of the proposed training method in the presence of non-IID speech data. It also shows a considerable improvement in the reduction of false and missed detection at the segmentation stage, while reducing the computational overhead. Improved accuracy and reduced computational cost makes the mechanism suitable for real-time speaker diarization across a distributed IoT audio network.

Multi-Armed Bandit Learning for Content Provisioning in Network of UAVs

This paper proposes an unmanned aerial vehicle (UAV) aided content management system in communication-challenged disaster scenarios. Without cellular infrastructure in such scenarios, community of stranded users can be provided access to situation-critical contents using a hybrid network of static and traveling UAVs. A set of relatively static anchor UAVs can download content from central servers and provide content access to its local users. A set of ferrying UAVs with wider mobility can provision content to users by shuffling them across different anchor UAVs while visiting different communities of users. The objective is to design a content dissemination system that on-the-fly learns content caching policies for maximizing content availability to the stranded users. This paper proposes a decentralized Top-k Multi-Armed Bandit Learning model for UAV-caching decision-making that takes geo-temporal differences in content popularity and heterogeneity in content demands into consideration. The proposed paradigm is able to combine the expected reward maximization attribute and a proposed multi-dimensional reward structure of Top-k Multi-Armed Bandit, for caching decision at the UAVs. This study is done for different user-specified tolerable access delay, heterogeneous popularity distributions, and inter-community geographical characteristics. Functional verification and performance evaluation of the proposed caching framework is done for a wide range of network size, UAV distribution, and content popularity.

Medium Access using Distributed Reinforcement Learning for IoTs with Low-Complexity Wireless Transceivers

This paper proposes a distributed Reinforcement Learning (RL) based framework that can be used for synthesizing MAC layer wireless protocols in IoT networks with low-complexity wireless transceivers. The proposed framework does not rely on complex hardware capabilities such as carrier sensing and its associated algorithmic complexities that are often not supported in wireless transceivers of low-cost and low-energy IoT devices. In this framework, the access protocols are first formulated as Markov Decision Processes (MDP) and then solved using RL. A distributed and multi-Agent RL framework is used as the basis for protocol synthesis. Distributed behavior makes the nodes independently learn optimal transmission strategies without having to rely on full network level information and direct knowledge of behavior of other nodes. The nodes learn to minimize packet collisions such that optimal throughput can be attained and maintained for loading conditions that are higher than what the known benchmark protocols (such as ALOHA) for IoT devices without complex transceivers. In addition, the nodes are observed to be able to learn to act optimally in the presence of heterogeneous loading and network topological conditions. Finally, the proposed learning approach allows the wireless bandwidth to be fairly distributed among network nodes in a way that is not dependent on such heterogeneities. Via simulation experiments, the paper demonstrates the performance of the learning paradigm and its abilities to make nodes adapt their optimal transmission strategies on the fly in response to various network dynamics.

Towards Multi-agent Reinforcement Learning for Wireless Network Protocol Synthesis

This paper proposes a multi-agent reinforcement learning based medium access framework for wireless networks. The access problem is formulated as a Markov Decision Process (MDP), and solved using reinforcement learning with every network node acting as a distributed learning agent. The solution components are developed step by step, starting from a single-node access scenario in which a node agent incrementally learns to control MAC layer packet loads for reining in self-collisions. The strategy is then scaled up for multi-node fully-connected scenarios by using more elaborate reward structures. It also demonstrates preliminary feasibility for more general partially connected topologies. It is shown that by learning to adjust MAC layer transmission probabilities, the protocol is not only able to attain theoretical maximum throughput at an optimal load, but unlike classical approaches, it can also retain that maximum throughput at higher loading conditions. Additionally, the mechanism is agnostic to heterogeneous loading while preserving that feature. It is also shown that access priorities of the protocol across nodes can be parametrically adjusted. Finally, it is also shown that the online learning feature of reinforcement learning is able to make the protocol adapt to time-varying loading conditions.