A Communication-Efficient Decentralized Actor-Critic Algorithm

In this paper, we study the problem of reinforcement learning in multi-agent systems where communication among agents is limited. We develop a decentralized actor-critic learning framework in which each agent performs several local updates of its policy and value function, where the latter is approximated by a multi-layer neural network, before exchanging information with its neighbors. This local training strategy substantially reduces the communication burden while maintaining coordination across the network. We establish finite-time convergence analysis for the algorithm under Markov-sampling. Specifically, to attain the $\varepsilon$-accurate stationary point, the sample complexity is of order $\mathcal{O}(\varepsilon^{-3})$ and the communication complexity is of order $\mathcal{O}(\varepsilon^{-1}\tau^{-1})$, where tau denotes the number of local training steps. We also show how the final error bound depends on the neural network's approximation quality. Numerical experiments in a cooperative control setting illustrate and validate the theoretical findings.

Fault Localisation in Infinite-Dimensional Linear Electrical Networks

We present a novel fault localisation methodology for linear time-invariant electrical networks with infinite-dimensional edge dynamics and uncertain fault dynamics. The theory accommodates instability and also bounded propagation delays in the network. The goal is to estimate the location of a fault along a given network edge, using sensors positioned arbitrarily throughout the network. Passive faults of unknown impedance are considered, along with stable faults of known impedance. To illustrate the approach, we tackle a significant use-case: a multi-conductor transmission line, with dynamics modelled by the Telegrapher's equation, subject to a line-to-ground fault. Frequency-domain insights are used to reformulate the general fault localisation problem into a non-convex scalar optimisation problem, of which the true fault location is guaranteed to be a global minimiser. Numerical experiments are run to quantify localisation performance over a range of fault resistances.