A Scale-Independent Multi-Objective Reinforcement Learning with Convergence Analysis

Many sequential decision-making problems need optimization of different objectives which possibly conflict with each other. The conventional way to deal with a multi-task problem is to establish a scalar objective function based on a linear combination of different objectives. However, for the case of having conflicting objectives with different scales, this method needs a trial-and-error approach to properly find proper weights for the combination. As such, in most cases, this approach cannot guarantee an optimal Pareto solution. In this paper, we develop a single-agent scale-independent multi-objective reinforcement learning on the basis of the Advantage Actor-Critic (A2C) algorithm. A convergence analysis is then done for the devised multi-objective algorithm providing a convergence-in-mean guarantee. We then perform some experiments over a multi-task problem to evaluate the performance of the proposed algorithm. Simulation results show the superiority of developed multi-objective A2C approach against the single-objective algorithm.

Time-Varying Optimization with Optimal Parametric Function

In this paper, we consider a class of nonlinear constrained optimization problems. We formulate this problem as a time-varying optimization using continuous-time parametric functions and derive a dynamical system for tracking the optimal solution. We then re-parameterize the dynamical system to express it as a linear combination of the parametric functions. Calculus of variations is applied to optimize the parametric functions, such that the optimality distance of the solution is minimized. Accordingly, an iterative dynamic algorithm is devised to find the solution with an efficient convergence rate. We benchmark the performance of the proposed algorithm with the prediction-correction method from the optimality and computational complexity point-of-views.

Optimal Bandwidth Allocation for Multicast-Cache-Aided on-Demand Streaming in Wireless Networks

We consider a hybrid delivery scheme for streaming content, combining cache-enabled Orthogonal Multipoint Multicast (OMPMC) and on-demand Single-Point Unicast (SPUC) transmissions for heterogeneous networks. The OMPMC service transmits cached files through the whole network to interested users, and users not being satisfied by this service are assigned to the SPUC service to be individually served. The SPUC fetches the requested files from the core network and unicasts them to UEs using cellular beamforming transmissions. We optimize the delivery scheme to minimize the average resource consumption in the network. We formulate a constrained optimization problem over the cache placement and resource allocation of the OMPMC component, as well as the multi-user beamforming scheme of the SPUC component. We apply a path-following method to find the optimal traffic offloading solution. The solutions portray a contrast between the total amount of consumed resources and service outage probability. Simulation results show that the hybrid scheme provides a better tradeoff between the amount of network-wide consumed resources and the service outage probability, as compared to schemes from the literature.