Achievable Rate Analysis of the STAR-RIS Aided NOMA Uplink in the Face of Imperfect CSI and Hardware Impairments

Reconfigurable intelligent surfaces (RIS) are capable of beneficially ameliorating the propagation environment by appropriately controlling the passive reflecting elements. To extend the coverage area, the concept of simultaneous transmitting and reflecting reconfigurable intelligent surfaces (STAR-RIS) has been proposed, yielding supporting 360^circ coverage user equipment (UE) located on both sides of the RIS. In this paper, we theoretically formulate the ergodic sum-rate of the STAR-RIS assisted non-orthogonal multiple access (NOMA) uplink in the face of channel estimation errors and hardware impairments (HWI). Specifically, the STAR-RIS phase shift is configured based on the statistical channel state information (CSI), followed by linear minimum mean square error (LMMSE) channel estimation of the equivalent channel spanning from the UEs to the access point (AP). Afterwards, successive interference cancellation (SIC) is employed at the AP using the estimated instantaneous CSI, and we derive the theoretical ergodic sum-rate upper bound for both perfect and imperfect SIC decoding algorithm. The theoretical analysis and the simulation results show that both the channel estimation and the ergodic sum-rate have performance floor at high transmit power region caused by transceiver hardware impairments.

Reconfigurable Intelligent Surface Aided Amplitude- and Phase-Modulated Downlink Transmission

New reconfigurable intelligent surface (RIS) based amplitude and phase modulation schemes are proposed as an evolution how the phase-only modulation schemes available in the literature. Explicitly, both the amplitude-phase shift keying (A-PSK) and quadrature amplitude-phase shift keying (QA-PSK) are conceived, where the RIS is assumed to be part of a transmitter to deliver information to the multi-antenna aided downlink receiver. In the proposed design, the RIS is partitioned into multiple blocks, and the information bits are conveyed by controlling both the ON-OFF state and the phase shift of the RIS elements in each block. Since the propagation paths spanning from each RIS block to the receiver can be coherently combined as a benefit of appropriately configuring the phase of the RIS elements, the received signal constellations can be designed by controlling both the ON-OFF pattern of the RIS blocks as well as the phase shift of the RIS elements. Both the theoretical analysis and the simulation results show that our proposed RIS-aided modulation schemes outperform the state-of-the-art RIS-based PSK modulation both in terms of its discrete-input-continuous-output memoryless channel (DCMC) capacity and its symbol error probability, especially in the high signal-to-noise-ratio (SNR) region, when considering realistic finite resolution RIS phase shifts.

Performance Analysis of Active RIS-aided Systems in the Face of Imperfect CSI and Phase Shift Noise

The linear minimal mean square error (LMMSE) estimator for active reconfigurable intelligent surface (RIS)-aided wireless systems is formulated. Furthermore, based on the moment-matching method, we employ the Gamma distribution to approximate the distribution of the instantaneous received signal-to-interference-plus-noise ratio (SINR), and then derive the closed-form outage probability and ergodic channel capacity in the presence of realistic channel estimation errors, the thermal noise of RIS amplifiers and the RIS phase shift noise. Our theoretical analysis and simulation results show that the introduction of RIS amplifiers is equivalent to increasing of the transmit power, and also present the performance degradation resulting from the channel estimation error and the RIS phase noise.

Statistical CSI-based Beamforming for RIS-Aided Multiuser MISO Systems using Deep Reinforcement Learning

The paper presents a joint beamforming algorithm using statistical channel state information (S-CSI) for reconfigurable intelligent surfaces (RIS) for multiuser MISO wireless communications. We used S-CSI, which is a long-term average of the cascaded channel as opposed to instantaneous CSI utilized in most existing works. Through this method, the overhead of channel estimation is dramatically reduced. We propose a proximal policy optimization (PPO) algorithm which is a well-known actor-critic based reinforcement learning (RL) algorithm to solve the optimization problem. To test the efficacy of this algorithm, simulation results are presented along with evaluations of key system parameters, including the Rician factor and RIS location, on the achievable sum rate of the users.

Sim2real for Reinforcement Learning Driven Next Generation Networks

The next generation of networks will actively embrace artificial intelligence (AI) and machine learning (ML) technologies for automation networks and optimal network operation strategies. The emerging network structure represented by Open RAN (O-RAN) conforms to this trend, and the radio intelligent controller (RIC) at the centre of its specification serves as an ML applications host. Various ML models, especially Reinforcement Learning (RL) models, are regarded as the key to solving RAN-related multi-objective optimization problems. However, it should be recognized that most of the current RL successes are confined to abstract and simplified simulation environments, which may not directly translate to high performance in complex real environments. One of the main reasons is the modelling gap between the simulation and the real environment, which could make the RL agent trained by simulation ill-equipped for the real environment. This issue is termed as the sim2real gap. This article brings to the fore the sim2real challenge within the context of O-RAN. Specifically, it emphasizes the characteristics, and benefits that the digital twins (DT) could have as a place for model development and verification. Several use cases are presented to exemplify and demonstrate failure modes of the simulations trained RL model in real environments. The effectiveness of DT in assisting the development of RL algorithms is discussed. Then the current state of the art learning-based methods commonly used to overcome the sim2real challenge are presented. Finally, the development and deployment concerns for the RL applications realisation in O-RAN are discussed from the view of the potential issues like data interaction, environment bottlenecks, and algorithm design.

Smart Interference Management xApp using Deep Reinforcement Learning

Interference continues to be a key limiting factor in cellular radio access network (RAN) deployments. Effective, data-driven, self-adapting radio resource management (RRM) solutions are essential for tackling interference, and thus achieving the desired performance levels particularly at the cell-edge. In future network architecture, RAN intelligent controller (RIC) running with near-real-time applications, called xApps, is considered as a potential component to enable RRM. In this paper, based on deep reinforcement learning (RL) xApp, a joint sub-band masking and power management is proposed for smart interference management. The sub-band resource masking problem is formulated as a Markov Decision Process (MDP) that can be solved employing deep RL to approximate the policy functions as well as to avoid extremely high computational and storage costs of conventional tabular-based approaches. The developed xApp is scalable in both storage and computation. Simulation results demonstrate advantages of the proposed approach over decentralized baselines in terms of the trade-off between cell-centre and cell-edge user rates, energy efficiency and computational efficiency.

Reconfigurable Intelligent Surfaces Relying on Non-Diagonal Phase Shift Matrices

Reconfigurable intelligent surfaces (RIS) have been actively researched as a potential technique for future wireless communications, which intelligently ameliorate the signal propagation environment. In the conventional design, each RIS element configures and reflects its received signal independently of all other RIS elements, which results in a diagonal phase shift matrix. By contrast, we propose a novel RIS architecture, where the incident signal impinging on one element can be reflected from another element after an appropriate phase shift adjustment, which increases the flexibility in the design of RIS phase shifts, hence, potentially improving the system performance. The resultant RIS phase shift matrix also has off-diagonal elements, as opposed to the pure diagonal structure of the conventional design. Compared to the state-of-art fully-connected/group-connected RIS structures, our proposed RIS architecture has lower complexity, while attaining a higher channel gain than the group-connected RIS structure, and approaching that of the fully-connected RIS structure. We formulate and solve the problem of maximizing the achievable rate of our proposed RIS architecture by jointly optimizing the transmit beamforming and the non-diagonal phase shift matrix based on alternating optimization and semi-define relaxation (SDR) methods. Moreover, the closed-form expressions of the channel gain, the outage probability and bit error ratio (BER) are derived. Simulation results demonstrate that our proposed RIS architecture results in an improved performance in terms of the achievable rate compared to the conventional architecture, both in single-user as well as in multi-user scenarios.

RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN

Radio access network (RAN) technologies continue to witness massive growth, with Open RAN gaining the most recent momentum. In the O-RAN specifications, the RAN intelligent controller (RIC) serves as an automation host. This article introduces principles for machine learning (ML), in particular, reinforcement learning (RL) relevant for the O-RAN stack. Furthermore, we review state-of-the-art research in wireless networks and cast it onto the RAN framework and the hierarchy of the O-RAN architecture. We provide a taxonomy of the challenges faced by ML/RL models throughout the development life-cycle: from the system specification to production deployment (data acquisition, model design, testing and management, etc.). To address the challenges, we integrate a set of existing MLOps principles with unique characteristics when RL agents are considered. This paper discusses a systematic life-cycle model development, testing and validation pipeline, termed: RLOps. We discuss all fundamental parts of RLOps, which include: model specification, development and distillation, production environment serving, operations monitoring, safety/security and data engineering platform. Based on these principles, we propose the best practices for RLOps to achieve an automated and reproducible model development process.