Efficient Federated Split Learning for Large Language Models over Communication Networks

Fine-tuning pre-trained large language models (LLM) in a distributed manner poses significant challenges on resource-constrained edge devices. To address this challenge, we propose FedsLLM, a novel framework that integrates split federated learning with parameter-efficient fine-tuning techniques. By leveraging model splitting and Low-Rank Adaptation (LoRA), FedsLLM reduces the computational burden on edge devices. Furthermore, the introduction of a federated server facilitates parallel training and enhances privacy. To accommodate heterogeneous communication conditions and diverse computational capabilities of edge devices, as well as the impact of LoRA rank selection on model convergence and training cost, we formulate a joint optimization problem. The formulated problem jointly optimizes subchannel allocation, power control, model splitting point selection, and LoRA rank configuration, all aimed at minimizing total training delay. An alternating optimization algorithm is developed to efficiently solve this problem and accelerate the training process. Simulation results demonstrate that the proposed FedsLLM framework achieves comparable model accuracy while significantly reducing client-side computational requirements. Furthermore, the proposed resource allocation scheme and adaptive LoRA rank selection strategy notably reduce the training latency compared to conventional approaches.

Wireless Large AI Model: Shaping the AI-Native Future of 6G and Beyond

The emergence of sixth-generation and beyond communication systems is expected to fundamentally transform digital experiences through introducing unparalleled levels of intelligence, efficiency, and connectivity. A promising technology poised to enable this revolutionary vision is the wireless large AI model (WLAM), characterized by its exceptional capabilities in data processing, inference, and decision-making. In light of these remarkable capabilities, this paper provides a comprehensive survey of WLAM, elucidating its fundamental principles, diverse applications, critical challenges, and future research opportunities. We begin by introducing the background of WLAM and analyzing the key synergies with wireless networks, emphasizing the mutual benefits. Subsequently, we explore the foundational characteristics of WLAM, delving into their unique relevance in wireless environments. Then, the role of WLAM in optimizing wireless communication systems across various use cases and the reciprocal benefits are systematically investigated. Furthermore, we discuss the integration of WLAM with emerging technologies, highlighting their potential to enable transformative capabilities and breakthroughs in wireless communication. Finally, we thoroughly examine the high-level challenges hindering the practical implementation of WLAM and discuss pivotal future research directions.

Integrated Sensing, Computing, and Semantic Communication with Fluid Antenna for Metaverse

The integration of sensing and communication (ISAC) is pivotal for the Metaverse but faces challenges like high data volume and privacy concerns. This paper proposes a novel integrated sensing, computing, and semantic communication (ISCSC) framework, which uses semantic communication to transmit only contextual information, reducing data overhead and enhancing efficiency. To address the sensitivity of semantic communication to channel conditions, fluid antennas (FAs) are introduced, enabling dynamic adaptability. The FA-enabled ISCSC framework considers multiple users and extended targets composed of a series of scatterers, formulating a joint optimization problem to maximize the data rate while ensuring sensing accuracy and meeting computational and power constraints. An alternating optimization (AO) method decomposes the problem into subproblems for ISAC beamforming, FA positioning, and semantic extraction. Simulations confirm the framework's effectiveness in improving data rates and sensing performance.

Analogical Learning for Cross-Scenario Generalization: Framework and Application to Intelligent Localization

Existing learning models often exhibit poor generalization when deployed across diverse scenarios. It is mainly due to that the underlying reference frame of the data varies with the deployment environment and settings. However, despite the data of each scenario has its distinct reference frame, its generation generally follows the same underlying physical rule. Based on these findings, this article proposes a brand-new universal deep learning framework named analogical learning (AL), which provides a highly efficient way to implicitly retrieve the reference frame information associated with a scenario and then to make accurate prediction by relative analogy across scenarios. Specifically, an elegant bipartite neural network architecture called Mateformer is designed, the first part of which calculates the relativity within multiple feature spaces between the input data and a small amount of embedded data from the current scenario, while the second part uses these relativity to guide the nonlinear analogy. We apply AL to the typical multi-scenario learning problem of intelligent wireless localization in cellular networks. Extensive experiments show that AL achieves state-of-the-art accuracy, stable transferability and robust adaptation to new scenarios without any tuning, and outperforming conventional methods with a precision improvement of nearly two orders of magnitude. All data and code are available at https://github.com/ziruichen-research/ALLoc.

Fair Resource Allocation in UAV-based Semantic Communication System with Fluid Antenna

In this paper, the problem of maximization of the minimum equivalent rate in a unmanned-aerial-vehicle (UAV)-based multi-user semantic communication system is investigated. In the considered model, a multi-antenna UAV employs semantic extraction techniques to compress the data ready to be sent to the users, which are equipped with fluid antennas. Our aim is to jointly optimize the trajectory of the UAV, the transmit beamforming and the semantic compression rate at the UAV, as well as the selection of activated ports in fluid antenna system (FAS), to maximize the minimum equivalent transmission rate among all user. An alternating algorithm is designed to solve the problem. Simulation results validate the effectiveness of the proposed algorithm.

TeleMoM: Consensus-Driven Telecom Intelligence via Mixture of Models

Large language models (LLMs) face significant challenges in specialized domains like telecommunication (Telecom) due to technical complexity, specialized terminology, and rapidly evolving knowledge. Traditional methods, such as scaling model parameters or retraining on domain-specific corpora, are computationally expensive and yield diminishing returns, while existing approaches like retrieval-augmented generation, mixture of experts, and fine-tuning struggle with accuracy, efficiency, and coordination. To address this issue, we propose Telecom mixture of models (TeleMoM), a consensus-driven ensemble framework that integrates multiple LLMs for enhanced decision-making in Telecom. TeleMoM employs a two-stage process: proponent models generate justified responses, and an adjudicator finalizes decisions, supported by a quality-checking mechanism. This approach leverages strengths of diverse models to improve accuracy, reduce biases, and handle domain-specific complexities effectively. Evaluation results demonstrate that TeleMoM achieves a 9.7\% increase in answer accuracy, highlighting its effectiveness in Telecom applications.

Large-Scale AI in Telecom: Charting the Roadmap for Innovation, Scalability, and Enhanced Digital Experiences

This white paper discusses the role of large-scale AI in the telecommunications industry, with a specific focus on the potential of generative AI to revolutionize network functions and user experiences, especially in the context of 6G systems. It highlights the development and deployment of Large Telecom Models (LTMs), which are tailored AI models designed to address the complex challenges faced by modern telecom networks. The paper covers a wide range of topics, from the architecture and deployment strategies of LTMs to their applications in network management, resource allocation, and optimization. It also explores the regulatory, ethical, and standardization considerations for LTMs, offering insights into their future integration into telecom infrastructure. The goal is to provide a comprehensive roadmap for the adoption of LTMs to enhance scalability, performance, and user-centric innovation in telecom networks.

Multi-Cell Coordinated Beamforming for Integrate Communication and Multi-TMT Localization

This paper investigates integrated localization and communication in a multi-cell system and proposes a coordinated beamforming algorithm to enhance target localization accuracy while preserving communication performance. Within this integrated sensing and communication (ISAC) system, the Cramer-Rao lower bound (CRLB) is adopted to quantify the accuracy of target localization, with its closed-form expression derived for the first time. It is shown that the nuisance parameters can be disregarded without impacting the CRLB of time of arrival (TOA)-based target localization. Capitalizing on the derived CRLB, we formulate a nonconvex coordinated beamforming problem to minimize the CRLB while satisfying signal-to-interference-plus-noise ratio (SINR) constraints in communication. To facilitate the development of solution, we reformulate the original problem into a more tractable form and solve it through semi-definite programming (SDP). Notably, we show that the proposed algorithm can always obtain rank-one global optimal solutions under mild conditions. Finally, numerical results demonstrate the superiority of the proposed algorithm over benchmark algorithms and reveal the performance trade-off between localization accuracy and communication SINR.

Optimizing Wireless Resource Management and Synchronization in Digital Twin Networks

In this paper, we investigate an accurate synchronization between a physical network and its digital network twin (DNT), which serves as a virtual representation of the physical network. The considered network includes a set of base stations (BSs) that must allocate its limited spectrum resources to serve a set of users while also transmitting its partially observed physical network information to a cloud server to generate the DNT. Since the DNT can predict the physical network status based on its historical status, the BSs may not need to send their physical network information at each time slot, allowing them to conserve spectrum resources to serve the users. However, if the DNT does not receive the physical network information of the BSs over a large time period, the DNT's accuracy in representing the physical network may degrade. To this end, each BS must decide when to send the physical network information to the cloud server to update the DNT, while also determining the spectrum resource allocation policy for both DNT synchronization and serving the users. We formulate this resource allocation task as an optimization problem, aiming to maximize the total data rate of all users while minimizing the asynchronization between the physical network and the DNT. To address this problem, we propose a method based on the GRUs and the value decomposition network (VDN). Simulation results show that our GRU and VDN based algorithm improves the weighted sum of data rates and the similarity between the status of the DNT and the physical network by up to 28.96%, compared to a baseline method combining GRU with the independent Q learning.

Generative AI Empowered Semantic Feature Multiple Access (SFMA) Over Wireless Networks

This paper investigates a novel generative artificial intelligence (GAI) empowered multi-user semantic communication system called semantic feature multiple access (SFMA) for video transmission, which comprises a base station (BS) and paired users. The BS generates and combines semantic information of several frames simultaneously requested by paired users into a single signal. Users recover their frames from this combined signal and input the recovered frames into a GAI-based video frame interpolation model to generate the intermediate frame. To optimize transmission rates and temporal gaps between simultaneously transmitted frames, we formulate an optimization problem to maximize the system sum rate while minimizing temporal gaps. Since the standard signal-to-interference-plus-noise ratio (SINR) equation does not accurately capture the performance of our semantic communication system, we introduce a weight parameter into the SINR equation to better represent the system's performance. Due to its dependence on transmit power, we propose a three-step solution. First, we develop a user pairing algorithm that pairs two users with the highest preference value, a weighted combination of semantic transmission rate and temporal gap. Second, we optimize inter-group power allocation by formulating an optimization problem that allocates proper transmit power across all user groups to maximize system sum rates while satisfying each user's minimum rate requirement. Third, we address intra-group power allocation to enhance each user's performance. Simulation results demonstrate that our method improves transmission rates by up to 24.8%, 45.8%, and 66.1% compared to fixed-power non-orthogonal multiple access (F-NOMA), orthogonal joint source-channel coding (O-JSCC), and orthogonal frequency division multiple access (OFDMA), respectively.