CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild

Deriving co-speech 3D gestures has seen tremendous progress in virtual avatar animation. Yet, the existing methods often produce stiff and unreasonable gestures with unseen human speech inputs due to the limited 3D speech-gesture data. In this paper, we propose CoCoGesture, a novel framework enabling vivid and diverse gesture synthesis from unseen human speech prompts. Our key insight is built upon the custom-designed pretrain-fintune training paradigm. At the pretraining stage, we aim to formulate a large generalizable gesture diffusion model by learning the abundant postures manifold. Therefore, to alleviate the scarcity of 3D data, we first construct a large-scale co-speech 3D gesture dataset containing more than 40M meshed posture instances across 4.3K speakers, dubbed GES-X. Then, we scale up the large unconditional diffusion model to 1B parameters and pre-train it to be our gesture experts. At the finetune stage, we present the audio ControlNet that incorporates the human voice as condition prompts to guide the gesture generation. Here, we construct the audio ControlNet through a trainable copy of our pre-trained diffusion model. Moreover, we design a novel Mixture-of-Gesture-Experts (MoGE) block to adaptively fuse the audio embedding from the human speech and the gesture features from the pre-trained gesture experts with a routing mechanism. Such an effective manner ensures audio embedding is temporal coordinated with motion features while preserving the vivid and diverse gesture generation. Extensive experiments demonstrate that our proposed CoCoGesture outperforms the state-of-the-art methods on the zero-shot speech-to-gesture generation. The dataset will be publicly available at: https://mattie-e.github.io/GES-X/

Deep Cooperation in ISAC System: Resource, Node and Infrastructure Perspectives

With the mobile communication system evolving into 6th-generation (6G), the Internet of Everything (IoE) is becoming reality, which connects human, big data and intelligent machines to support the intelligent decision making, reconfiguring the traditional industries and human life. The applications of IoE require not only pure communication capability, but also high-accuracy and large-scale sensing capability. With the emerging integrated sensing and communication (ISAC) technique, exploiting the mobile communication system with multi-domain resources, multiple network elements, and large-scale infrastructures to realize cooperative sensing is a crucial approach to satisfy the requirements of high-accuracy and large-scale sensing in IoE. In this article, the deep cooperation in ISAC system including three perspectives is investigated. In the microscopic perspective, namely, within a single node, the cooperation at the resource-level is performed to improve sensing accuracy by fusing the sensing information carried in the time-frequency-space-code multi-domain resources. In the mesoscopic perspective, the sensing accuracy could be improved through the cooperation of multiple nodes including Base Station (BS), User Equipment (UE), and Reconfigurable Intelligence Surface (RIS), etc. In the macroscopic perspective, the massive number of infrastructures from the same operator or different operators could perform cooperative sensing to extend the sensing coverage and improve the sensing continuity. This article may provide a deep and comprehensive view on the cooperative sensing in ISAC system to enhance the performance of sensing, supporting the applications of IoE.

Integrated Sensing and Communication Driven Digital Twin for Intelligent Machine Network

Intelligent machines (IMs), including industrial machines, unmanned aerial vehicles (UAVs), and unmanned vehicles, etc., could perform effective cooperation in complex environment when they form IM network. The efficient environment sensing and communication are crucial for IM network, enabling the real-time and stable control of IMs. With the emergence of integrated sensing and communication (ISAC) technology, IM network is empowered with ubiquitous sensing capabilities, which is helpful in improving the efficiency of communication and sensing with the mutual benefit of them. However, the massive amount of sensing information brings challenges for the processing, storage and application of sensing information. In this article, ISAC driven digital twin (DT) is proposed for IM network, and the architecture and enabling technologies are revealed. ISAC driven DT structurally stores the sensing information, which is further applied to optimize communication, networking and control schemes of IMs, promoting the widespread applications of IMs.

Communication-Assisted Sensing in 6G Networks

The exploration of coordination gain achieved through the synergy of sensing and communication (S&C) functions plays a vital role in improving the performance of integrated sensing and communication systems. This paper focuses on the optimal waveform design for communication-assisted sensing (CAS) systems within the context of 6G perceptive networks. In the CAS process, the base station actively senses the targets through device-free wireless sensing and simultaneously transmits the pertinent information to end-users. In our research, we establish a CAS framework grounded in the principles of rate-distortion theory and the source-channel separation theorem (SCT) in lossy data transmission. This framework provides a comprehensive understanding of the interplay between distortion, coding rate, and channel capacity. The purpose of waveform design is to minimize the sensing distortion at the user end while adhering to the SCT and power budget constraints. In the context of target response matrix estimation, we propose two distinct waveform strategies: the separated S&C and dual-functional waveform schemes. In the former strategy, we develop a simple one-dimensional search algorithm, shedding light on a notable power allocation tradeoff between the S&C waveform. In the latter scheme, we conceive a heuristic mutual information optimization algorithm for the general case, alongside a modified gradient projection algorithm tailored for the scenarios with independent sensing sub-channels. Additionally, we identify the presence of both subspace tradeoff and water-filling tradeoff. Finally, we validate the effectiveness of the proposed algorithms through numerical simulations.

Joint Localization and Communication Enhancement in Uplink Integrated Sensing and Communications System with Clock Asynchronism

In this paper, we propose a joint single-base localization and communication enhancement scheme for the uplink (UL) integrated sensing and communications (ISAC) system with asynchronism, which can achieve accurate single-base localization of user equipment (UE) and significantly improve the communication reliability despite the existence of timing offset (TO) due to the clock asynchronism between UE and base station (BS). Our proposed scheme integrates the CSI enhancement into the multiple signal classification (MUSIC)-based AoA estimation and thus imposes no extra complexity on the ISAC system. We further exploit a MUSIC-based range estimation method and prove that it can suppress the time-varying TO-related phase terms. Exploiting the AoA and range estimation of UE, we can estimate the location of UE. Finally, we propose a joint CSI and data signals-based localization scheme that can coherently exploit the data and the CSI signals to improve the AoA and range estimation, which further enhances the single-base localization of UE. The extensive simulation results show that the enhanced CSI can achieve equivalent bit error rate performance to the minimum mean square error (MMSE) CSI estimator. The proposed joint CSI and data signals-based localization scheme can achieve decimeter-level localization accuracy despite the existing clock asynchronism and improve the localization mean square error (MSE) by about 8 dB compared with the maximum likelihood (ML)-based benchmark method.

Sensing-assisted Accurate and Fast Beam Management for Cellular-connected mmWave UAV Network

Beam management, including initial access (IA) and beam tracking, is essential to the millimeter-wave Unmanned Aerial Vehicle (UAV) network. However, conventional communication-only and feedback-based schemes suffer a high delay and low accuracy of beam alignment since they only enable the receiver to passively hear the information of the transmitter from the radio domain. This paper presents a novel sensing-assisted beam management approach, the first solution that fully utilizes the information from the visual domain to improve communication performance. We employ both integrated sensing and communication and computer vision techniques and design an extended Kalman filtering method for beam tracking and prediction. Besides, we also propose a novel dual identity association solution to distinguish multiple UAVs in dynamic environments. Real-world experiments and numerical results show that the proposed solution outperforms the conventional methods in IA delay, association accuracy, tracking error, and communication performance.

Toward Intelligent and Efficient 6G Networks: JCSC Enabled On-Purpose Machine Communications

Driven by the vision of "intelligent connection of everything" toward 6G, the collective intelligence of networked machines can be fully exploited to improve system efficiency by shifting the paradigm of wireless communication design from naive maximalist approaches to intelligent value-based approaches. In this article, we propose an on-purpose machine communication framework enabled by joint communication, sensing, and computation (JCSC) technology, which employs machine semantics as the interactive information flow. Naturally, there are potential technical barriers to be solved before the widespread adoption of on-purpose communications, including the conception of machine purpose, fast and concise networking strategy, and semantics-aware information exchange mechanism during the process of task-oriented cooperation. Hence, we discuss enabling technologies complemented by a range of open challenges. The simulation result shows that the proposed framework can significantly reduce networking overhead and improve communication efficiency.

Toward Trusted and Swift UAV Communication: ISAC-Enabled Dual Identity Mapping

The UAV network has recently emerged as a capable carrier for ubiquitous wireless intelligent communication in the B5G/6G era. Nevertheless, the separation of dual identity raises challenges from the perspective of communication efficiency and security, including tedious communication feedback and malicious Sybil attacks. Meanwhile, thanks to the emerging integrated sensing and communication (ISAC) technology, the sensing ability incorporated in communication advances crucial opportunities for accurately and efficiently mapping identity from dual domains. This tutorial discusses the exciting intersection of ISAC and the future intelligent and efficient UAV network. We first describe the motivation scenario and present the framework of the proposed novel ISAC-enabled dual identity solution. The detailed modules of identity production, mapping, management, and authentication are discussed. By endowing UAVs with an advanced capability: opening their eyes when communicating with each other, we detail three typical applications and the advantages of our proposal. Finally, a series of key enabling techniques, open challenges, and potential solutions for ISAC-enabled dual-domain identity are discussed. This tutorial for the intelligent and efficient UAV network brings new insight on providing dual-domain identity via ISAC technology, with an eye on trusted and swift communication research tailored for the 6G UAV network.

Topology-Aware Resilient Routing Protocol for FANETs: An Adaptive Q-Learning Approach

Flying ad hoc networks (FANETs) play a crucial role in numerous military and civil applications since it shortens mission duration and enhances coverage significantly compared with a single unmanned aerial vehicle (UAV). Whereas, designing an energy-efficient FANET routing protocol with a high packet delivery rate (PDR) and low delay is challenging owing to the dynamic topology changes. In this article, we propose a topology-aware resilient routing strategy based on adaptive Q-learning (TARRAQ) to accurately capture topology changes with low overhead and make routing decisions in a distributed and autonomous way. First, we analyze the dynamic behavior of UAV nodes via the queuing theory, and then the closed-form solutions of neighbors' change rate (NCR) and neighbors' change interarrival time (NCIT) distribution are derived. Based on the real-time NCR and NCIT, a resilient sensing interval (SI) is determined by defining the expected sensing delay of network events. Besides, we also present an adaptive Q-learning approach that enables UAVs to make distributed, autonomous, and adaptive routing decisions, where the above SI ensures that the action space can be updated in time at a low cost. The simulation results verify the accuracy of the topology dynamic analysis model and also prove that our TARRAQ outperforms the Q-learning-based topology-aware routing (QTAR), mobility prediction-based virtual routing (MPVR), and greedy perimeter stateless routing based on energy-efficient hello (EE-Hello) in terms of 25.23%, 20.24%, and 13.73% lower overhead, 9.41%, 14.77%, and 16.70% higher PDR, and 5.12%, 15.65%, and 11.31% lower energy consumption, respectively.

Seeing is Believing: Detecting Sybil Attack in FANET by Matching Visual and Auditory Domains

The flying ad hoc network (FANET) will play a crucial role in the B5G/6G era since it provides wide coverage and on-demand deployment services in a distributed manner. The detection of Sybil attacks is essential to ensure trusted communication in FANET. Nevertheless, the conventional methods only utilize the untrusted information that UAV nodes passively ``heard'' from the ``auditory" domain (AD), resulting in severe communication disruptions and even collision accidents. In this paper, we present a novel VA-matching solution that matches the neighbors observed from both the AD and the ``visual'' domain (VD), which is the first solution that enables UAVs to accurately correlate what they ``see'' from VD and ``hear'' from AD to detect the Sybil attacks. Relative entropy is utilized to describe the similarity of observed characteristics from dual domains. The dynamic weight algorithm is proposed to distinguish neighbors according to the characteristics' popularity. The matching model of neighbors observed from AD and VD is established and solved by the vampire bat optimizer. Experiment results show that the proposed VA-matching solution removes the unreliability of individual characteristics and single domains. It significantly outperforms the conventional RSSI-based method in detecting Sybil attacks. Furthermore, it has strong robustness and achieves high precision and recall rates.