Sherman
Abstract:Vision-language-action (VLA) models inherit rich visual-semantic priors from pre-trained vision-language backbones, but adapting them to robotic control remains challenging. Full fine-tuning (FFT) is prone to overfitting on downstream robotic data and catastrophic forgetting of pretrained vision-language capabilities. Parameter-efficient fine-tuning (PEFT) better preserves pre-trained knowledge, yet existing PEFT methods still struggle to adapt effectively to robot control tasks. To address this gap, we propose VLA-GSE, a parameter-efficient VLA fine-tuning framework that improves control adaptation while retaining PEFT's knowledge preservation advantage. Specifically, VLA-GSE (Generalized and Specialized Experts) is initialized by spectrally decomposing the frozen backbone, assigning leading singular components to generalized experts (shared experts) and disjoint residual components to specialized experts (routed experts). This decomposition improves adaptation capacity under a fixed trainable-parameter budget. Under a comparable parameter budget, VLA-GSE updates only 2.51% of the full model parameters and consistently outperforms strong FFT and PEFT baselines. It achieves 81.2% average zero-shot success on LIBERO-Plus, preserves pre-trained VLM capability comparably to LoRA on multimodal understanding benchmarks, and improves real-world manipulation success under multiple distribution shifts. Code is available at: https://github.com/YuhuaJiang2002/VLA-GSE
Abstract:World action models (WAMs) provide a powerful generative framework for embodied control, yet transferring knowledge across heterogeneous WAMs remains challenging due to mismatched latent interfaces, high adaptation cost, and the rigidity of conventional distillation objectives. We propose \textbf{CKT-WAM}, a parameter-efficient \textbf{C}ontext \textbf{K}nowledge \textbf{T}ransfer framework that transfers teacher WAM's knowledge into a student WAM through a compact context in the text embedding space, rather than output imitation or dense hidden-state matching. Specifically, CKT-WAM extracts intermediate teacher hidden states, reduces the number of tokens via compressors' learnable-query cross attention (LQCA), and transforms them through an always-on generalized adapter, a lightweight router, and sparsely activated specialized adapters. The resulting context is then appended to the student's conditioning textual embeddings, thereby injecting the transferred knowledge into the student with minimal architectural modification. Experiments show that CKT-WAM consistently improves zero-shot generalization and achieves the best overall performance on LIBERO-Plus, reaching 86.1\% total success rate with only 1.17\% trainable parameters, while approaching full fine-tuning performance. Beyond simulation, CKT-WAM also demonstrates strong real-world long-horizon manipulation ability, achieving the best average success rate of 83.3\% across four multi-step and long-horizon tasks. Code is available at https://github.com/YuhuaJiang2002/CKT-WAM.
Abstract:In this paper, we propose an unmanned aerial vehicle (UAV) and bird recognition scheme with signal processing and deep learning for integrated sensing and communications (ISAC) system. We first provide the basic scene of low-altitude targets monitoring, and formulate the motion equations and echo signals for UAVs and birds. Next, we extract the centralized micro-Doppler (cmD) spectrum and the high resolution range profile (HRRP) of the low-altitude target from the echo signals. Then we design a dual feature fusion enabled low-altitude target recognition network with convolutional neural network (CNN), which employs both the images of cmD spectrum and HRRP as inputs to jointly distinguish between UAV and bird. Meanwhile, we generate 237600 cmD and HRRP image samples to train, validate, and evaluate the designed low-altitude target recognition network. The proposed scheme is termed as AirGuard, whose effectiveness has been demonstrated by simulation results.




Abstract:Vision-language-action (VLA) models have recently emerged as a powerful paradigm for building generalist robots. However, traditional VLA models that generate actions through flow matching (FM) typically rely on rigid and uniform time schedules, i.e., synchronous FM (SFM). Without action context awareness and asynchronous self-correction, SFM becomes unstable in long-horizon tasks, where a single action error can cascade into failure. In this work, we propose asynchronous flow matching VLA (AsyncVLA), a novel framework that introduces temporal flexibility in asynchronous FM (AFM) and enables self-correction in action generation. AsyncVLA breaks from the vanilla SFM in VLA models by generating the action tokens in a non-uniform time schedule with action context awareness. Besides, our method introduces the confidence rater to extract confidence of the initially generated actions, enabling the model to selectively refine inaccurate action tokens before execution. Moreover, we propose a unified training procedure for SFM and AFM that endows a single model with both modes, improving KV-cache utilization. Extensive experiments on robotic manipulation benchmarks demonstrate that AsyncVLA is data-efficient and exhibits self-correction ability. AsyncVLA achieves state-of-the-art results across general embodied evaluations due to its asynchronous generation in AFM. Our code is available at https://github.com/YuhuaJiang2002/AsyncVLA.
Abstract:Specialized Generalist Models (SGMs) aim to preserve broad capabilities while achieving expert-level performance in target domains. However, traditional LLM structures including Transformer, Linear Attention, and hybrid models do not employ specialized memory mechanism guided by task information. In this paper, we present Nirvana, an SGM with specialized memory mechanism, linear time complexity, and test-time task information extraction. Besides, we propose the Task-Aware Memory Trigger ($\textit{Trigger}$) that flexibly adjusts memory mechanism based on the current task's requirements. In Trigger, each incoming sample is treated as a self-supervised fine-tuning task, enabling Nirvana to adapt its task-related parameters on the fly to domain shifts. We also design the Specialized Memory Updater ($\textit{Updater}$) that dynamically memorizes the context guided by Trigger. We conduct experiments on both general language tasks and specialized medical tasks. On a variety of natural language modeling benchmarks, Nirvana achieves competitive or superior results compared to the existing LLM structures. To prove the effectiveness of Trigger on specialized tasks, we test Nirvana's performance on a challenging medical task, i.e., Magnetic Resonance Imaging (MRI). We post-train frozen Nirvana backbone with lightweight codecs on paired electromagnetic signals and MRI images. Despite the frozen Nirvana backbone, Trigger guides the model to adapt to the MRI domain with the change of task-related parameters. Nirvana achieves higher-quality MRI reconstruction compared to conventional MRI models as well as the models with traditional LLMs' backbone, and can also generate accurate preliminary clinical reports accordingly.
Abstract:Multi-user millimeter-wave communication relies on narrow beams and dense cell deployments to ensure reliable connectivity. However, tracking optimal beams for multiple mobile users across multiple base stations (BSs) results in significant signaling overhead. Recent works have explored the capability of out-of-band (OOB) modalities in obtaining spatial characteristics of wireless channels and reducing pilot overhead in single-BS single-user/multi-user systems. However, applying OOB modalities for multi-BS selection towards dense cell deployments leads to high coordination overhead, i.e, excessive computing overhead and high latency in data exchange. How to leverage OOB modalities to eliminate pilot overhead and achieve efficient multi-BS coordination in multi-BS systems remains largely unexplored. In this paper, we propose a novel OOB modality synergy (OMS) based mobility management scheme to realize multi-user beam prediction and proactive BS selection by synergizing two OOB modalities, i.e., vision and location. Specifically, mobile users are initially identified via spatial alignment of visual sensing and location feedback, and then tracked according to the temporal correlation in image sequence. Subsequently, a binary encoding map based gain and beam prediction network (BEM-GBPN) is designed to predict beamforming gains and optimal beams for mobile users at each BS, such that a central unit can control the BSs to perform user handoff and beam switching. Simulation results indicate that the proposed OMS-based mobility management scheme enhances beam prediction and BS selection accuracy and enables users to achieve 91% transmission rates of the optimal with zero pilot overhead and significantly improve multi-BS coordination efficiency compared to existing methods.
Abstract:The emergence of sixth-generation and beyond communication systems is expected to fundamentally transform digital experiences through introducing unparalleled levels of intelligence, efficiency, and connectivity. A promising technology poised to enable this revolutionary vision is the wireless large AI model (WLAM), characterized by its exceptional capabilities in data processing, inference, and decision-making. In light of these remarkable capabilities, this paper provides a comprehensive survey of WLAM, elucidating its fundamental principles, diverse applications, critical challenges, and future research opportunities. We begin by introducing the background of WLAM and analyzing the key synergies with wireless networks, emphasizing the mutual benefits. Subsequently, we explore the foundational characteristics of WLAM, delving into their unique relevance in wireless environments. Then, the role of WLAM in optimizing wireless communication systems across various use cases and the reciprocal benefits are systematically investigated. Furthermore, we discuss the integration of WLAM with emerging technologies, highlighting their potential to enable transformative capabilities and breakthroughs in wireless communication. Finally, we thoroughly examine the high-level challenges hindering the practical implementation of WLAM and discuss pivotal future research directions.




Abstract:In this document, we revise the results of [1] based on more reasonable assumptions regarding data shuffling and parameter setup of deep neural networks (DNNs). Thus, the simulation results can now more reasonably demonstrate the performance of both the proposed and compared beam alignment methods. We revise the simulation steps and make moderate modifications to the design of the vehicle distribution feature (VDF) for the proposed vision based beam alignment when the MS location is available (VBALA). Specifically, we replace the 2D grids of the VDF with 3D grids and utilize the vehicle locations to expand the dimensions of the VDF. Then, we revise the simulation results of Fig. 11, Fig. 12, Fig. 13, Fig. 14, and Fig. 15 in [1] to reaffirm the validity of the conclusions.




Abstract:Integrated sensing and communications (ISAC) has emerged as a transformative paradigm for next-generation wireless systems. In this paper, we present a novel ISAC scheme that leverages the diffusion Schrodinger bridge (DSB) to realize the sensing of electromagnetic (EM) property of a target as well as the reconstruction of the wireless channel. The DSB framework connects EM property sensing and channel reconstruction by establishing a bidirectional process: the forward process transforms the distribution of EM property into the channel distribution, while the reverse process reconstructs the EM property from the channel. To handle the difference in dimensionality between the high-dimensional sensing channel and the lower-dimensional EM property, we generate latent representations using an autoencoder network. The autoencoder compresses the sensing channel into a latent space that retains essential features, which incorporates positional embeddings to process spatial context. The simulation results demonstrate the effectiveness of the proposed DSB framework, which achieves superior reconstruction of the targets shape, relative permittivity, and conductivity. Moreover, the proposed method can also realize high-fidelity channel reconstruction given the EM property of the target. The dual capability of accurately sensing the EM property and reconstructing the channel across various positions within the sensing area underscores the versatility and potential of the proposed approach for broad application in future ISAC systems.




Abstract:In this paper, we propose a novel scheme to estimate the six dimensional (6D) motion parameters of dynamic target for monostatic integrated sensing and communications (ISAC) system. We first provide a generic ISAC framework for dynamic target sensing based on massive multiple input and multiple output (MIMO) array. Next, we derive the relationship between the sensing channel of ISAC base station (BS) and the 6D motion parameters of dynamic target. Then, we employ the array signal processing methods to estimate the horizontal angle, pitch angle, distance, and virtual velocity of dynamic target. Since the virtual velocities observed by different antennas are different, we adopt plane fitting to estimate the dynamic target's radial velocity, horizontal angular velocity, and pitch angular velocity from these virtual velocities. Simulation results demonstrate the effectiveness of the proposed 6D motion parameters estimation scheme, which also confirms a new finding that one single BS with massive MIMO array is capable of estimating the horizontal angular velocity and pitch angular velocity of dynamic target.