A Simple Efficiency Incremental Learning Framework via Vision-Language Model with Nonlinear Multi-Adapters

Incremental Learning (IL) aims to learn new tasks while preserving previously acquired knowledge. Integrating the zero-shot learning capabilities of pre-trained vision-language models into IL methods has marked a significant advancement. However, these methods face three primary challenges: (1) the need for improved training efficiency; (2) reliance on a memory bank to store previous data; and (3) the necessity of a strong backbone to augment the model's capabilities. In this paper, we propose SimE, a Simple and Efficient framework that employs a vision-language model with adapters designed specifically for the IL task. We report a remarkable phenomenon: there is a nonlinear correlation between the number of adaptive adapter connections and the model's IL capabilities. While increasing adapter connections between transformer blocks improves model performance, adding more adaptive connections within transformer blocks during smaller incremental steps does not enhance, and may even degrade the model's IL ability. Extensive experimental results show that SimE surpasses traditional methods by 9.6% on TinyImageNet and outperforms other CLIP-based methods by 5.3% on CIFAR-100. Furthermore, we conduct a systematic study to enhance the utilization of the zero-shot capabilities of CLIP. We suggest replacing SimE's encoder with a CLIP model trained on larger datasets (e.g., LAION2B) and stronger architectures (e.g., ViT-L/14).

Can 3GPP New Radio Non-Terrestrial Networks Meet the IMT-2020 Requirements for Satellite Radio Interface Technology?

The International Telecommunication Union defined the requirements for 5G in the International Mobile Telecommunications 2020 (IMT-2020) standard in 2017. Since then, advances in technology and standardization have made the ubiquitous deployment of 5G via satellite a practical possibility, for example, in locations where terrestrial networks (TNs) are not available. However, it may be difficult for satellite networks to achieve the same performance as TNs. To address this, the IMT-2020 requirements for satellite radio interface technology have recently been established. In this paper, these requirements are evaluated through system simulations for the 3rd Generation Partnership Project New Radio non-terrestrial networks with a low Earth orbit satellite. The focus is on the throughput, area traffic capacity, and spectral efficiency requirements. It is observed that the downlink (DL) requirements can be met for user equipment with 2 receive antenna elements. The results also reveal that frequency reuse factor 1 (FRF1) may outperform FRF3 in DL with a dual-antenna setup, which is a surprising finding since FRF3 is typically considered to outperform FRF1 due to better interference reduction. For uplink (UL), 1 transmit antenna is sufficient to meet the requirements by a relatively large margin - a promising result given that UL is generally more demanding.