Large Models for Small Devices: Recent Advances and Empirical Analysis of Edge AI Deployment

Running large AI models on resource-constrained edge devices requires model compression to reduce model size and computation. What compresses well, however, need not deploy well. We survey dozens of recent works that report compression results on real hardware and extract practical deployment guidelines from them. Following these guidelines, we deploy compact language and image models on GPU, CPU, and Raspberry Pi platforms across question answering and image segmentation. No single technique wins across tasks. For question answering, Qwen3.5 0.8B reaches 93.85 SQuAD F1 and 92 EM under Q5_K_M GGUF quantization, while structured pruning at the same precision costs 16 F1 at a 1% ratio. For segmentation, the ranking reverses: default quantization leaves parameters and MACs unchanged, whereas pruning cuts model size by nearly 80% at near-constant mIoU. Pruning can even inflate the deployed artifact by 21-49% by breaking k-quant super-block alignment; combined with longer, less format-compliant outputs, this raises Raspberry Pi latency up to 3.4x. Compression can also manufacture the appearance of competence rather than destroy it visibly: one LoRA-recovered variant stays fully parseable and holds 71% strict BoolQ accuracy while sending 97 of 100 predictions to a single class, at 52.6% balanced accuracy. We explain these effects through neural-flow graph analysis and prefill-decode-level latency decomposition, and condense them into task-specific deployment research directions. The right technique depends on the task, the model, and the hardware. Our experiment code and artifacts are open-sourced at https://github.com/Arnavvvkumar/deployment

Vision Transformer Based User Equipment Positioning

Recently, Deep Learning (DL) techniques have been used for User Equipment (UE) positioning. However, the key shortcomings of such models is that: i) they weigh the same attention to the entire input; ii) they are not well suited for the non-sequential data e.g., when only instantaneous Channel State Information (CSI) is available. In this context, we propose an attention-based Vision Transformer (ViT) architecture that focuses on the Angle Delay Profile (ADP) from CSI matrix. Our approach, validated on the `DeepMIMO' and `ViWi' ray-tracing datasets, achieves an Root Mean Squared Error (RMSE) of 0.55m indoors, 13.59m outdoors in DeepMIMO, and 3.45m in ViWi's outdoor blockage scenario. The proposed scheme outperforms state-of-the-art schemes by $\sim$ 38\%. It also performs substantially better than other approaches that we have considered in terms of the distribution of error distance.

Rate Forecaster based Energy Aware Band Assignment in Multiband Networks

The high frequency communication bands (mmWave and sub-THz) promise tremendous data rates, however, they also have very high power consumption which is particularly significant for battery-power-limited user-equipment (UE). In this context, we design an energy aware band assignment system which reduces the power consumption while also achieving a target sum rate of M in T time-slots. We do this by using 1) Rate forecaster(s); 2) Channel forecaster(s) which forecasts T direct multistep ahead using a stacked (long short term memory) LSTM architecture. We propose an iterative rate updating algorithm which updates the target rate based on current rate and future predicted rates in a frame. The proposed approach is validated on the publicly available `DeepMIMO' dataset. Research findings shows that the rate forecaster based approach performs better than the channel forecaster. Furthermore, LSTM based predictions outperforms well celebrated Transformer predictions in terms of NRMSE and NMAE. Research findings reveals that the power consumption with this approach is ~ 300 mW lower compared to a greedy band assignment at a 1.5Gb/s target rate.