In this paper, a lightweight and accurate localization algorithm is proposed using measurements from multiple inertial measurement units (IMUs). IMU is a low-cost motion sensor which provides measurements on rotational velocity and gravity compensated linear acceleration of a moving platform, and it is widely used in localization problems. To date, most existing work that employs IMU sensor for localization focuses on algorithms or applications based on a single IMU. While this category of design yields acceptable accuracy and robustness for different use cases, the overall performance can be further improved by using multiple IMUs. To this end, we propose a lightweight and accurate algorithm for IMU-assisted localization, which is able to obtain noticeable performance gain without incurring additional computational cost. To achieve this, we first probabilistically map measurements from all IMUs onto a virtual IMU. This step is performed by stochastic estimation with least-squares estimators and probabilistic marginalization of inter-IMU rotational accelerations. Subsequently, propagation equations of both state vector and error state of the virtual IMU are also derived, which enables the use of the classical filter-based or optimization-based localization algorithms via the virtual IMU. Finally, results from both simulation and real-world tests are provided, which demonstrate that the proposed algorithm outperforms competing algorithms by noticeable margins.