Convolutional neural networks (CNNs) have demonstrated remarkable results in image classification tasks for benchmark and practical uses. The CNNs with deeper architectures have achieved higher performances recently thanks to their robustness to parallel shift of objects in images aw well as their numerous parameters and resulting high expression ability. However, the CNNs have a limited robustness to other geometric transformations such as scaling and rotation. This problem is considered to limit performance improvement of the deep CNNs but there is no established solution. This study focuses on scale transformation and proposes a novel network architecture called weight-shared multi-stage network (WSMS-Net), consisting of multiple stages of CNNs. The WSMS-Net is easily combined with existing deep CNNs, such as ResNet and DenseNet, and enables them to acquire a robustness to scaling of objects. The experimental results demonstrate that existing deep CNNs combined with the proposed WSMS-Net achieve higher accuracy for image classification tasks only with a little increase in the number of parameters.