Alert button
Picture for Atsushi Ike

Atsushi Ike

Alert button

Yet Another Accelerated SGD: ResNet-50 Training on ImageNet in 74.7 seconds

Mar 29, 2019
Masafumi Yamazaki, Akihiko Kasagi, Akihiro Tabuchi, Takumi Honda, Masahiro Miwa, Naoto Fukumoto, Tsuguchika Tabaru, Atsushi Ike, Kohta Nakashima

Figure 1 for Yet Another Accelerated SGD: ResNet-50 Training on ImageNet in 74.7 seconds
Figure 2 for Yet Another Accelerated SGD: ResNet-50 Training on ImageNet in 74.7 seconds
Figure 3 for Yet Another Accelerated SGD: ResNet-50 Training on ImageNet in 74.7 seconds
Figure 4 for Yet Another Accelerated SGD: ResNet-50 Training on ImageNet in 74.7 seconds

There has been a strong demand for algorithms that can execute machine learning as faster as possible and the speed of deep learning has accelerated by 30 times only in the past two years. Distributed deep learning using the large mini-batch is a key technology to address the demand and is a great challenge as it is difficult to achieve high scalability on large clusters without compromising accuracy. In this paper, we introduce optimization methods which we applied to this challenge. We achieved the training time of 74.7 seconds using 2,048 GPUs on ABCI cluster applying these methods. The training throughput is over 1.73 million images/sec and the top-1 validation accuracy is 75.08%.

Viaarxiv icon

An Automated CNN Recommendation System for Image Classification Tasks

Dec 27, 2016
Song Wang, Li Sun, Wei Fan, Jun Sun, Satoshi Naoi, Koichi Shirahata, Takuya Fukagai, Yasumoto Tomita, Atsushi Ike

Figure 1 for An Automated CNN Recommendation System for Image Classification Tasks
Figure 2 for An Automated CNN Recommendation System for Image Classification Tasks
Figure 3 for An Automated CNN Recommendation System for Image Classification Tasks
Figure 4 for An Automated CNN Recommendation System for Image Classification Tasks

Nowadays the CNN is widely used in practical applications for image classification task. However the design of the CNN model is very professional work and which is very difficult for ordinary users. Besides, even for experts of CNN, to select an optimal model for specific task may still need a lot of time (to train many different models). In order to solve this problem, we proposed an automated CNN recommendation system for image classification task. Our system is able to evaluate the complexity of the classification task and the classification ability of the CNN model precisely. By using the evaluation results, the system can recommend the optimal CNN model and which can match the task perfectly. The recommendation process of the system is very fast since we don't need any model training. The experiment results proved that the evaluation methods are very accurate and reliable.

* Submitted to ICME 2017 and all the methods in this paper are patented 
Viaarxiv icon