In this research, an attention-based depthwise separable neural network with Bayesian optimization (ADSNN-BO) is proposed to detect and classify rice disease from rice leaf images. Rice diseases frequently result in 20 to 40 \% corp production loss in yield and is highly related to the global economy. Rapid disease identification is critical to plan treatment promptly and reduce the corp losses. Rice disease diagnosis is still mainly performed manually. To achieve AI assisted rapid and accurate disease detection, we proposed the ADSNN-BO model based on MobileNet structure and augmented attention mechanism. Moreover, Bayesian optimization method is applied to tune hyper-parameters of the model. Cross-validated classification experiments are conducted based on a public rice disease dataset with four categories in total. The experimental results demonstrate that our mobile compatible ADSNN-BO model achieves a test accuracy of 94.65\%, which outperforms all of the state-of-the-art models tested. To check the interpretability of our proposed model, feature analysis including activation map and filters visualization approach are also conducted. Results show that our proposed attention-based mechanism can more effectively guide the ADSNN-BO model to learn informative features. The outcome of this research will promote the implementation of artificial intelligence for fast plant disease diagnosis and control in the agricultural field.