Since 2012 deep learning has dramatically improved the performance in many application areas such as image classification, object detection, speech recognition, drug discovery and etc. Deep learning algorithms promise to discover the intricate hidden information inside the data by leveraging the large dataset, modeling and computing power. Although deep learning techniques show medical expert level performance in a lot of cell level images analyzing applications, however some of the cell level images analyzing applications are still not explored or under explored. In this work, we explored the scattering based cell level Cryptosporidium and Giardia detection in the water with deep learning. Our experimental demonstrates that the new developed deep learning-based algorithm surpassed the hand-crafted SVM based algorithm with above 95.6 percentage in accuracy and 100fps in speed on embedded Jetson TX2 platform. Our research will lend to real-time and high accuracy cell level Cryptosporidium and Giardia detection system in the future.