Although Physics-Informed Neural Networks (PINNs) have been successfully applied to various differential equations, accurately solving perturbed convection-diffusion-reaction problems is still extremely challenging for PINNs. This paper investigates the source of the learning difficulties and finds that the rapid transition of potential solution in the layer region causes the failure of convergence. Based on this finding, we present a curriculum learning method that encourages neural networks to ``prioritize the learning on easier non-layer regions''. The method helps PINNs to dynamically adjust the training data weights, speed up the learning procedure, and ultimately significantly improve the accuracy of the network approximation. Extensive evaluation on multiple typical model equations shows that the proposed approach accurately captures the resolution of the layer regions, and achieves multiple orders of magnitude lower root-mean-squared error than ordinary PINNs. We provide our PyTorch code at https://github.com/WYu-Feng/CLPINN