Most blind deconvolution methods usually pre-define a large kernel size to guarantee the support domain. Blur kernel estimation error is likely to be introduced, and is proportional to kernel size. In this paper, we experimentally and theoretically show the reason of noises introduction in oversized kernel by demonstrating that sizeable kernels lead to lower optimization cost. To eliminate this adverse effect, we propose a low rank-based regularization on blur kernel by analyzing the structural information in degraded kernels. Compared with the sparsity prior, e.g., $\ell_\alpha$-norm, our regularization term can effectively suppress random noises in oversized kernels. On benchmark test dataset, the proposed method is compared with several state-of-the-art methods, and can achieve better quantitative score. Especially, the improvement margin is much more significant for oversized blur kernels. We also validate the proposed method on real-world blurry images.