Spatial perception problems are the fundamental building blocks of robotics and computer vision. However, in many real-world situations, they inevitably suffer from the issue of outliers, which hinders traditional solvers from making correct estimates. In this paper, we present a novel, general-purpose robust estimator IMOT (Iterative Multi-layered Otsu's Thresholding) using standard non-minimal solvers to rapidly reject outliers for spatial perception problems. First, we propose a new outlier-robust iterative optimizing framework where in each iteration all the measurement data are separated into two groups according to the residual errors and only the group with lower residual errors can be preserved for estimation in the next iteration. Second, we introduce and employ the well-known Otsu's method (from image processing) to conduct thresholding on the residual errors so as to obtain the best separation (grouping) statistically which maximizes the between-class variance. Third, to enhance robustness, we design a multi-layered Otsu's thresholding approach in combination with our framework to sift out the true inliers from outliers that might even occupy the majority of measurements. We test our robust estimator IMOT on 5 different spatial perception problems including: rotation averaging, rotation search, point cloud registration, category-level registration, and SLAM. Experiments show that IMOT is robust against 70%--90% of outliers and can typically converge in only 3--10 iterations, being 3--125 times faster than existing robust estimators: GNC and ADAPT. Moreover, IMOT is able to return robust results even without noise bound information.