In atomistic simulations of solids, ability to classify crystal phases and lattice defects in the presence of thermal fluctuations is essential for gaining deeper insights into the simulated dynamics. The need for accurate and efficient characterization methods is especially acute in presently emerging large-scale simulations of multi-phase systems far from equilibrium. Taking the perspective that delineating order and disorder features from ubiquitous thermal vibrations is akin to extracting signal from noise, we consider classification of ordered phases and identification of disordered crystal defects to be fundamentally the same problem and address them both with a unified approach: a denoising score function that removes thermal noise and recovers any underlying crystalline order-disorder. Built on a rotationally equivariant graph neural network (NequIP), the denoiser was trained entirely with synthetically noised structures and requires no simulation data during training. To demonstrate its denoising capabilities, the denoiser is shown to effectively remove thermal vibrations of BCC, FCC, and HCP crystal structures without impacting the underlying disordered defects, including point defects, dislocations, grain boundaries, and liquid disorder. In particular the denoiser was applied to two relatively complex MD simulations that present practical challenges: a Cu solidification trajectory involving a polymorphic nucleus, and a trajectory of BCC Ta undergoing plastic deformation resulting in dislocation networks and point defect clusters. In both cases the denoiser facilitates or trivializes the subsequent characterization of the order-disorder features. Lastly, we outline future work to extend our denoising model to more complex crystal structures and to multi-element systems.