Integrated sensing and communication (ISAC) system has received growing attention, especially in the context of B5G/6G development. Combining the reconfigurable intelligent surface (RIS) with wireless communication process, a novel passive sensing technique is formulated in this paper to estimate the direction of arrival (DOA) of the targets, where the control matrix of the RIS is used to to realize the multiple measurements with only one full-functional receiving channel. Unlike the existing methods, the interference signals introduced by wireless communication are also considered. To improve the DOA estimation, a novel atomic norm-based method is proposed to remove the interference signals by the sparse reconstruction. The DOA is estimated after the interference removal by a novel Hankel-based multiple signal classification (MUSIC) method. Then, an optimization method is also developed for the measurement matrix to reduce the power interference signals and keep the measurement matrix's randomness, which guarantees the performance of the sparse reconstruction. Finally, we derive the theoretical Cram\'{e}r-Rao lower bound (CRLB) for the proposed system on the DOA estimation. Simulation results show that the proposed method outperforms the existing methods in the DOA estimation and shows the corresponding CRLB with different distributions of the sensing node. The code about the proposed method is available online https://github.com/chenpengseu/PassiveDOA-ISAC-RIS.git.