Hysteresis-induced drift is a major issue in the detection of force induced during grinding and cutting operations. In this paper, we propose an external force estimation method based on the Mel spectrogram of the force obtained from a force sensor. We focus on the frequent strong correlation between the vibration frequency and the external force in operations with periodic vibrations. The frequency information is found to be more effective for an accurate force estimation than the amplitude in cases with large noise caused by vibration. We experimentally demonstrate that the force estimation method that combines the Mel spectrogram with a neural network is robust against drift.