Electrocardiogram (ECG) signal is an important physiological signal which contains cardiac information and is the basis to diagnosis cardiac related diseases. In this paper, several innovative and efficient methods based on adaptive filter and empirical mode decomposition (EMD) to denoise ECG signal contaminated by various kinds of noise, including baseline wander (BW), power line interference (PLI), electrode motion artifact (EM) and muscle artifact (MA), are proposed. We first present a novel method based on EMD and adaptive filter for the removal of BW and PLI in ECG signal. We then extend the method to the complex scenario where four most common noises, PLI, BW, EM and MA are present. The proposed Parallel EMD adaptive filter structure yields the best SNR improvement on the MIT-BIH arrhythmia database, corrupted by the four types of noises.