Mutual information (MI) is a popular similarity measure for image registration, whereby good registration can be achieved by maximizing the compactness of the clusters in the joint histogram. However, MI is sensitive to the "outlier" objects that appear in one image but not the other, and also suffers from local and biased maxima. We propose a novel joint saliency map (JSM) to highlight the corresponding salient structures in the two images, and emphatically group those salient structures into the smoothed compact clusters in the weighted joint histogram. This strategy could solve both the outlier and the local maxima problems. Experimental results show that the JSM-MI based algorithm is not only accurate but also robust for registration of challenging image pairs with outliers.