Ensemble methods have been widely used for improving the results of the best single classification model. Indeed, a large body of works have achieved better results mainly by applying one specific ensemble method. However, very few works analyze complex fusion schemes using heterogeneous ensemble strategies. This paper is three-fold: 1) It provides a tutorial of the most popular ensemble methods, 2) analyzes the best ensembles using MNIST as guiding thread and 3) shows that complex fusion architectures based on heterogeneous ensembles can be considered as a mode of taking benefit from diversity. We introduce a complex fusion design that achieves a new record in MNIST with only 10 misclassified images.