In this paper, we show that conditional inference trees and ensembles are suitable methods for modeling linguistic variation. As against earlier linguistic applications, however, we claim that their suitability is strongly increased if we combine prediction and interpretation. To that end, we have developed a statistical method, PrInDT (Prediction and Interpretation with Decision Trees), which we introduce and discuss in the present paper.