We present a progressive image decomposition method based on a novel non-linear filter named Sub-window Variance filter. Our method is specifically designed for image detail enhancement purpose; this application requires extraction of image details which are small in terms of both spatial and variation scales. We propose a local statistical edge model which develops its edge awareness using spatially defined image statistics. Our decomposition method is controlled by two intuitive parameters which allow the users to define what image details to suppress or enhance. By using the summed-area table acceleration method, our decomposition pipeline is highly parallel. The proposed filter is gradient preserving and this allows our enhancement results free from the gradient-reversal artefact. In our evaluations, we compare our method in various multi-scale image detail manipulation applications with other mainstream solutions.