Optical coherence tomography (OCT) has become a favorable device in the Dermatology discipline due to its moderate resolution and penetration depth. OCT images however contain a grainy pattern, called speckle, due to the use of a broadband source in the configuration of OCT. So far, a variety of filtering (de-speckling) techniques is introduced to reduce speckle in OCT images. Most of these methods are generic and can be applied to OCT images of different tissues. The ambition of this work is to provide a de-speckling framework specialized for filtering skin tissues for the community to utilize, adapt or build upon. In this paper, we present an adaptive cluster-based filtering framework, optimized for speckle reduction of OCT skin images. In this framework, by considering the layered structure of skin, first the OCT skin images are segmented into differentiable layers utilizing clustering algorithms, and then each cluster is de-speckled individually using adaptive filtering techniques. In this study, hierarchical clustering algorithm and adaptive Wiener filtering technique are utilized to develop the framework. The proposed method is tested on optical solid phantoms with predetermined optical properties. The method is also tested on healthy human skin images. The results show that the proposed cluster-based filtering method can effectively reduce the speckle and increase the signal-to-noise ratio and contrast while preserving the edges in the image. The proposed cluster-based filtering framework enables researchers to develop unsupervised learning solutions for de-speckling OCT skin images using adaptive filtering methods, or extend the framework to new applications.