The economic and production losses in agricultural industry worldwide are due to the presence of diseases in the several kinds of fruits. In this paper, a method for the classification of fruit diseases is proposed and experimentally validated. The image processing based proposed approach is composed of the following main steps; in the first step K-Means clustering technique is used for the defect segmentation, in the second step color and textural cues are extracted and fused from the segmented image, and finally images are classified into one of the classes by using a Multi-class Support Vector Machine. We have considered diseases of apple as a test case and evaluated our approach for three types of apple diseases namely apple scab, apple blotch and apple rot and normal apples without diseases. Our experimentation points out that the proposed fusion scheme can significantly support accurate detection and automatic classification of fruit diseases.