In this paper, we tackle the problem of image anomaly detection and segmentation. Anomaly detection is to make a binary decision whether an input image contains an anomaly or not, and anomaly segmentation aims to locate the defect in a pixel-level. SVDD is a longstanding algorithm for an anomaly detection. We extend its deep learning variant to patch-level using self-supervised learning. The extension enables the anomaly segmentation, and it improves the detection performance as well. As a result, we achieved a state-of-the-art performances on a standard industrial dataset, MVTec AD. Detailed analysis on the proposed method offers a useful insight about its behavior.