Deep learning has shown to have great potential in medical applications. In critical domains as such, it is of high interest to have trustworthy algorithms which are able to tell when reliable assessments cannot be guaranteed. Detecting out-of-distribution (OOD) samples is a crucial step towards building a safe classifier. Following a previous study, showing that it is possible to classify breast cancer in point-of-care ultrasound images, this study investigates OOD detection using three different methods: softmax, energy score and deep ensembles. All methods are tested on three different OOD data sets. The results show that the energy score method outperforms the softmax method, performing well on two of the data sets. The ensemble method is the most robust, performing the best at detecting OOD samples for all three OOD data sets.