Alert button
Picture for Christian Daul

Christian Daul

Alert button

A metric learning approach for endoscopic kidney stone identification

Jul 13, 2023
Jorge Gonzalez-Zapata, Francisco Lopez-Tiro, Elias Villalvazo-Avila, Daniel Flores-Araiza, Jacques Hubert, Andres Mendez-Vazquez, Gilberto Ochoa-Ruiz, Christian Daul

Figure 1 for A metric learning approach for endoscopic kidney stone identification
Figure 2 for A metric learning approach for endoscopic kidney stone identification
Figure 3 for A metric learning approach for endoscopic kidney stone identification
Figure 4 for A metric learning approach for endoscopic kidney stone identification

Several Deep Learning (DL) methods have recently been proposed for an automated identification of kidney stones during an ureteroscopy to enable rapid therapeutic decisions. Even if these DL approaches led to promising results, they are mainly appropriate for kidney stone types for which numerous labelled data are available. However, only few labelled images are available for some rare kidney stone types. This contribution exploits Deep Metric Learning (DML) methods i) to handle such classes with few samples, ii) to generalize well to out of distribution samples, and iii) to cope better with new classes which are added to the database. The proposed Guided Deep Metric Learning approach is based on a novel architecture which was designed to learn data representations in an improved way. The solution was inspired by Few-Shot Learning (FSL) and makes use of a teacher-student approach. The teacher model (GEMINI) generates a reduced hypothesis space based on prior knowledge from the labeled data, and is used it as a guide to a student model (i.e., ResNet50) through a Knowledge Distillation scheme. Extensive tests were first performed on two datasets separately used for the recognition, namely a set of images acquired for the surfaces of the kidney stone fragments, and a set of images of the fragment sections. The proposed DML-approach improved the identification accuracy by 10% and 12% in comparison to DL-methods and other DML-approaches, respectively. Moreover, model embeddings from the two dataset types were merged in an organized way through a multi-view scheme to simultaneously exploit the information of surface and section fragments. Test with the resulting mixed model improves the identification accuracy by at least 3% and up to 30% with respect to DL-models and shallow machine learning methods, respectively.

Viaarxiv icon

Deep learning-based image exposure enhancement as a pre-processing for an accurate 3D colon surface reconstruction

Apr 14, 2023
Ricardo Espinosa, Carlos Axel Garcia-Vega, Gilberto Ochoa-Ruiz, Dominique Lamarque, Christian Daul

Figure 1 for Deep learning-based image exposure enhancement as a pre-processing for an accurate 3D colon surface reconstruction
Figure 2 for Deep learning-based image exposure enhancement as a pre-processing for an accurate 3D colon surface reconstruction
Figure 3 for Deep learning-based image exposure enhancement as a pre-processing for an accurate 3D colon surface reconstruction
Figure 4 for Deep learning-based image exposure enhancement as a pre-processing for an accurate 3D colon surface reconstruction

This contribution shows how an appropriate image pre-processing can improve a deep-learning based 3D reconstruction of colon parts. The assumption is that, rather than global image illumination corrections, local under- and over-exposures should be corrected in colonoscopy. An overview of the pipeline including the image exposure correction and a RNN-SLAM is first given. Then, this paper quantifies the reconstruction accuracy of the endoscope trajectory in the colon with and without appropriate illumination correction

* This article has been submitted to GRESTI 2023 for review 
Viaarxiv icon

Deep Prototypical-Parts Ease Morphological Kidney Stone Identification and are Competitively Robust to Photometric Perturbations

Apr 08, 2023
Daniel Flores-Araiza, Francisco Lopez-Tiro, Jonathan El-Beze, Jacques Hubert, Miguel Gonzalez-Mendoza, Gilberto Ochoa-Ruiz, Christian Daul

Figure 1 for Deep Prototypical-Parts Ease Morphological Kidney Stone Identification and are Competitively Robust to Photometric Perturbations
Figure 2 for Deep Prototypical-Parts Ease Morphological Kidney Stone Identification and are Competitively Robust to Photometric Perturbations
Figure 3 for Deep Prototypical-Parts Ease Morphological Kidney Stone Identification and are Competitively Robust to Photometric Perturbations
Figure 4 for Deep Prototypical-Parts Ease Morphological Kidney Stone Identification and are Competitively Robust to Photometric Perturbations

Identifying the type of kidney stones can allow urologists to determine their cause of formation, improving the prescription of appropriate treatments to diminish future relapses. Currently, the associated ex-vivo diagnosis (known as Morpho-constitutional Analysis, MCA) is time-consuming, expensive and requires a great deal of experience, as it requires a visual analysis component that is highly operator dependant. Recently, machine learning methods have been developed for in-vivo endoscopic stone recognition. Deep Learning (DL) based methods outperform non-DL methods in terms of accuracy but lack explainability. Despite this trade-off, when it comes to making high-stakes decisions, it's important to prioritize understandable Computer-Aided Diagnosis (CADx) that suggests a course of action based on reasonable evidence, rather than a model prescribing a course of action. In this proposal, we learn Prototypical Parts (PPs) per kidney stone subtype, which are used by the DL model to generate an output classification. Using PPs in the classification task enables case-based reasoning explanations for such output, thus making the model interpretable. In addition, we modify global visual characteristics to describe their relevance to the PPs and the sensitivity of our model's performance. With this, we provide explanations with additional information at the sample, class and model levels in contrast to previous works. Although our implementation's average accuracy is lower than state-of-the-art (SOTA) non-interpretable DL models by 1.5 %, our models perform 2.8% better on perturbed images with a lower standard deviation, without adversarial training. Thus, Learning PPs has the potential to create more robust DL models.

* This paper has been accepted at the LatinX in Computer Vision Research Workshop at CVPR2023 as a full paper and it will appear on the CVPR proceedings 
Viaarxiv icon

Improving automatic endoscopic stone recognition using a multi-view fusion approach enhanced with two-step transfer learning

Apr 06, 2023
Francisco Lopez-Tiro, Elias Villalvazo-Avila, Juan Pablo Betancur-Rengifo, Jonathan El-Beze, Jacques Hubert, Gilberto Ochoa-Ruiz, Christian Daul

Figure 1 for Improving automatic endoscopic stone recognition using a multi-view fusion approach enhanced with two-step transfer learning
Figure 2 for Improving automatic endoscopic stone recognition using a multi-view fusion approach enhanced with two-step transfer learning
Figure 3 for Improving automatic endoscopic stone recognition using a multi-view fusion approach enhanced with two-step transfer learning

This contribution presents a deep-learning method for extracting and fusing image information acquired from different viewpoints, with the aim to produce more discriminant object features for the identification of the type of kidney stones seen in endoscopic images. The model was further improved with a two-step transfer learning approach and by attention blocks to refine the learned feature maps. Deep feature fusion strategies improved the results of single view extraction backbone models by more than 6% in terms of accuracy of the kidney stones classification.

* This paper has been submitted to GRETSI 2023 for reviewing 
Viaarxiv icon

Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies

Nov 05, 2022
Elias Villalvazo-Avila, Francisco Lopez-Tiro, Jonathan El-Beze, Jacques Hubert, Miguel Gonzalez-Mendoza, Gilberto Ochoa-Ruiz, Christian Daul

Figure 1 for Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies
Figure 2 for Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies
Figure 3 for Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies
Figure 4 for Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies

This contribution presents a deep learning method for the extraction and fusion of information relating to kidney stone fragments acquired from different viewpoints of the endoscope. Surface and section fragment images are jointly used during the training of the classifier to improve the discrimination power of the features by adding attention layers at the end of each convolutional block. This approach is specifically designed to mimic the morpho-constitutional analysis performed in ex-vivo by biologists to visually identify kidney stones by inspecting both views. The addition of attention mechanisms to the backbone improved the results of single view extraction backbones by 4% on average. Moreover, in comparison to the state-of-the-art, the fusion of the deep features improved the overall results up to 11% in terms of kidney stone classification accuracy.

* This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible 
Viaarxiv icon

Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging

Oct 26, 2022
Axel Garcia-Vega, Ricardo Espinosa, Luis Ramirez-Guzman, Thomas Bazin, Luis Falcon-Morales, Gilberto Ochoa-Ruiz, Dominique Lamarque, Christian Daul

Figure 1 for Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging
Figure 2 for Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging
Figure 3 for Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging
Figure 4 for Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging

Endoscopy is the most widely used imaging technique for the diagnosis of cancerous lesions in hollow organs. However, endoscopic images are often affected by illumination artefacts: image parts may be over- or underexposed according to the light source pose and the tissue orientation. These artifacts have a strong negative impact on the performance of computer vision or AI-based diagnosis tools. Although endoscopic image enhancement methods are greatly required, little effort has been devoted to over- and under-exposition enhancement in real-time. This contribution presents an extension to the objective function of LMSPEC, a method originally introduced to enhance images from natural scenes. It is used here for the exposure correction in endoscopic imaging and the preservation of structural information. To the best of our knowledge, this contribution is the first one that addresses the enhancement of endoscopic images using deep learning (DL) methods. Tested on the Endo4IE dataset, the proposed implementation has yielded a significant improvement over LMSPEC reaching a SSIM increase of 4.40% and 4.21% for over- and underexposed images, respectively.

* This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible 
Viaarxiv icon

Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning

Oct 24, 2022
Francisco Lopez-Tiro, Juan Pablo Betancur-Rengifo, Arturo Ruiz-Sanchez, Ivan Reyes-Amezcua, Jonathan El-Beze, Jacques Hubert, Michel Daudon, Gilberto Ochoa-Ruiz, Christian Daul

Figure 1 for Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning
Figure 2 for Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning
Figure 3 for Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning
Figure 4 for Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning

Knowing the cause of kidney stone formation is crucial to establish treatments that prevent recurrence. There are currently different approaches for determining the kidney stone type. However, the reference ex-vivo identification procedure can take up to several weeks, while an in-vivo visual recognition requires highly trained specialists. Machine learning models have been developed to provide urologists with an automated classification of kidney stones during an ureteroscopy; however, there is a general lack in terms of quality of the training data and methods. In this work, a two-step transfer learning approach is used to train the kidney stone classifier. The proposed approach transfers knowledge learned on a set of images of kidney stones acquired with a CCD camera (ex-vivo dataset) to a final model that classifies images from endoscopic images (ex-vivo dataset). The results show that learning features from different domains with similar information helps to improve the performance of a model that performs classification in real conditions (for instance, uncontrolled lighting conditions and blur). Finally, in comparison to models that are trained from scratch or by initializing ImageNet weights, the obtained results suggest that the two-step approach extracts features improving the identification of kidney stones in endoscopic images.

* This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible 
Viaarxiv icon

A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models

Jul 06, 2022
Axel Garcia-Vega, Ricardo Espinosa, Gilberto Ochoa-Ruiz, Thomas Bazin, Luis Eduardo Falcon-Morales, Dominique Lamarque, Christian Daul

Figure 1 for A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models
Figure 2 for A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models
Figure 3 for A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models
Figure 4 for A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models

Endoscopy is the most widely used medical technique for cancer and polyp detection inside hollow organs. However, images acquired by an endoscope are frequently affected by illumination artefacts due to the enlightenment source orientation. There exist two major issues when the endoscope's light source pose suddenly changes: overexposed and underexposed tissue areas are produced. These two scenarios can result in misdiagnosis due to the lack of information in the affected zones or hamper the performance of various computer vision methods (e.g., SLAM, structure from motion, optical flow) used during the non invasive examination. The aim of this work is two-fold: i) to introduce a new synthetically generated data-set generated by a generative adversarial techniques and ii) and to explore both shallow based and deep learning-based image-enhancement methods in overexposed and underexposed lighting conditions. Best quantitative results (i.e., metric based results), were obtained by the deep-learnnig-based LMSPEC method,besides a running time around 7.6 fps)

Viaarxiv icon

Evaluating object detector ensembles for improving the robustness of artifact detection in endoscopic video streams

Jun 15, 2022
Pedro Esteban Chavarrias-Solano, Carlos Axel Garcia-Vega, Francisco Javier Lopez-Tiro, Gilberto Ochoa-Ruiz, Thomas Bazin, Dominique Lamarque, Christian Daul

Figure 1 for Evaluating object detector ensembles for improving the robustness of artifact detection in endoscopic video streams
Figure 2 for Evaluating object detector ensembles for improving the robustness of artifact detection in endoscopic video streams
Figure 3 for Evaluating object detector ensembles for improving the robustness of artifact detection in endoscopic video streams
Figure 4 for Evaluating object detector ensembles for improving the robustness of artifact detection in endoscopic video streams

In this contribution we use an ensemble deep-learning method for combining the prediction of two individual one-stage detectors (i.e., YOLOv4 and Yolact) with the aim to detect artefacts in endoscopic images. This ensemble strategy enabled us to improve the robustness of the individual models without harming their real-time computation capabilities. We demonstrated the effectiveness of our approach by training and testing the two individual models and various ensemble configurations on the "Endoscopic Artifact Detection Challenge" dataset. Extensive experiments show the superiority, in terms of mean average precision, of the ensemble approach over the individual models and previous works in the state of the art.

Viaarxiv icon