Discrimination of Radiologists Utilizing Eye-Tracking Technology and Machine Learning: A Case Study

Perception-related errors comprise most diagnostic mistakes in radiology. To mitigate this problem, radiologists employ personalized and high-dimensional visual search strategies, otherwise known as search patterns. Qualitative descriptions of these search patterns, which involve the physician verbalizing or annotating the order he/she analyzes the image, can be unreliable due to discrepancies in what is reported versus the actual visual patterns. This discrepancy can interfere with quality improvement interventions and negatively impact patient care. This study presents a novel discretized feature encoding based on spatiotemporal binning of fixation data for efficient geometric alignment and temporal ordering of eye movement when reading chest X-rays. The encoded features of the eye-fixation data are employed by machine learning classifiers to discriminate between faculty and trainee radiologists. We include a clinical trial case study utilizing the Area Under the Curve (AUC), Accuracy, F1, Sensitivity, and Specificity metrics for class separability to evaluate the discriminability between the two subjects in regard to their level of experience. We then compare the classification performance to state-of-the-art methodologies. A repeatability experiment using a separate dataset, experimental protocol, and eye tracker was also performed using eight subjects to evaluate the robustness of the proposed approach. The numerical results from both experiments demonstrate that classifiers employing the proposed feature encoding methods outperform the current state-of-the-art in differentiating between radiologists in terms of experience level. This signifies the potential impact of the proposed method for identifying radiologists' level of expertise and those who would benefit from additional training.

Incorporation of Eye-Tracking and Gaze Feedback to Characterize and Improve Radiologist Search Patterns of Chest X-rays: A Randomized Controlled Clinical Trial

Diagnostic errors in radiology often occur due to incomplete visual assessments by radiologists, despite their knowledge of predicting disease classes. This insufficiency is possibly linked to the absence of required training in search patterns. Additionally, radiologists lack consistent feedback on their visual search patterns, relying on ad-hoc strategies and peer input to minimize errors and enhance efficiency, leading to suboptimal patterns and potential false negatives. This study aimed to use eye-tracking technology to analyze radiologist search patterns, quantify performance using established metrics, and assess the impact of an automated feedback-driven educational framework on detection accuracy. Ten residents participated in a controlled trial focused on detecting suspicious pulmonary nodules. They were divided into an intervention group (received automated feedback) and a control group. Results showed that the intervention group exhibited a 38.89% absolute improvement in detecting suspicious-for-cancer nodules, surpassing the control group's improvement (5.56%, p-value=0.006). Improvement was more rapid over the four training sessions (p-value=0.0001). However, other metrics such as speed, search pattern heterogeneity, distractions, and coverage did not show significant changes. In conclusion, implementing an automated feedback-driven educational framework improved radiologist accuracy in detecting suspicious nodules. The study underscores the potential of such systems in enhancing diagnostic performance and reducing errors. Further research and broader implementation are needed to consolidate these promising results and develop effective training strategies for radiologists, ultimately benefiting patient outcomes.