A deep neural network predicts survival after heart imaging better than cardiologists

Predicting future clinical events, such as death, is an important task in medicine that helps physicians guide appropriate action. Neural networks have particular promise to assist with medical prediction tasks related to clinical imaging by learning patterns from large datasets. Significant advances have been made in predicting complex diagnoses from medical imaging[1-5]. Predicting future events, then, is a natural but relatively unexplored extension of those efforts. Moreover, neural networks have not yet been applied to medical videos on a large scale, such as ultrasound of the heart (echocardiography). Here we show that a large dataset of 723,754 clinically-acquired echocardiographic videos (approx. 45 million images) linked to longitudinal follow-up data in 27,028 patients can be used to train a deep neural network to predict 1-year survival with good accuracy. We also demonstrate that prediction accuracy can be further improved by adding highly predictive clinical variables from the electronic health record. Finally, in a blinded, independent test set, the trained neural network was more accurate in discriminating 1-year survival outcomes than two expert cardiologists. These results therefore highlight the potential of neural networks to add new predictive power to clinical image interpretations.

Improving Classification Rate of Schizophrenia Using a Multimodal Multi-Layer Perceptron Model with Structural and Functional MR

The wide variety of brain imaging technologies allows us to exploit information inherent to different data modalities. The richness of multimodal datasets may increase predictive power and reveal latent variables that otherwise would have not been found. However, the analysis of multimodal data is often conducted by assuming linear interactions which impact the accuracy of the results. We propose the use of a multimodal multi-layer perceptron model to enhance the predictive power of structural and functional magnetic resonance imaging (sMRI and fMRI) combined. We also use a synthetic data generator to pre-train each modality input layers, alleviating the effects of the small sample size that is often the case for brain imaging modalities. The proposed model improved the average and uncertainty of the area under the ROC curve to 0.850+-0.051 compared to the best results on individual modalities (0.741+-0.075 for sMRI, and 0.833+-0.050 for fMRI).