Many cardiac diseases are associated with structural remodeling of the myocardium. Cardiac magnetic resonance (CMR) imaging with contrast enhancement, such as late gadolinium enhancement (LGE), has unparalleled capability to visualize fibrotic tissue remodeling, allowing for direct characterization of the pathophysiological abnormalities leading to arrhythmias and sudden cardiac death (SCD). Automating segmentation of the ventricles with fibrosis distribution could dramatically enhance the utility of LGE-CMR in heart disease clinical research and in the management of patients with risk of arrhythmias and SCD. Here we describe an anatomically-informed deep learning (DL) approach to myocardium and scar segmentation and clinical feature extraction from LGE-CMR images. The technology enables clinical use by ensuring anatomical accuracy and complete automation. Algorithm performance is strong for both myocardium segmentation ($98\%$ accuracy and $0.79$ Dice score in a hold-out test set) and evaluation measures shown to correlate with heart disease, such as scar amount ($6.3\%$ relative error). Our approach for clinical feature extraction, which satisfies highly complex geometric constraints without stunting the learning process, has the potential of a broad applicability in computer vision beyond cardiology, and even outside of medicine.