While machine learning models capable of producing uncertainty estimates are becoming widespread, uncalibrated uncertainty estimates are often overconfident, and often assume predetermined probability distributions over the error which do not match the empirical calibration error. Most work on calibrating uncertainty estimates focuses on classification rather than regression, which introduces novel challenges. We present a calibration method referred to as Calibrating Regression Uncertainty Distributions Empirically (CRUDE) that does not assume a fixed uncertainty distribution, instead making the weaker assumption that error distributions have a constant shape across the output space, shifted by predicted mean and scaled by predicted standard deviation. CRUDE requires no training of the calibration estimator aside from a parameter to account for consistent bias in the predicted mean. Our method is distribution-agnostic and provides sharper and more accurate uncertainty estimates than state of the art techniques, as demonstrated by calibration and sharpness measures across many datasets.