Deep neural networks enable learning directly on the data without the domain knowledge needed to construct a feature set. This approach has been extremely successful in almost all machine learning applications. We propose a new framework that also learns directly from the data, without extracting a feature set. We apply this framework to detecting bursts in the EEG of premature infants. The EEG is recorded within days of birth in a cohort of infants without significant brain injury and born <30 weeks of gestation. The method first transforms the time-domain signal to the time--frequency domain and then trains a machine learning method, a gradient boosting machine, on each time-slice of the time--frequency distribution. We control for oversampling the time--frequency distribution with a significant reduction (<1%) in memory and computational complexity. The proposed method achieves similar accuracy to an existing multi-feature approach: area under the characteristic curve of 0.98 (with 95% confidence interval of 0.96 to 0.99), with a median sensitivity of 95% and median specificity of 94%. The proposed framework presents an accurate, simple, and computational efficient implementation as an alternative to both the deep learning approach and to the manual generation of a feature set.