A Generalized Variable Importance Metric and Estimator for Black Box Machine Learning Models

The aim of this study is to define importance of predictors for black box machine learning methods, where the prediction function can be highly non-additive and cannot be represented by statistical parameters. In this paper we defined a ``Generalized Variable Importance Metric (GVIM)'' using the true conditional expectation function for a continuous or a binary response variable. We further showed that the defined GVIM can be represented as a function of the Conditional Average Treatment Effect (CATE) squared for multinomial and continuous predictors. Then we propose how the metric can be estimated using using any machine learning models. Finally we showed the properties of the estimator using multiple simulations.