Explainable Machine Learning: The Importance of a System-Centric Perspective

The landscape in the context of several signal processing applications and even education appears to be significantly affected by the emergence of machine learning (ML) and in particular deep learning (DL).The main reason for this is the ability of DL to model complex and unknown relationships between signals and the tasks of interest. Particularly, supervised DL algorithms have been fairly successful at recognizing perceptually or semantically useful signal information in different applications. In all of these, the training process uses labeled data to learn a mapping function (typically implicitly) from signals to the desired information (class label or target label). The trained DL model is then expected to correctly recognize/classify relevant information in a given test signal. A DL based framework is therefore, in general, very appealing since the features and characteristics of the required mapping are learned almost exclusively from the data without resorting to explicit model/system development. The focus on implicit modeling however also raises the issue of lack of explainability/interpretability of the resultant DL based mapping or the black box problem. As a result, explainable ML/DL is an active research area where the primary goal is to elaborate how the ML/DL model arrived at a prediction. We however note that despite the efforts, the commentary on black box problem appears to lack a technical discussion from the view point of: a) its origin and underlying reasons, and b) its practical implications on the design and deployment of ML/DL systems. Accordingly, a reasonable question that can be raised is as follows. Can the traditional system-centric approach (which places emphasis on explicit system modeling) provide useful insights into the nature of black box problem, and help develop more transparent ML/DL systems?

Does Explainable Machine Learning Uncover the Black Box in Vision Applications?

Machine learning (ML) in general and deep learning (DL) in particular has become an extremely popular tool in several vision applications (like object detection, super resolution, segmentation, object tracking etc.). Almost in parallel, the issue of explainability in ML (i.e. the ability to explain/elaborate the way a trained ML model arrived at its decision) in vision has also received fairly significant attention from various quarters. However, we argue that the current philosophy behind explainable ML suffers from certain limitations, and the resulting explanations may not meaningfully uncover black box ML models. To elaborate our assertion, we first raise a few fundamental questions which have not been adequately discussed in the corresponding literature. We also provide perspectives on how explainablity in ML can benefit by relying on more rigorous principles in the related areas.