Iterative Feature Exclusion Ranking for Deep Tabular Learning

Tabular data is a common format for storing information in rows and columns to represent data entries and their features. Although deep neural networks have become the main approach for modeling a wide range of domains including computer vision and NLP, many of them are not well-suited for tabular data. Recently, a few deep learning models have been proposed for deep tabular learning, featuring an internal feature selection mechanism with end-to-end gradient-based optimization. However, their feature selection mechanisms are unidimensional, and hence fail to account for the contextual dependence of feature importance, potentially overlooking crucial interactions that govern complex tasks. In addition, they overlook the bias of high-impact features and the risk associated with the limitations of attention generalization. To address this limitation, this study proposes a novel iterative feature exclusion module that enhances the feature importance ranking in tabular data. The proposed module iteratively excludes each feature from the input data and computes the attention scores, which represent the impact of the features on the prediction. By aggregating the attention scores from each iteration, the proposed module generates a refined representation of feature importance that captures both global and local interactions between features. The effectiveness of the proposed module is evaluated on four public datasets. The results demonstrate that the proposed module consistently outperforms state-of-the-art methods and baseline models in feature ranking and classification tasks. The code is publicly available at https://github.com/abaraka2020/Iterative-Feature-Exclusion-Ranking-Module and https://github.com/mohalim/IFENet

A Survey on Deep Learning and State-of-the-art Applications

Deep learning, a branch of artificial intelligence, is a computational model that uses multiple layers of interconnected units (neurons) to learn intricate patterns and representations directly from raw input data. Empowered by this learning capability, it has become a powerful tool for solving complex problems and is the core driver of many groundbreaking technologies and innovations. Building a deep learning model is a challenging task due to the algorithm`s complexity and the dynamic nature of real-world problems. Several studies have reviewed deep learning concepts and applications. However, the studies mostly focused on the types of deep learning models and convolutional neural network architectures, offering limited coverage of the state-of-the-art of deep learning models and their applications in solving complex problems across different domains. Therefore, motivated by the limitations, this study aims to comprehensively review the state-of-the-art deep learning models in computer vision, natural language processing, time series analysis and pervasive computing. We highlight the key features of the models and their effectiveness in solving the problems within each domain. Furthermore, this study presents the fundamentals of deep learning, various deep learning model types and prominent convolutional neural network architectures. Finally, challenges and future directions in deep learning research are discussed to offer a broader perspective for future researchers.

WSense: A Robust Feature Learning Module for Lightweight Human Activity Recognition

In recent times, various modules such as squeeze-and-excitation, and others have been proposed to improve the quality of features learned from wearable sensor signals. However, these modules often cause the number of parameters to be large, which is not suitable for building lightweight human activity recognition models which can be easily deployed on end devices. In this research, we propose a feature learning module, termed WSense, which uses two 1D CNN and global max pooling layers to extract similar quality features from wearable sensor data while ignoring the difference in activity recognition models caused by the size of the sliding window. Experiments were carried out using CNN and ConvLSTM feature learning pipelines on a dataset obtained with a single accelerometer (WISDM) and another obtained using the fusion of accelerometers, gyroscopes, and magnetometers (PAMAP2) under various sliding window sizes. A total of nine hundred sixty (960) experiments were conducted to validate the WSense module against baselines and existing methods on the two datasets. The results showed that the WSense module aided pipelines in learning similar quality features and outperformed the baselines and existing models with a minimal and uniform model size across all sliding window segmentations. The code is available at https://github.com/AOige/WSense.