Fine-Tuning the Retrieval Mechanism for Tabular Deep Learning

While interests in tabular deep learning has significantly grown, conventional tree-based models still outperform deep learning methods. To narrow this performance gap, we explore the innovative retrieval mechanism, a methodology that allows neural networks to refer to other data points while making predictions. Our experiments reveal that retrieval-based training, especially when fine-tuning the pretrained TabPFN model, notably surpasses existing methods. Moreover, the extensive pretraining plays a crucial role to enhance the performance of the model. These insights imply that blending the retrieval mechanism with pretraining and transfer learning schemes offers considerable potential for advancing the field of tabular deep learning.

Lifelog Patterns Analyzation using Graph Embedding based on Deep Neural Network

Recently, as the spread of smart devices increases, the amount of data collected through sensors is increasing. A lifelog is a kind of big data to analyze behavior patterns in the daily life of individuals collected from various smart de-vices. However, sensor data is a low-level signal that makes it difficult for hu-mans to recognize the situation directly and cannot express relations clearly. It is also difficult to identify the daily behavior pattern because it records heterogene-ous data by various sensors. In this paper, we propose a method to define a graph structure with node and edge and to extract the daily behavior pattern from the generated lifelog graph. We use the graph convolution method to embeds the lifelog graph and maps it to low dimension. The graph convolution layer im-proves the expressive power of the daily behavior pattern by implanting the life-log graph in the non-Euclidean space and learns the patterns of graphs. Experi-mental results show that the proposed method automatically extracts meaningful user patterns from UbiqLog dataset. In addition, we confirm the usefulness by comparing our method with existing methods to evaluate performance.