Funny-Valen-Tine: Planning Solution Distribution Enhances Machine Abstract Reasoning Ability

Visual abstract reasoning problems hold immense importance in the field of image processing. Both Bongard-Logo and Raven's Progressive Matrices (RPM) belong to this domain, with Bongard-Logo categorized as image clustering reasoning and RPM involving image progression pattern reasoning. This paper introduces Valen, a novel baseline model under probabilistic highlighting models. Valen exhibits remarkable performance in solving both RPM and Bongard-Logo problems, offering a versatile solution. Our investigation delves into the underlying mechanisms of probability-highlighting solvers, realizing they approximate solutions to reasoning problem instances as distributions delineated by primary and auxiliary samples. We propose that the learning objective is not the distribution of correct solutions but one defined by both primary and auxiliary samples. To bridge discrepancies, we introduced the Tine method, an adversarial learning-based approach to assist Valen in estimating a solution distribution closer to the correct one, albeit with issues like unstable training. Reflecting on Tine, we propose modeling the sample distribution of reasoning problems as a mixture of Gaussian distributions, leading to the Funny method. This effectively enables Valen to capture the true form of the correct solution distribution. Furthermore, we designed the SBR method to model the distribution of progressive patterns representation similarly. Overall, the Funny, Tine, and SBR methods significantly improve Valen's performance, providing new ideas and methods for studying visual abstract reasoning problems.

Funny Valen-Tine: Solving visual abstract reasoning problems through defining the solution distribution

Visual abstract reasoning problems hold immense importance in the field of image processing. Both Bongard-Logo and Raven's Progressive Matrices (RPM) belong to this domain, with Bongard-Logo categorized as image clustering reasoning and RPM involving image progression pattern reasoning. This paper introduces Valen, a novel baseline model under probabilistic highlighting models. Valen exhibits remarkable performance in solving both RPM and Bongard-Logo problems, offering a versatile solution. Our investigation delves into the underlying mechanisms of probability-highlighting solvers, realizing they approximate solutions to reasoning problem instances as distributions delineated by primary and auxiliary samples. We propose that the learning objective is not the distribution of correct solutions but one defined by both primary and auxiliary samples. To bridge discrepancies, we introduced the Tine method, an adversarial learning-based approach to assist Valen in estimating a solution distribution closer to the correct one, albeit with issues like unstable training. Reflecting on Tine, we propose modeling the sample distribution of reasoning problems as a mixture of Gaussian distributions, leading to the Funny method. This effectively enables Valen to capture the true form of the correct solution distribution. Furthermore, we designed the SBR method to model the distribution of progressive patterns representation similarly. Overall, the Funny, Tine, and SBR methods significantly improve Valen's performance, providing new ideas and methods for studying visual abstract reasoning problems.

Solving the bongard-logo problem by modeling a probabilistic model

Abstract reasoning problems challenge the perceptual and cognitive abilities of AI algorithms, demanding deeper pattern discernment and inductive reasoning beyond explicit image features. This study introduces PMoC, a tailored probability model for the Bongard-Logo problem, achieving high reasoning accuracy by constructing independent probability models. Additionally, we present Pose-Transformer, an enhanced Transformer-Encoder designed for complex abstract reasoning tasks, including Bongard-Logo, RAVEN, I-RAVEN, and PGM. Pose-Transformer incorporates positional information learning, inspired by capsule networks' pose matrices, enhancing its focus on local positional relationships in image data processing. When integrated with PMoC, it further improves reasoning accuracy. Our approach effectively addresses reasoning difficulties associated with abstract entities' positional changes, outperforming previous models on the OIG, D3$\times$3 subsets of RAVEN, and PGM databases. This research contributes to advancing AI's capabilities in abstract reasoning and cognitive pattern recognition.

D4C glove-train: solving the RPM and Bongard-logo problem by distributing and Circumscribing concepts

This paper achieves significant progress in the field of abstract reasoning, particularly in addressing Raven's Progressive Matrices (RPM) and Bongard-Logo problems. We propose the D2C approach, which redefines conceptual boundaries in these domains and bridges the gap between high-level concepts and their low-dimensional representations. Based on this, we further introduce the D3C method that handles Bongard-Logo problems and significantly improves reasoning accuracy by estimating the distribution of image representations and measuring their Sinkhorn distance. To enhance computational efficiency, we introduce the D3C-cos variant, which provides an efficient and accurate solution for RPM problems by constraining distribution distances. Additionally, we present Lico-Net, a network that combines D3C and D3C-cos to achieve state-of-the-art performance in both problem-solving and interpretability. Finally, we extend our approach to D4C, employing adversarial strategies to further refine conceptual boundaries and demonstrate notable improvements for both RPM and Bongard-Logo problems. Overall, our contributions offer a new perspective and practical solutions to the field of abstract reasoning.

Triple-CFN: Restructuring Conceptual Spaces for Enhancing Abstract Reasoning process

Abstract reasoning problems pose significant challenges to artificial intelligence algorithms, demanding cognitive capabilities beyond those required for perception tasks. This study introduces the Triple-CFN approach to tackle the Bongard-Logo problem, achieving notable reasoning accuracy by implicitly reorganizing the concept space of conflicting instances. Additionally, the Triple-CFN paradigm proves effective for the RPM problem with necessary modifications, yielding competitive results. To further enhance performance on the RPM issue, we develop the Meta Triple-CFN network, which explicitly structures the problem space while maintaining interpretability on progressive patterns. The success of Meta Triple-CFN is attributed to its paradigm of modeling the conceptual space, equivalent to normalizing reasoning information. Based on this ideology, we introduce the Re-space layer, enhancing the performance of both Meta Triple-CFN and Triple-CFN. This paper aims to contribute to advancements in machine intelligence by exploring innovative network designs for addressing abstract reasoning problems, paving the way for further breakthroughs in this domain.

Multi-Viewpoint and Multi-Evaluation with Felicitous Inductive Bias Boost Machine Abstract Reasoning Ability

Great endeavors have been made to study AI's ability in abstract reasoning, along with which different versions of RAVEN's progressive matrices (RPM) are proposed as benchmarks. Previous works give inkling that without sophisticated design or extra meta-data containing semantic information, neural networks may still be indecisive in making decisions regarding RPM problems, after relentless training. Evidenced by thorough experiments and ablation studies, we showcase that end-to-end neural networks embodied with felicitous inductive bias, intentionally design or serendipitously match, can solve RPM problems elegantly, without the augment of any extra meta-data or preferences of any specific backbone. Our work also reveals that multi-viewpoint with multi-evaluation is a key learning strategy for successful reasoning. Finally, potential explanations for the failure of connectionist models in generalization are provided. We hope that these results will serve as inspections of AI's ability beyond perception and toward abstract reasoning. Source code can be found in https://github.com/QinglaiWeiCASIA/RavenSolver.

EiHi Net: Out-of-Distribution Generalization Paradigm

This paper develops a new EiHi net to solve the out-of-distribution (OoD) generalization problem in deep learning. EiHi net is a model learning paradigm that can be blessed on any visual backbone. This paradigm can change the previous learning method of the deep model, namely find out correlations between inductive sample features and corresponding categories, which suffers from pseudo correlations between indecisive features and labels. We fuse SimCLR and VIC-Reg via explicitly and dynamically establishing the original - positive - negative sample pair as a minimal learning element, the deep model iteratively establishes a relationship close to the causal one between features and labels, while suppressing pseudo correlations. To further validate the proposed model, and strengthen the established causal relationships, we develop a human-in-the-loop strategy, with few guidance samples, to prune the representation space directly. Finally, it is shown that the developed EiHi net makes significant improvements in the most difficult and typical OoD dataset Nico, compared with the current SOTA results, without any domain ($e.g.$ background, irrelevant features) information.