EFFECTIVE ABSTRACT REASONING WITH DUAL-CONTRAST NETWORK

Abstract

As a step towards improving the abstract reasoning capability of machines, we aim to solve Raven's Progressive Matrices (RPM) with neural networks, since solving RPM puzzles is highly correlated with human intelligence. Unlike previous methods that use auxiliary annotations or assume hidden rules to produce appropriate feature representation, we only use the ground truth answer of each question for model learning, aiming for an intelligent agent to have a strong learning capability with a small amount of supervision. Based on the RPM problem formulation, the correct answer filled into the missing entry of the third row/column has to best satisfy the same rules shared between the first two rows/columns. Thus we design a simple yet effective Dual-Contrast Network (DCNet) to exploit the inherent structure of RPM puzzles. Specifically, a rule contrast module is designed to compare the latent rules between the filled row/column and the first two rows/columns; a choice contrast module is designed to increase the relative differences between candidate choices. Experimental results on the RAVEN and PGM datasets show that DCNet outperforms the state-of-the-art methods by a large margin of 5.77%. Further experiments on few training samples and model generalization also show the effectiveness of DCNet. Code is available at https://github.com/visiontao/dcnet.

1. INTRODUCTION

Abstract reasoning capability is a critical component of human intelligence, which relates to the ability of understanding and interpreting patterns, and further solving problems. Recently, as a step towards improving the abstract reasoning ability of machines, many methods (Santoro et al., 2018; Zhang et al., 2019a; b; Zheng et al., 2019; Zhuo & Kankanhalli, 2020) are developed to solve Raven's Progress Matrices (RPM) (Domino & Domino, 2006; Raven & Court, 1938) , since it is widely believed that RPM lies at the heart of human intelligence. As the example shown in Figure 1 , given a 3 × 3 problem matrix with a final missing piece, the test taker has to find the logical rules shared between the first two rows or columns, and then pick the correct answer from 8 candidate choices to best complete the matrix. Since the logical rules hidden in RPM questions are complex and unknown, solving RPM with machines remains a challenging task. As described in (Carpenter et al., 1990) , the logical rules applied in a RPM question are manifested as visual structures. For a single image in the question, the logical rules could consist of several basic attributes, e.g., shape, color, size, number, and position. For the images in a row or column, the logical rules could be applied row-wise or column-wise and formulated with an unknown relationship, e.g., AND, OR, XOR, and so on (Santoro et al., 2018; Zhang et al., 2019a) . If we can extract the explicit rules of each question, the problem can be easily solved by using a heuristicsbased search method (Zhang et al., 2019a) . However, given an arbitrary RPM question, the logical rules are unknown. What's worse -even the number of rules is unknown. As a result, an intelligent machine needs to simultaneously learn the representation of these hidden rules and find the correct answer to satisfy all of the applied rules. With the success of deep learning in computer vision, solving RPM puzzles with neural networks has become popular. Because the learned features might be inconsistent with the logical rules, many supervised learning methods, e.g., DRT (Zhang et al., 2019a) , WReN (Santoro et al., 2018) and 

