DOES DEEP LEARNING LEARN TO ABSTRACT? A SYSTEMATIC PROBING FRAMEWORK

Abstract

Abstraction is a desirable capability for deep learning models, which means to induce abstract concepts from concrete instances and flexibly apply them beyond the learning context. At the same time, there is a lack of clear understanding about both the presence and further characteristics of this capability in deep learning models. In this paper, we introduce a systematic probing framework to explore the abstraction capability of deep learning models from a transferability perspective. A set of controlled experiments are conducted based on this framework, providing strong evidence that two probed pre-trained language models (PLMs), T5 and GPT2, have the abstraction capability. We also conduct in-depth analysis, thus shedding further light: (1) the whole training phase exhibits a "memorize-thenabstract" two-stage process; (2) the learned abstract concepts are gathered in a few middle-layer attention heads, rather than evenly distributed throughout the model; (3) the probed abstraction capabilities exhibit robustness against concept mutations, and are more robust to low-level/source-side mutations than high-level/target-side ones; (4) generic pre-training is critical to the emergence of abstraction capability, and PLMs exhibit better abstraction with larger model sizes and data scales.

1. INTRODUCTION

Whereas concrete concepts are typically concerned only with things in the world, abstract concepts are about internal events. - Barsalou et al. (1999) Abstraction means capturing the general patterns (often referred to as abstract concepts) efficiently in a specific learning context and reusing these patterns flexibly beyond the context (Mitchell, 2021; Kumar et al., 2022; Giunchiglia & Walsh, 1992; Hull, 1920) . For instance, the abstraction on language means recognizing the underlying syntax and semantics behind concrete sentences. It is thought to be one of the fundamental faculties in human cognition for effectively learning, understanding and robustly generalizing, and has been studied for a long time in cognitive psychology and behavioral sciences (Gentner & Medina, 1998; Barsalou et al., 1999; Shivhare & Kumar, 2016; Konidaris, 2019) . The abstraction capability is also critical for deep learning, but many previous studies suggested that the surprising success of deep learning may come from the memorization of some surface patterns (also called superficial correlations or shortcuts) (Geirhos et al., 2020; Du et al., 2022) , such as some special tokens (Niven & Kao, 2020; Gururangan et al., 2018 ), overlapping contexts (Lai et al., 2021; Sen & Saffari, 2020) , and familiar vocabularies (Aji et al., 2020) . It is still unclear whether the models just memorize these patterns without abstractions, or they do learn abstract concepts (yet overwhelmed by surface patterns when applied in a similar context as in training). Therefore, this paper aims to take a step forward to probe the abstraction capability of deep learning models, keeping the effects of abstract concepts and surface patterns decoupled and controlled individually. Our key idea is to probe the abstraction capability from a transferability perspective, since surface patterns are always bounded with task-specific characteristics while abstract concepts can be more generally reused. We consider designing multiple tasks with shared abstract concepts and totally different surface patterns, then tracing whether the learning on one task can boost the performance on another. Figure 1 demonstrates a motivating example. Motivating Example As shown in Figure 1 , suppose we want to examine whether a model can learn the abstract rule (i.e., the symbolic mapping rule x 1 x 2 → X 1 X 2 , in which x i and X i are general variable slots) from the task A, or just memorize surface maps (e.g., ab → AB, in which a and A are task-specific symbols). To reveal the different transferability of two learning mechanisms, we utilize a probing task B that contains the same underlying abstract rule as task A but does not overlap with its symbol set. If the model could learn the abstract rule from task A, it would reuse it to interpret new context, thus effectively solving task B. But if not, memorizing some surface maps that are bounded with task-specific symbols is less effective to solve task B. Motivated by this example, we design a systematic framework for probing abstraction capability. This framework considers a set of probing tasks along with three procedures of experiments based on the transfer learning paradigm. The use of abstract concepts and task-specific characteristics in probing tasks are separately controlled. To probe the abstraction capability of language models, this work mainly considers grammar as the abstract conceptfoot_0 . The grammar of a formal language is a set of hidden rules behind concrete sentences and determines how terminals are combined into sentences that are valid to the syntax. We want to explore whether the model can be aware of the grammar, or simply memorize some specific word combinations. We instantiate our framework as a grammar probe that is constructed from the designed formal grammar and terminal sets. The probing results show strong evidence that two probed PLMs (specifically, T5-Base (Raffel et al., 2020) and GPT2-Medium (Radford et al., 2019) ) have the abstraction capability to learn abstract concepts from concrete instances, rather than just simply memorizing surface patterns. After probing the existence of abstraction capability, we further explore the following questions. RQ1: What is the characteristic of the training dynamics on learning abstract concepts? RQ2: How are these learned abstract concepts distributed in the model? Concentrated in a few modules or evenly distributed in whole model? RQ3: How robust is the abstraction capability on tasks with mutated abstract concepts? RQ4: How would generic pre-training and general factors influence abstraction? Here we outline some interesting findings from our in-depth investigations: (1) the training phase exhibits a "memorize-then-abstract" two-stage process; (2) the abstract concepts learned in our probes are gathered in a few middle-layer heads; (3) abstraction capability is more robust to source-side/low-level mutations than to target-side/high-level ones; (4) generic pre-training is critical to the emergence of abstraction capability, and larger model size and data scale are beneficial. Contributions 1) We propose a systematic probing framework for abstraction capability, guiding the design of controlled tasks and procedures from a transferability perspective. 2) We instantiate this framework with concrete tasks and show strong evidence that two probed PLMs have the abstraction capability. 3) We further analyze this capability and provide insightful conclusions by investigating the above research questions. Our code and data are publicly available at https://github.com/microsoft/ContextualSP/tree/master/abstraction_probing.



We also probed other abstract concepts such as operation semantics in Appendix D.



Motivating example: the abstract concepts learned in task A can be effectively reused in task B, but surface patterns are useless. Unused patterns or concepts are whitened after the update.

