UTC-IE: A UNIFIED TOKEN-PAIR CLASSIFICATION ARCHITECTURE FOR INFORMATION EXTRACTION

Abstract

Information Extraction (IE) spans several tasks with different output structures, such as named entity recognition, relation extraction and event extraction. Previously, those tasks were solved with different models because of diverse task output structures. Through re-examining IE tasks, we find that all of them can be interpreted as extracting spans and span relations. We propose using the start and end token of a span to pinpoint the span in texts, and using the start-to-start and end-to-end token pairs of two spans to determine the relation. Hence, we can unify all IE tasks under the same token-pair classification formulation. Based on the reformulation, we propose a Unified Token-pair Classification architecture for Information Extraction (UTC-IE), where we introduce Plusformer on top of the token-pair feature matrix. Specifically, it models axis-aware interaction with plusshaped self-attention and local interaction with Convolutional Neural Network over token pairs. Experiments show that our approach outperforms task-specific and unified models on all tasks in 10 datasets, and achieves better or comparable results on 2 joint IE datasets. Moreover, UTC-IE speeds up over state-of-the-art models on IE tasks significantly in most datasets, which verifies the effectiveness of our architecture.

1. INTRODUCTION

Information Extraction (IE) aims to identify and classify structured information from unstructured texts (Andersen et al., 1992; Grishman, 2019) . IE consists of a wide range of tasks, such as named entity recognition (NER), joint entity relation extraction (RE)foot_0 and event extraction (EE)foot_1 . In the last decade, many paradigms have been proposed to solve IE tasks, such as sequence labeling (McCallum & Li, 2003; Huang et al., 2015; Zheng et al., 2017; Yu et al., 2020a ), span-based classification (Jiang et al., 2020; Yu et al., 2020b; Wang et al., 2021; Ye et al., 2022) , MRC-based methods (Levy et al., 2017; Li et al., 2020; Liu et al., 2020) and generation-based methods (Zeng et al., 2018; Yan et al., 2021a; Hsu et al., 2022) . The above work mainly concentrates on solving individual tasks, but it is desired to have a unified model to solve all IE tasks without designing dedicated modules. Besides, tackling all IE tasks with one model can facilitate knowledge sharing between different tasks. Therefore, various attempts have been made to unify all IE tasks with one model structure. Wadden et al. ( 2019 



Joint entity relation extraction aims to extract both entities and relations. In our paper, we call it relation extraction (RE) for simplicity. Event extraction covers trigger extraction and argument extraction, where we first conduct argument span detection and then conduct argument role classification in our architecture.



); Lin et al. (2020); Nguyen et al. (2021) encode all IE tasks' target structure as graphs and design graph-based methods to predict them; Paolini et al. (2021); Lu et al. (2022) solve general IE tasks in a generative way with a text-to-text or text-to-structure framework. However, graph-based models tend to be complex to design, and generative models are time-consuming to decode. In our work, we creatively propose a simple yet effective paradigm for unified IE. Inspired by Jiang et al. (2020), we re-examine IE tasks and consider that all of them are fundamentally span extraction (entity extraction in NER and RE, trigger classification and argument span detection in EE)

