GN-TRANSFORMER: FUSING AST AND SOURCE CODE INFORMATION IN GRAPH NETWORKS

Abstract

As opposed to natural languages, source code understanding is influenced by grammar relations between tokens regardless of their identifier name. Considering graph representation of source code such as Abstract Syntax Tree (AST) and Control Flow Graph (CFG), can capture a token's grammatical relationships that are not obvious from the source code. Most existing methods are late fusion and underperform when supplementing the source code text with a graph representation. We propose a novel method called GN-Transformer to fuse representations learned from graph and text modalities under the Graph Networks (GN) framework with attention mechanism. Our method learns the embedding on a constructed graph called Syntax-Code Graph (SCG). We perform experiments on the structure of SCG, an ablation study on the model design and the hyperparamaters to conclude that the performance advantage is from the fusion method and not the specific details of the model. The proposed method achieved state of the art performance in two code summarization datasets and across three metrics.

1. INTRODUCTION

Code summarization is the task of generating a readable summary that describes the functionality of a snippet. Such task requires a high-level comprehension of a source code snippet thus it is an effective task to evaluate whether a Deep Learning Model is able to capture complex relations and structures inside code. Programming languages are context-free formal language, an unambiguous representation, Abstract Syntax Tree (AST), could be derived from a source code snippet. A parse tree based representation of code is precise and without noise. An AST accurately describes the structure of a snippet and relationships between tokens which provides valuable supplementary information for code understanding. 



Figure 1: Examples of generated summarizations on Java (red blocks) and Python (blue blocks) test set. Transformer is a vanilla Transformer, Transformer (full) implements relative positional encoding and copy mechanism from Ahmad et al. (2020).

