LEARNING AXIOMS TO COMPUTE VERIFIABLE SYM-BOLIC EXPRESSION EQUIVALENCE PROOFS USING GRAPH-TO-SEQUENCE NETWORKS Anonymous

Abstract

We target the problem of proving semantic equivalence between two complex expressions represented as typed trees, and demonstrate our system on expressions from a rich multi-type symbolic language for linear algebra. We propose the first graph-to-sequence deep learning system to generate axiomatic proofs of equivalence between program pairs. We generate expressions which include scalars, vectors and matrices and 16 distinct operators combining them, with 147 distinct axioms of equivalence. We study the robustness of the system to generate proofs of increasing length, demonstrating how incremental graph-to-sequence networks can learn to represent complex and verifiable symbolic reasoning. It achieves 93% average true positive coverage on 10,000 test cases while ensuring zero false positives by design.

1. INTRODUCTION

Deep neural networks have excelled at a variety of classification and reinforcement learning tasks (Goodfellow et al., 2016) . However, their stochastic nature tends to hinder their ability to learn representations useful for manipulating symbolic information. For applications requiring a guarantee of correctness, such as occur in relation to determining the semantic equivalence between two programs (or symbolic expressions), a system that produces provably correct output must be developed. In this work we target the problem of automatically computing whether two input programs are semantically equivalent (Kaplan, 1969) , under a well-defined axiomatic system for equivalence using semantics-preserving rewrite rules (Dershowitz, 1985) . Program equivalence is summarized as determining whether two programs would always produce the same outputs for all possible inputs, and is a central problem in computing (Kaplan, 1969; Godlin & Strichman, 2008; Verdoolaege et al., 2009) . The problem ranges from undecidable, e.g. Goldblatt & Jackson (2012) , to trivial in cases of testing the equivalence of a program with itself. Our work directly studies the subset of programs represented by symbolic linear algebra expressions which include scalar, vector, and matrix types for both constants and variables, and 16 different operators with 147 distinct axioms of equivalence, creating a foundation for future work with broader program semantics. For example, the expression using matrices, scalars, and a vector: (A + B)I((a + (b -b))/a) v -A v can be proven equivalent to B v by applying 10 axioms in sequence; our work generates the proof steps between these expressions. While prior work has shown promise for deep networks to compute some forms of program equivalence (Xu et al., 2017; Alon et al., 2019; Rabin et al., 2020) , the system typically outputs only a probability of equivalence, without any reasoning or insight that can be verified easily: false positives can be produced. Programs can be represented as a tree (or graph) of symbols, and deep networks for symbolic reasoning have been studied, e.g. to compute the derivative of a symbolic expression (Lample & Charton, 2020), but such systems can produce output which is not directly explained by the system itself. In this work, we take a fundamentally different approach to the problem of symbolic program reasoning with deep networks: we make the system produce the sequence of steps that lead to rewriting one program into another, that is the reasoning for (or proof of) equivalence between the two programs, instead of producing directly the result of this reasoning (e.g., the transformed expression (Lample & Charton, 2020)). We propose a method for generating training samples using probabilistic applications of production rules within a formal grammar, and then develop a graph-to-sequence (Li et al., 2016; Beck et al., 2018) neural network system for program equivalence, trained to learn and combine rewrite rules to rewrite one program into another. It can deterministically prove equivalence, entirely avoids false positives, and quickly invalidates incorrect answers produced by the network (no deterministic answer is provided in this case, only a probability of non-equivalence). In a nutshell, we develop the first graph-to-sequence neural network system to accelerate the search in the space of possible combinations of transformation rules (i.e., axioms of equivalence in the input language) to make two graphs representing programs structurally identical without violating their original semantics. We make the following contributions: (i) We propose a machine learning system for program equivalence which ensures correctness for all non-equivalent programs input, and a deterministically checkable output for equivalent programs (no false positives, specificity = 100%). (ii) We introduce pe-graph2axiom, the first incremental graph-to-sequence neural network system targeting program equivalence to the best of our knowledge. We provide the first implementation of such graph-to-sequence systems in the popular OpenNMT-py framework (Klein et al., 2017) . (iii) We present a complete implementation of our system operating on a rich language for multitype linear algebra expressions. Our system provides a correct rewrite rule sequence between two equivalent programs for 93% of the 10,000 test cases. The correctness of the rewrite rule is deterministically checkable in all cases in negligible time.

2. PROGRAM EQUIVALENCE USING DEEP LEARNING

We outline below the key intuitions and concepts for our program equivalence system. Extensive details, including a full formal framework for program equivalence reasoning, are presented in supplementary material, Secs. A-B. We illustrate below four very simple computations, represented as graphs, that are all equivalent under various axioms of natural arithmetic. Rewrite rules as axioms of equivalence In this work we represent programs with symbolic expressions made of variables (e.g., a, b, c), operators (e.g., +, * ) and neutral/absorbing elements (e.g., 1). We consider a rich linear algebra expression language, supporting three variable types (scalars as shown in P1-P4, vectors, and matrices) and 5 different variables per type; 16 operators including operators mixing different variable types such as vector-matrix product. We represent these programs as dataflow graphs Buck & Lee (1993) with a root node that is to compute a single value. P1 is equivalent to P2 if we consider the axiom A1 : 1 N * x = x, ∀x ∈ N. This axiom is also a clear rewrite rule: the LHS expression 1 N * x (with x ∈ N) can be matched and replaced by the RHS expression x anywhere in the program without altering its semantics. An axiom, or equivalently here a graph rewrite rule, may be applied repeatedly to different subtrees. When applying A1 on a specific location, the node a of P 1, we obtain an equivalent and yet syntactically different program, we note P 1 ≡ A1(a, P 1). To assess the validity of a transformation sequence S where P 2 = S(P 1), one simply needs to check for S, in sequence, that each axiom is applicable at that program point, apply it to obtain a new temporary program, and repeat the process for each axiom in the complete sequence. If the sequence is verified to be valid, and S(P 1) is structurally equivalent to P 2, then we have proved P 1 ≡ P 2, and S forms the complete proof of equivalence between the two programs. These equivalences can be composed, incrementally, to form a complex transformation. Using A2 : x * (y + z) = x * y + x * z, ∀x, y, z ∈ N and A3 : x + y = y + x, ∀x, y ∈ N, we have P 1 ≡ P 4 ≡ A3(+, A2( * , A1(a, P 1))), a verifiable proof of equivalence under our axioms between the programs 1a(b + c) and ac + ab, which involved structural changes including node deletion, creation and edge modification. Note the non-unicity of a sequence: by possibly many ways a program can be rewritten into another one, for example the sequence P 4 ≡ A2( * , A3(+, A1(a, P 1))) also correctly rewrites P 1 into P 4. Conversely, a sequence may not exist: for example no sequence of the 3 above axioms allows rewriting a + b into a * b. We call these non-equivalent in our system, that is precisely if there is no sequence of axioms that can be applied to rewrite one program into the other.



* c + a * b

