LEARNING EXPLANATIONS THAT ARE HARD TO VARY

Abstract

In this paper, we investigate the principle that good explanations are hard to vary in the context of deep learning. We show that averaging gradients across examples -akin to a logical OR (_) of patterns -can favor memorization and 'patchwork' solutions that sew together different strategies, instead of identifying invariances. To inspect this, we first formalize a notion of consistency for minima of the loss surface, which measures to what extent a minimum appears only when examples are pooled. We then propose and experimentally validate a simple alternative algorithm based on a logical AND (^), that focuses on invariances and prevents memorization in a set of real-world tasks. Finally, using a synthetic dataset with a clear distinction between invariant and spurious mechanisms, we dissect learning signals and compare this approach to well-established regularizers.



Consider the top of Figure 1 , which shows a view from above of the loss surface obtained as we vary a two dimensional parameter vector θ " pθ 1 , θ 2 q, for a fictional dataset containing two observations x A and x B . Note the two global minima on the top-right and bottom-left. Depending on the initial values of θ -marked as white circles -gradient descent converges to one of the two minima. Judging solely by the value of the loss function, which is zero in both cases, the two minima look equally good. However, looking at the loss surfaces for x A and x B separately, as shown below, a crucial difference between those two minima appears: Starting from the same initial parameter configurations and following the gradient of the loss, ∇ θ Lpθ, x i q, the probability of finding the same minimum on the top-right in either case is zero. In contrast, the minimum in the lower-left corner has a significant overlap across the two loss surfaces, so gradient descent can converge to it even if training on x A (or x B ) only. Note that after averaging there is no way to tell what the two loss surfaces looked like: Are we destroying information that is potentially important? In this paper, we argue that the answer is yes. In particular, we hypothesize that if the goal is to find invariant mechanisms in the data, these can be identified by finding explanations (e.g. model parameters) that are hard to vary across examples. A notion of invariance implies something that stays the same, as something else changes. We assume that data comes from different environments: Using the notion of consistency, we define Invariant Learning Consistency (ILC), a measure of the expected consistency of the solution found by a learning algorithm on a given hypothesis class. The ILC can be improved by changing the hypothesis class or the learning algorithm, and in the last part of the paper we focus on the latter. We then analyse why current practices in deep learning provide little incentive for networks to learn invariances, and show that standard training is instead set up with the explicit objective of greedily maximizing speed of learning, i.e., progress on the training loss. When learning "as fast as possible" is not the main objective, we show we can trade-off some "learning speed" for prioritizing learning the invariances  295 8 0Z0Z0Z 0Z 7 Z0Z0Zp Z0 6 pZ0Z0Z pZ 5 jpZ0ZP Z0 4 0Z0ZnZ PA 3 OPZ0Z0 Z0 2 0ZKZ0Z 0Z 1 Z0Z0Z0 Z0 a b c d e f g h 296 8 0Z0Z0Z 0Z 7 snZ0Z0 o0 6 0j0Z0Z PZ 5 o0ZPZ0 Z0 4 Po0Z0M 0Z 3 ZKZ0Z0 Z0 2 0ZRZ0Z 0Z 1 Z0Z0Z0 Z0 a b c d e f g h 297 8 0Z0Z0Z 0Z 7 Z0Zns0 Z0 6 BZ0ako 0Z 5 Z0Z0o0 Z0 4 0O0Z0Z PZ 3 Z0Z0Z0 Z0 2 0ZKZNZ 0Z 1 Z0Z0Z0 ZR a b c d e f g h 298 8 0Z0Z0Z 0Z 7 ZRZ0Z0 Z0 6 0Z0ako 0Z 5 Z0Z0on Z0 4 0Z0oKZ PZ 3 Z0ZPZ0 Z0 2 0Z0Z0Z 0Z 1 Z0Z0Z0 Z0 a b c d e f g h 299 8 rZ0l0j 0s 7 opZ0a0 op 6 0Z0o0m 0Z 5 Z0Z0o0 M0 4 0Z0ZPZ bZ 3 OQM0A0 Z0 2 0O0Z0O PO 1 Z0ZRS0 J0 a b c d e f g h 300 8 0ZrZ0j 0s 7 obl0op Zp 6 0onZ0M 0Z 5 Z0m0Z0 A0 4 0Z0o0Z 0Z 3 ZPZPZP O0 2 PZ0MQZ 0O 1 ZKZ0S0 ZR a b c d e f g h 5334 Problems, Combinations & Games 1.1 Mate in 1 301 8 rZbZkZ 0s 7 ZpopZ0 Zp 6 pZ0ZpL 0Z 5 ZNO0Z0 Z0 4 0OKZ0Z 0Z 3 O0Z0On ZP 2 0Z0Z0O 0Z 1 Z0Z0Z0 Z0 a b c d e f g h 302 8 0Z0ZkZ 0Z 7 Z0Z0Op Z0 6 0Z0O0Z 0Z 5 Z0Z0Z0 Z0 4 0Z0Z0Z 0Z 3 Z0Z0ZQ Z0 2 0J0Z0Z 0Z 1 Z0Z0l0 Z0 a b c d e f g h 303 8 0Z0ZkZ 0Z 7 Z0Z0Z0 Z0 6 0Z0Z0O NZ 5 Z0ZQZ0 Z0 4 0Z0Z0Z 0Z 3 Z0Z0Zp Z0 2 0Z0ZnZ 0J 1 Z0Z0Zq Z0 a b c d e f g h 304 8 0Z0Z0s 0Z 7 j0Z0Z0 Z0 6 No0Z0Z 0Z 5 Z0Z0Z0 Zp 4 0O0ZbZ pO 3 Z0Z0Or Z0 2 RZ0Z0O KA 1 Z0Z0Z0 Z0 a b c d e f g h 305 8 0Z0Z0Z 0Z 7 Z0Z0Z0 Z0 6 0Z0Z0Z 0Z 5 Z0Z0Z0 Z0 4 0Z0Z0Z 0Z 3 Z0Z0ZN Z0 2 0orZ0Z PO 1 s0j0J0 ZR a b c d e f g h 306 8 ra0Z0Z 0Z 7 j0o0Z0 ZR 6 PZPZ0Z 0Z 5 OpJ0Z0 Z0 4 0Z0Z0Z 0Z 3 Z0Z0Z0 Z0 2 0Z0Z0Z 0Z 1 Z0Z0ZB Z0 a b c d e f g h 5334 Problems, Combinations & Games 2.1 White to Move #2 421 8 rZkZNZ0Z 7 oRZRZ0Z0 6 KZ0Z0Z0Z 5 Z0Z0ZnZ0 4 0Z0Z0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 422 8 0Z0Z0Z0Z 7 ZNZ0Z0Z0 6 RZpZ0Z0Z 5 ZkZ0Z0Z0 4 0MpZ0Z0Z 3 Z0J0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 423 8 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0ZNZB 4 0Z0Z0Z0Z 3 Z0Z0o0ok 2 0Z0ZRZ0Z 1 Z0Z0Z0J0 a b c d e f g h 424 8 0Z0Z0Z0Z 7 Z0o0Z0S0 6 0ZRZNZ0j 5 Z0Z0Z0Z0 4 0Z0Z0Z0o 3 Z0Z0Z0ZK 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 425 8 0Z0Z0Z0Z 7 Z0Z0Z0Zp 6 0Z0Z0Z0L 5 Z0Z0Z0Z0 4 0Z0ZKZko 3 Z0Z0Z0Z0 2 0Z0Z0ZPZ 1 Z0Z0ZNZ0 a b c d e f g h 426 8 0Z0Z0Z0Z 7 Z0ZRZ0Z0 6 0o0Z0Z0Z 5 ZkZpZ0Z0 4 0Z0O0Z0Z 3 Z0J0Z0Z0 2 QZ0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 5334 Problems, Combinations & Games 2.1 White to Move #2 355 8 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0ZQZ0 4 0Z0Z0mBZ 3 Z0A0j0Z0 2 0Z0Z0Z0Z 1 Z0Z0ZKZ0 a b c d e f g h 356 8 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0Z0Z0 4 0Z0o0ZNZ 3 Z0ZKZ0Z0 2 0Z0Z0ZpL 1 Z0Z0ZkZ0 a b c d e f g h 357 8 0Z0Z0ZRZ 7 Z0Z0ZKm0 6 0Z0Z0Z0Z 5 Z0Z0Z0Z0 4 0Z0Z0Z0Z 3 Z0Z0Z0S0 2 0Z0Z0Z0Z 1 Z0Z0Z0Ak a b c d e f g h 358 8 0Z0ZKZ0Z 7 Z0Z0Z0Z0 6 0ZpZkZ0Z 5 Z0Z0O0Z0 4 0Z0OQZ0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 359 8 0Z0Z0Z0Z 7 jPO0Z0Z0 6 0SnZ0Z0Z 5 Z0J0Z0Z0 4 0Z0Z0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 360 8 0Z0Z0ArZ 7 Z0Z0Z0O0 6 0Z0Z0JBj 5 Z0Z0Z0Z0 4 0Z0Z0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 5334 Problems, Combinations & Games 2.1 White to Move #2 421 8 rZkZNZ0Z 7 oRZRZ0Z0 6 KZ0Z0Z0Z 5 Z0Z0ZnZ0 4 0Z0Z0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 422 8 0Z0Z0Z0Z 7 ZNZ0Z0Z0 6 RZpZ0Z0Z 5 ZkZ0Z0Z0 4 0MpZ0Z0Z 3 Z0J0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 423 8 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0ZNZB 4 0Z0Z0Z0Z 3 Z0Z0o0ok 2 0Z0ZRZ0Z 1 Z0Z0Z0J0 a b c d e f g h 424 8 0Z0Z0Z0Z 7 Z0o0Z0S0 6 0ZRZNZ0j 5 Z0Z0Z0Z0 4 0Z0Z0Z0o 3 Z0Z0Z0ZK 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 425 8 0Z0Z0Z0Z 7 Z0Z0Z0Zp 6 0Z0Z0Z0L 5 Z0Z0Z0Z0 4 0Z0ZKZko 3 Z0Z0Z0Z0 2 0Z0Z0ZPZ 1 Z0Z0ZNZ0 a b c d e f g h 426 8 0Z0Z0Z0Z 7 Z0ZRZ0Z0 6 0o0Z0Z0Z 5 ZkZpZ0Z0 4 0Z0O0Z0Z 3 Z0J0Z0Z0 2 QZ0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h

R c7

Qc5 5334 Problems, Combinations & Games 1.1 Mate in 1 295 8 0Z0Z0Z 0Z 7 Z0Z0Zp Z0 6 pZ0Z0Z pZ 5 jpZ0ZP Z0 4 0Z0ZnZ PA 3 OPZ0Z0 Z0 2 0ZKZ0Z 0Z 1 Z0Z0Z0 Z0 a b c d e f g h 296 8 0Z0Z0Z 0Z 7 snZ0Z0 o0 6 0j0Z0Z PZ 5 o0ZPZ0 Z0 4 Po0Z0M 0Z 3 ZKZ0Z0 Z0 2 0ZRZ0Z 0Z 1 Z0Z0Z0 Z0 a b c d e f g h 297 8 0Z0Z0Z 0Z 7 Z0Zns0 Z0 6 BZ0ako 0Z 5 Z0Z0o0 Z0 4 0O0Z0Z PZ 3 Z0Z0Z0 Z0 2 0ZKZNZ 0Z 1 Z0Z0Z0 ZR a b c d e f g h 298 8 0Z0Z0Z 0Z 7 ZRZ0Z0 Z0 6 0Z0ako 0Z 5 Z0Z0on Z0 4 0Z0oKZ PZ 3 Z0ZPZ0 Z0 2 0Z0Z0Z 0Z 1 Z0Z0Z0 Z0 a b c d e f g h 299 8 rZ0l0j 0s 7 opZ0a0 op 6 0Z0o0m 0Z 5 Z0Z0o0 M0 4 0Z0ZPZ bZ 3 OQM0A0 Z0 2 0O0Z0O PO 1 Z0ZRS0 J0 a b c d e f g h 300 8 0ZrZ0j 0s 7 obl0op Zp 6 0onZ0M 0Z 5 Z0m0Z0 A0 4 0Z0o0Z 0Z 3 ZPZPZP O0 2 PZ0MQZ 0O 1 ZKZ0S0 ZR a b c d e f g h 5334 Problems, Combinations & Games 1.1 Mate in 1 301 8 rZbZkZ 0s 7 ZpopZ0 Zp 6 pZ0ZpL 0Z 5 ZNO0Z0 Z0 4 0OKZ0Z 0Z 3 O0Z0On ZP 2 0Z0Z0O 0Z 1 Z0Z0Z0 Z0 a b c d e f g h 302 8 0Z0ZkZ 0Z 7 Z0Z0Op Z0 6 0Z0O0Z 0Z 5 Z0Z0Z0 Z0 4 0Z0Z0Z 0Z 3 Z0Z0ZQ Z0 2 0J0Z0Z 0Z 1 Z0Z0l0 Z0 a b c d e f g h 303 8 0Z0ZkZ 0Z 7 Z0Z0Z0 Z0 6 0Z0Z0O NZ 5 Z0ZQZ0 Z0 4 0Z0Z0Z 0Z 3 Z0Z0Zp Z0 2 0Z0ZnZ 0J 1 Z0Z0Zq Z0 a b c d e f g h 304 8 0Z0Z0s 0Z 7 j0Z0Z0 Z0 6 No0Z0Z 0Z 5 Z0Z0Z0 Zp 4 0O0ZbZ pO 3 Z0Z0Or Z0 2 RZ0Z0O KA 1 Z0Z0Z0 Z0 a b c d e f g h 305 8 0Z0Z0Z 0Z 7 Z0Z0Z0 Z0 6 0Z0Z0Z 0Z 5 Z0Z0Z0 Z0 4 0Z0Z0Z 0Z 3 Z0Z0ZN Z0 2 0orZ0Z PO 1 s0j0J0 ZR a b c d e f g h 306 8 ra0Z0Z 0Z 7 j0o0Z0 ZR 6 PZPZ0Z 0Z 5 OpJ0Z0 Z0 4 0Z0Z0Z 0Z 3 Z0Z0Z0 Z0 2 0Z0Z0Z 0Z 1 Z0Z0ZB Z0 a b c d e f g h 5334 Problems, Combinations & Games 2.1 White to Move #2 355 8 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0ZQZ0 4 0Z0Z0mBZ 3 Z0A0j0Z0 2 0Z0Z0Z0Z 1 Z0Z0ZKZ0 a b c d e f g h 356 8 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0Z0Z0 4 0Z0o0ZNZ 3 Z0ZKZ0Z0 2 0Z0Z0ZpL 1 Z0Z0ZkZ0 a b c d e f g h 357 8 0Z0Z0ZRZ 7 Z0Z0ZKm0 6 0Z0Z0Z0Z 5 Z0Z0Z0Z0 4 0Z0Z0Z0Z 3 Z0Z0Z0S0 2 0Z0Z0Z0Z 1 Z0Z0Z0Ak a b c d e f g h 358 8 0Z0ZKZ0Z 7 Z0Z0Z0Z0 6 0ZpZkZ0Z 5 Z0Z0O0Z0 4 0Z0OQZ0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 359 8 0Z0Z0Z0Z 7 jPO0Z0Z0 6 0SnZ0Z0Z 5 Z0J0Z0Z0 4 0Z0Z0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 360 8 0Z0Z0ArZ 7 Z0Z0Z0O0 6 0Z0Z0JBj 5 Z0Z0Z0Z0 4 0Z0Z0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h Kg 2 5 3 3 4 P r o b l e m s , C o m b i n a t i o n s & G a m e s 1 . 1 M a t e i n 1 3 0 1 8 r Z b Z k Z 0 s 7 Z p o p Z 0 Z p 6 p Z 0 Z p L 0 Z 5 Z N O 0 Z 0 Z 0 4 0 O K Z 0 Z 0 Z 3 O 0 Z 0 O n Z P 2 0 Z 0 Z 0 O 0 Z 1 Z 0 Z 0 Z 0 Z 0 a b c d e f g h 3 0 2 8 0 Z 0 Z k Z 0 Z 7 Z 0 Z 0 O p Z 0 6 0 Z 0 O 0 Z 0 Z 5 Z 0 Z 0 Z 0 Z 0 4 0 Z 0 Z 0 Z 0 Z 3 Z 0 Z 0 Z Q Z 0 2 0 J 0 Z 0 Z 0 Z 1 Z 0 Z 0 l 0 Z 0 a b c d e f g h 3 0 3 8 0 Z 0 Z k Z 0 Z 7 Z 0 Z 0 Z 0 Z 0 6 0 Z 0 Z 0 O N Z 5 Z 0 Z Q Z 0 Z 0 4 0 Z 0 Z 0 Z 0 Z 3 Z 0 Z 0 Z p Z 0 2 0 Z 0 Z n Z 0 J 1 Z 0 Z 0 Z q Z 0 a b c d e f g h 3 0 4 8 0 Z 0 Z 0 s 0 Z 7 j 0 Z 0 Z 0 Z 0 6 N o 0 Z 0 Z 0 Z 5 Z 0 Z 0 Z 0 Z p 4 0 O 0 Z b Z p O 3 Z 0 Z 0 O r Z 0 2 R Z 0 Z 0 O K A 1 Z 0 Z 0 Z 0 Z 0 a b c d e f g h 3 0 5 8 0 Z 0 Z 0 Z 0 Z 7 Z 0 Z 0 Z 0 Z 0 6 0 Z 0 Z 0 Z 0 Z 5 Z 0 Z 0 Z 0 Z 0 4 0 Z 0 Z 0 Z 0 Z 3 Z 0 Z 0 Z N Z 0 2 0 o r Z 0 Z P O 1 s 0 j 0 J 0 Z R a b c d e f g h 3 0 6 8 r a 0 Z 0 Z 0 Z 7 j 0 o 0 Z 0 Z R 6 P Z P Z 0 Z 0 Z 5 O p J 0 Z 0 Z 0 4 0 Z 0 Z 0 Z 0 Z 3 Z 0 Z 0 Z 0 Z 0 2 0 Z 0 Z 0 Z 0 Z 1 Z 0 Z 0 Z B Z 0 a b c d e f g h B f8 5334 Problems, Combinations & Games 2.1 White to Move #2 421 8 rZkZNZ0Z 7 oRZRZ0Z0 6 KZ0Z0Z0Z 5 Z0Z0ZnZ0 4 0Z0Z0Z0Z 3 Z0Z0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 422 8 0Z0Z0Z0Z 7 ZNZ0Z0Z0 6 RZpZ0Z0Z 5 ZkZ0Z0Z0 4 0MpZ0Z0Z 3 Z0J0Z0Z0 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 423 8 0Z0Z0Z0Z 7 Z0Z0Z0Z0 6 0Z0Z0Z0Z 5 Z0Z0ZNZB 4 0Z0Z0Z0Z 3 Z0Z0o0ok 2 0Z0ZRZ0Z 1 Z0Z0Z0J0 a b c d e f g h 424 8 0Z0Z0Z0Z 7 Z0o0Z0S0 6 0ZRZNZ0j 5 Z0Z0Z0Z0 4 0Z0Z0Z0o 3 Z0Z0Z0ZK 2 0Z0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 425 8 0Z0Z0Z0Z 7 Z0Z0Z0Zp 6 0Z0Z0Z0L 5 Z0Z0Z0Z0 4 0Z0ZKZko 3 Z0Z0Z0Z0 2 0Z0Z0ZPZ 1 Z0Z0ZNZ0 a b c d e f g h 426 8 0Z0Z0Z0Z 7 Z0ZRZ0Z0 6 0o0Z0Z0Z 5 ZkZpZ0Z0 4 0Z0O0Z0Z 3 Z0J0Z0Z0 2 QZ0Z0Z0Z 1 Z0Z0Z0Z0 a b c d e f g h 5334 Problems, Combinations & Games 2.1 White to Move #2 421 8 rZkZN Z0Z 7 oRZRZ 0Z0 6 KZ0Z0 Z0Z 5 Z0Z0Z nZ0 4 0Z0Z0 Z0Z 3 Z0Z0Z 0Z0 2 0Z0Z0 Z0Z 1 Z0Z0Z 0Z0 a b c d e f g h 422 8 0Z0Z0 Z0Z 7 ZNZ0Z 0Z0 6 RZpZ0 Z0Z 5 ZkZ0Z 0Z0 4 0MpZ0 Z0Z 3 Z0J0Z 0Z0 2 0Z0Z0 Z0Z 1 Z0Z0Z 0Z0 a b c d e f g h 423 8 0Z0Z0 Z0Z 7 Z0Z0Z 0Z0 6 0Z0Z0 Z0Z 5 Z0Z0Z NZB 4 0Z0Z0 Z0Z 3 Z0Z0o 0ok 2 0Z0ZR Z0Z 1 Z0Z0Z 0J0 a b c d e f g h 424 8 0Z0Z0 Z0Z 7 Z0o0Z 0S0 6 0ZRZN Z0j 5 Z0Z0Z 0Z0 4 0Z0Z0 Z0o 3 Z0Z0Z 0ZK 2 0Z0Z0 Z0Z 1 Z0Z0Z 0Z0 a b c d e f g h 425 8 0Z0Z0 Z0Z 7 Z0Z0Z 0Zp 6 0Z0Z0 Z0L 5 Z0Z0Z 0Z0 4 0Z0ZK Zko 3 Z0Z0Z 0Z0 2 0Z0Z0 ZPZ 1 Z0Z0Z NZ0 a b c d e f g h 426 8 0Z0Z0 Z0Z 7 Z0ZRZ 0Z0 6 0o0Z0 Z0Z 5 ZkZpZ 0Z0 4 0Z0O0 Z0Z 3 Z0J0Z 0Z0 2 QZ0Z0 Z0Z 1 Z0Z0Z 0Z0 a b c d e f g h An example. Take these two second-hand books of chess puzzles. We can learn the two independent shortcuts (blue arrows for the left book OR handwritten solutions on the right), or actually learn to play chess (the invariant mechanism). While both strategies solve other problems from the same books (i.i.d.), only the latter generalises to new chess puzzle books (o.o.d.). How to distinguish the two? We would not have learned about the red arrows had we trained on the book on the right, and vice versa with the hand-written notes.

2. EXPLANATIONS THAT ARE HARD TO VARY

We consider datasets tD e u ePE , with |E| " d, and D e " px e i , y e i q, i e " 1, . . . , n e . Here x e i P X Ď R m is the vector containing the observed inputs, and y e i P Y Ď R p the targets. The superscript e P E indexes some aspect of the data collection process, and can be interpreted as an environment label. Our objective is to infer a function f : X Ñ Y -which we call mechanism -assigning a target y e i to each input x e i ; as explained in the introduction, we assume that such function is shared across all environments. For estimation purposes, f may be parametrized by a neural network with continuous activations; for weights θ P Θ Ď R n , we denote the neural network output at x P X as f θ pxq. Gradient-based optimization. To find an appropriate model f θ , standard optimizers rely on gradients from a pooled loss function L : R n Ñ R. This function measures the average performance of the neural network when predicting data labels, across all environments: Lpθq :" i qPD e pf px e i ; θq, y e i q; where : R p ˆRp Ñ r0, `8q is usually chosen to be the L2 loss or the cross-entropy loss. The parameter updates according to gradient descent (GD) are given by θ k`1 GD " θ k GD ´η∇Lpθ k GD q, where η ą 0 is the learning rate. Under some standard assumptions (Lee et al., 2016) , pθ k GD q kě0 converges to a local minimizer of L, with probability one. When do we not learn invariances? We start by describing what might prevent learning invariances in standard gradient-based optimization. (i) Training stops once the loss is low enough. If optimization learned spurious patterns by the time it converged, invariances will not be learned anymore. This depends on the rate at which different patterns are learned. The rates at which invariant patterns emerge (and vice-versa, the spurious patterns do not) can be improved by e.g.: (a) careful architecture design, e.g. as done by hardcoding spatial equivariance in networks; (b) fine-tuning models pre-trained on large amounts of data, where strong features already emerged and can be readily selected.



Figure 1: Loss landscapes of a two-parameter model. Averaging gradients forgoes information that can identify patterns shared across different environments.

We formalize a notion of consistency, which characterizes to what extent a minimum of the loss surface appears only when data from different environments are pooled. Minima with low consistency are 'patchwork' solutions, which (we hypothesize) sew together different strategies and should not be expected to generalize to new environments. An intuitive description of this principle was proposed by physicist David Deutsch: "good explanations are hard to vary" (Deutsch, 2011).

. A practical instantiation of ILC leads to o.o.d. generalization on a challenging synthetic task where several established regularizers fail to generalize; moreover, following the memorization task from Zhang et al. (2017), ILC prevents convergence on CIFAR-10 with random labels, as no shared mechanism is present, and similarly when a portion of training labels is incorrect. Lastly, we set up a behavioural cloning task based on the game CoinRun (Cobbe et al., 2019b), and observe better generalization on new unseen levels.

|E| řePE L e pθq, with L e pθq :" 1

