GROUP-WISE VERIFIABLE DISTRIBUTED COMPUTING FOR MACHINE LEARNING UNDER ADVERSARIAL ATTACKS

Abstract

Distributed computing has been a promising solution in machine learning to accelerate the training procedure on large-scale dataset by utilizing multiple workers in parallel. However, there remain two major issues that still need to be addressed: i) adversarial attacks from malicious workers, and ii) the effect of slow workers known as stragglers. In this paper, we tackle both problems simultaneously by proposing Group-wise Verifiable Coded Computing (GVCC), which leverages coding techniques and group-wise verification to provide robustness to adversarial attacks and resiliency to straggler effects in distributed computing. The key idea of GVCC is to verify a group of computation results from workers at a time, while providing resilience to stragglers through encoding tasks assigned to workers with Group-wise Verifiable Codes. Experimental results show that GVCC outperforms the existing methods in terms of overall processing time and verification time for executing matrix multiplication, which is a key computational component in machine learning and deep learning.

1. INTRODUCTION

Recently, machine learning and big data analysis have achieved huge success in various areas such as computer vision, natural language processing, and reinforcement learning, etc. Since they usually demand a massive amount of computation on a large dataset, there has been increasing interest in distributed systems, where one node is used as a master and the others are used as workers. One possible option is distributed computing (Dalcín et al., 2005; 2011) , where the workers compute partial computation task received from the master. In distributed computing, the master divides and distributes tasks (which require far small memory than the original task) to workers, and they compute the assigned tasks and send results back to a master. Distributed computing can be utilized to compute matrix multiplication in machine learning, the most important and frequent computation block. In a distributed setting, however, there exist two foremost considerations to embed distributed computing in machine learning applications, i) stragglers and ii) adversarial workers. Stragglers are workers that return their computation results much slower than others. It has been reported that stragglers can be a serious bottleneck to performing large-scale computation tasks (Dean & Barroso, 2013; Huang et al., 2017; Tandon et al., 2017) . To handle straggler effects, coded computing was first suggested in Lee et al. (2018) . In coded computing, a master encodes a computation task with a coding technique while retaining redundancy in task allocation. Due to the redundancy arisen from the coding technique, a master does not need all results of tasks to achieve the final output and can ignore stragglers. This approach has been applied to various computation tasks, especially on matrix multiplication (Dutta et al., 2016; Yu et al., 2017; Park et al., 2018; Reisizadeh et al., 2019; Dutta et al., 2019; Yu et al., 2020) . Moreover, some of the workers could be adversarial workers, which send perturbed results to the master to contaminate or degrade the performance of neural networks. Many studies (Biggio et al., 2012; Blanchard et al., 2017; El Mhamdi et al., 2018; Sohn et al., 2020; Bagdasaryan et al., 2020; Wang et al., 2020) demonstrate that adversarial workers slow down the overall training process and To be robust against adversarial attacks, a master can check whether the computation results from workers are correct or not to figure out adversarial workers. Based on this idea, there have been some recent studies that suggest an encoding scheme for assigning computation tasks to workers to tolerate wrong computation results when obtaining the final computation product, and to identify adversarial workers from the returned computation results (Yu et al., 2019; Soleymani et al., 2021; Hong et al., 2022) . Another line of research, which we call verifiable computing, has focused on identifying adversarial workers with a fast verification procedure. In verifiable computing, a master verifies the correctness of computation results by using a verification key (Freivalds, 1977; Tang et al., 2022) . In this paper, we focus on designing a fast and robust distributed computing system for matrix multiplication. We use both a coding approach and verifiable computing to combat adversarial attacks from malicious workers and straggler effects. Our system model is depicted in Fig 1(b) , where a master encodes the computation tasks, distributes them to the workers, and decodes the final product after receiving and verifying the computation results from the workers. To be specific, we suggest a novel encoding scheme that enables verifying a large number of computation results at a time, and decoding the final computation product from a part of computation results from the workers to mitigate straggler effects as well. We experiment with various settings and demonstrate that our proposed scheme can identify adversarial workers and obtain the final computation product much faster than existing methods. The main contributions of this paper are as follows. • We propose Group-wise Verifiable Coded Computing (GVCC), handling both straggler effects and adversarial attacks in distributed computing for matrix multiplication tasks. To be more specific, we suggest group-wise verifiable codes (GVC) for encoding the computation tasks of workers and provide a suitable group-wise verification algorithm. We first provide a process of group-wise verification trial (GVT), then propose a two-stage verification algorithm based on GVT. We also provide modified verification algorithm that can be utilized under straggler effects. 

2. PROBLEM SETTING

We consider a distributed computing scenario for matrix multiplication task with one master and E workers {W i } E i=1 . A master wants to perform a matrix multiplication C = AB on input matrices A ∈ F a×b q and B ∈ F b×c q for a sufficiently large finite field F q . The master generates encoded matrices using A and B, then distributes them to workers to allocate tasks. To be specific, the master encodes Ãi , Bi and sends them to W i for i ∈ [1 : E]. After receiving Ãi and Bi , each worker W i computes the allocated task We assume that some of the workers could be adversarial workers or stragglers. We list our assumptions on adversarial workers and stragglers as follows: • There exist l adversarial workers W L , where L denotes a set of adversarial workers for L ⊂ [1 : E] and |L| = l. • Stragglers occur randomly depending on the network condition and currently available computing resource at each worker. • Adversarial workers have knowledge about input matrices A and B, and all the protocol of distributed computing including encoding, task allocation, and decoding, only except the random key r that is used for verification.foot_1 • A master cannot be aware of the number of adversarial workers in advance. • Adversarial workers cannot collude with each other, which implies that they cannot access the information about assigned tasks of other workers. To improve the performance of distributed computing systems under adversarial attacks and stragglers, we aim to reduce verification time and overall processing time of distributed matrix multiplication.

3. BACKGROUND: VERIFIABLE COMPUTING AND CODED COMPUTING

In this section, we first introduce two methods that are leveraged in GVCC: i) verifiable computing that enables identifying adversarial workers and ii) coded computing that mitigates the effects of stragglers.

3.1. VERIFIABLE COMPUTING FOR ADVERSARIAL ATTACK

The authors of Freivalds (1977) proposed a verification method (Freivalds' method) that uses a random key to verify the computation results. Freivalds' method for matrix multiplication can be summarized in three steps as follows. Computation assignment: A master assigns a matrix multiplication task C = AB for A ∈ F a×b q and B ∈ F b×c q to an available worker. The worker executes the assigned task, and returns its result C ′ to the master. Verification key generation: The master generates a random key r ∈ F 1×a q and computes a verification key s 1 = rA. Correctness check: The master computes s 2 = s 1 B = rAB = rC and s 2 ′ = rC ′ from the received result. By comparing s 2 and s ′ 2 , the master can verify whether C ′ is correct or not. It has been known that the master can identify adversarial workers using this verification process with probability 1 -1 q in the finite field F q (Sahraei & Avestimehr, 2019) . In this verification procedure, the master computes ab + bc + ca scalar products while the original task C = AB requires abc scalar products.

3.2. CODED COMPUTING FOR STRAGGLERS

In distributed computing, several methods have been proposed to mitigate straggler effects. One traditional method is to replicate a computational task and allocate it to multiple workers (Wang et al., 2014) . However, this replication-based method is inefficient as it requires K additional workers to tolerate a single straggler, when the computational task is divided into K smaller tasks. To handle straggler effects more efficiently, coded computing was suggested in Lee et al. (2018) . By using a coding theoretic approach for allocation of tasks to workers, the master can retrieve the final product after receiving enough number of workers. We now define an important metric that has been used in many coded computing researches. Recovery threshold R: The minimum number of computation results of tasks from non-adversarial workers that the master requires to obtain the final product for the worst-case scenario. 3In coded computing, the master can decode the final output from R out of E results and it is able to tolerate up to E -R stragglers. Therefore, coded computing requires S additional workers for handling S stragglers, while the replication-based method requires KS additional workers for S stragglers.

4. GROUP-WISE VERIFIABLE CODED COMPUTING

In this section, we provide GVC and propose an algorithm for group-wise verification. We also suggest a modified algorithm for GVCC under straggler effects.

4.1. ENCODING OF GROUP-WISE VERIFIABLE CODES (GVC)

We now explain the encoding procedure of GVC that facilitates group-wise verification at the master. In GVC, the master divides input matrices A and B into sub-matrices of equal size A w ∈ F a m ×b q for w ∈ [1 : m], and B z ∈ F b× c n q for z ∈ [1 : n]. Then input matrices and their product C = AB are given by A =    A 1 . . . A m    , B = [ B 1 • • • B n ] , C =    A 1 B 1 • • • A 1 B n . . . . . . . . . A m B 1 • • • A m B n    . To generate the encoded matrices, the master uses encoding functions p A and p B constructed by using sub-blocks of A and B as coefficients, which are given by p A (x) = Σ m i=1 A i x i-1 , p B (x) = Σ n j=1 B j f j-1 (x), where x represents the variable of encoding functions p A and p B . f (x) is the basic building component for constructing p B and we now introduce the conditions on f (x) that enable groupwise verification of GVC.foot_3  Condition I : The polynomial function f (x) used in the encoding function p B (x) should satisfy the following conditions. i) f (x) is an mth order polynomial function. ii) f (x) has at least E m sets of m evaluation points α 1,1 , α 1,2 , ..., α E m ,m that satisfy f (α t,1 ) = f (α t,2 ) = • • • = f (α t,m ) = w t for t ∈ 1 : E m . cUsing the encoding functions and evaluation points satisfying Condition I, the master generates the encoded matrices Ãt,u and Bt,u for t ∈ 1 : E m and u ∈ [1 : m] as Ãt,u = p A (α t,u ) = Σ m i=1 A i α i-1 t,u , (3) Bt,u = p B (α t,u ) = Σ n j=1 B j f j-1 (α t,u ) = Σ n j=1 B j w j-1 t = Bt, * , where w t = f (α t,u ) for u ∈ [1 : m], which are the values of encoding functions p A and p B at the evaluation points x = α t,u . Remark 1. (Quantization and embedding into the finite field) In this paper, we assume that input matrices are defined on a finite field F q , while the dataset for machine learning and polynomial function f (x) used for encoding are based on the real field R. To express real field data and function in the finite field, we quantize matrices A, B, and f (x) using v bits, and embed them into a finite field F q of integers at modulo a prime q. Quantization of real numbers and embedding into the finite field have been used in several previous works (Yu et al., 2017; Ji et al., 2021; Tang et al., 2022) . To analyze the effect of quantization in training of the deep neural network, we provide an experiment in Appendix B. Remark 2. (Polynomials for GVC) GVC can be constructed using any polynomial satisfying Condition I. For instance, Chebyshev polynomial, which has been used in several coded computing studies (Fahim & Cadambe, 2021; Hong et al., 2021) , can be used as f (x) by using quantization as stated in Remark 1. We also provide the proof in Appendix C. Remark 3. (Reduced Computational overhead for encoding) In GVC, since it is guaranteed Bt,u = Bt, * , ∀u ∈ [1 : m] for t ∈ 1 : E m , the master encodes (1 + 1 m )E matrices to assign E tasks, instead of encoding 2E matrices as in the conventional methods. Thus, the computational overhead for encoding can be lowered by GVC.

4.2. TASK ASSIGNMENT AND COMPUTING AT WORKERS

To assign the tasks to workers, the master randomly rearranges workers into E m groups of same size, which can be denoted as W t,u for t ∈ 1 : E m and u ∈ [1 : m], and distributes Ãt,u , Bt,u to W t,u . Each worker W t,u computes the assigned task Ct,u = Ãt,u Bt,u , which corresponds to the value of the evaluation function p C (x) at the evaluation point x = α t,u , where p C (x) is given by p C (x) = p A (x) × p B (x) = Σ m i=1 Σ n j=1 A i B j x i-1 f j-1 (x). After each worker finishes the assigned task, it returns its computation result to the master.

4.3. GROUP-WISE VERIFICATION FOR GVC

After receiving the results from workers, the master begins a verification process. We first explain a group-wise verification process and suggest two-stage verification algorithm based on this process.

4.3.1. A GROUP-WISE VERIFICATION TRIAL (GVT)

Let us denote workers with finished computation among {W t,u } m u=1 by T t for t ∈ [1 : E m ]. By GVC, a master can verify results from group T t in a single trial. In conventional verifiable computing approach, a master verifies the computation result C′ i for i ∈ [1 : E] individually. However, if the tasks are encoded by GVC, it is ensured that Σ Tt Ct,u = Σ Tt Ãt,u Bt,u = Σ Tt Ãt,u Bt, * = Σ Tt Ãt,u × Bt, * for a group of workers T t among {W t,u } m u=1 where |T t | = k (k ≤ m). Therefore, the master can verify k computation results Ãt,u Bt,u in a group by verifying Σ Tt Ãt,u × Bt, * instead. On the basis of this fact, we suggest a process for group-wise verification trial (GVT). Verification key generation: For each trial of group-wise verification, a master generates a random key rt ∈ F 1× a m and computes a verification key s1,t = rt Σ Tt Ãt,u . Group-wise Correctness check: The master computes s2,t = 1 k s1,t Σ Tt Bt,u = s1,t × Bt, * = rt Σ Tt Ãt,u × Bt, * = rt Σ Tt ( Ãt,u Bt,u ) = rt Σ Tt Ct,u and s′ 2,t = rt Σ Tt C′ t,u from the received computation results. If s2,t ̸ = s′ 2,t , then the master marks the output of verification trial as Positive, which implies there exist more than one adversarial worker in the tested group T t , and marks as Negative otherwise.foot_5 

4.3.2. TWO-STAGE VERIFICATION ALGORITHM

We now provide a two-stage verification algorithm of GVC, given by Algorithm 1. 66Algorithm 1: Group-wise Verifiable Coded Computing 1 Two-stage group-wise verification: After receiving all computation results from workers, the master begins group-wise verification. 2 for t ∈ [1 : E m ] do 3 The master verifies m workers using GVT in Section 4.3.1, where T t = {W t,u } m u=1 . Put all workers in T t to a set of non-adversarial workers T n .

// Stage 1

The master verifies each worker in T p using GVT for k = 1, and put non-adversarial workers whose verification result is negative to T n until |T n | = R. // Stage 2 Two-stage group-wise verification: After receiving all computation results of tasks from workers, the master starts group-wise verification based on GVT. However, one thing we should take into account is that GVC can verify up to m results at once. Thus, we propose a two-stage group-wise verification algorithm adjusted to the parameter m. • Stage 1: The master groups {W t,u } m u=1 as T t for t ∈ [1 : E m ]. The master performs a group-wise verification on T t using GVT. • Stage 2: The master verifies each computation result in T t if group-wise verification result for T t is positive, until it achieves R non-adversarial workers to obtain C.

4.4. DECODING AT THE MASTER

After verifying R non-adversarial computation results, the master starts decoding. As we can see in equation 1 and equation 4, A i B j for i ∈ [1 : m], j ∈ [1 : n] can be retrieved from p C (x) to achieve C. Since Ct,u = p C (α t,u ) and p C (x) is a polynomial of degree mn-1, it can be interpolated from mn distinct value of Ct,u . Hence, the master needs any fastest mn non-adversarial computation results to decode the final output C. Thus, the recovery threshold of GVC is R = mn. Decoding can be done by the inversion of the coefficient matrix in    C1 . . . Cmn    =       β 1 0 . . . β 1 m-1 f (β 1 ) . . . β 1 m-1 f (β 1 ) n-1 . . . . . . . . . β mn 0 . . . β mn m-1 f (β mn ) . . . β mn m-1 f (β mn ) n-1    ⊗ I a m × a m    ×    A 1 B 1 . . . A m B n    , where Ck and β k for k ∈ [1 : mn] denote the fastest mn computation results and the evaluation points of them, respectively and ⊗ denotes Kronecker product. It can be also achieved by interpolation (Kedlaya & Umans, 2011) of p C (x) and extracting A i B j from p C (x) by repeated division of f (x). We provide more detailed decoding process in Appendix E. Remark 4. (Optimal Recovery threshold) It has been known in Yu et al. (2017) that the optimal recovery threshold for coded matrix multiplication is mn when A and B are divided into m and n partitions in a row-wise and a column-wise manner, respectively. Thus GVC also achieves the optimal recovery threshold. Remark 5. (Upper bound of tolerating adversarial attacks) Since GVCC requires mn computation results to obtain the final product, GVCC can tolerate up to l upper = E -S -mn adversarial workers, where S denotes the number of stragglers.

4.5. MODIFIED VERIFICATION ALGORITHM OF GVCC UNDER STRAGGLER EFFECTS

We now provide a modified algorithm that can be used under straggler effects. Two-stage group-wise verification in modified GVCC: • Stage 1: The master groups {W t,u } m u=1 as T t for t ∈ [1 : E m ] as in GVCC. The master establishes a certain threshold z. The master starts a group-wise verification for the computation results of all groups received up to that time when the master receives z computation results from workers. • Stage 2: The master first individually verifies the computation results that arrived late, and verifies each computation result in T t if group-wise verification result for T t is positive, until it finds R non-adversarial workers to obtain C. By setting a proper threshold z, the modified verification algorithm can efficiently deal with straggler effects. However, it may lose some of the benefits of group-wise verification as a trade-off because the master verifies a smaller group of workers in a single trial. The impact on overall processing time will be analyzed in Section 6 via experiment.

5. RELATED WORKS

In distributed computing, there have been several studies on handling adversarial attacks and straggler effects simultaneously. First, to have tolerance over adversarial attacks and provide resiliency to straggler effects, Lagrange coded computing (LCC) has been suggested in Yu et al. (2019) . In LCC, a master requires 2K additional non-adversarial workers to be robust against K adversarial workers. In addition, a list-decoding approach has been applied to distributed matrix multiplication in Subramaniam et al. ( 2019), and it reduces the additional cost of tolerating adversarial attacks by a factor of two asymptotically compared to LCC. However, the computational complexity to perform list-decoding increases quadratically with the number of workers. Recently, verifiable computing has been introduced to coded computing schemes. The authors of Tang et al. (2022) have suggested AVCC to counter straggler and adversarial workers. In AVCC, a master only requires K additional computation results from workers if there exist K adversarial workers, because it can verify each computation result using a verification key. However, since the master can verify only a single computation result at a time, the verification process can be a bottleneck for overall processing if the master utilizes numerous workers in the system. In addition, there have been several studies that are based on a group-wise identification to find adversarial workers, while mitigating straggler effects using coded matrix multiplication. First, group testing algorithm has been applied to coded computing in Solanki et al. (2019) . However, since it has a constraint on the minimum testable size for group testing, the number of test trials can be significantly large. To mitigate this limitation, hierarchical group testing (HGT) has been proposed in Hong et al. (2022) . However, HGT imposes high computational complexity on a master since it requires matrix inversion or Reed-Solomon decoding to identify adversarial workers, whereas verifiable computing requires matrix-vector multiplications to verify them. In this section, we provide the performance of GVCC and modified GVCC in terms of overall processing time, and compare with the two existing schemes, AVCC (Tang et al., 2022) and HGT (Hong et al., 2022) . We proceed distributed matrix multiplication in a cluster of Amazon EC2 cloud, and use one t2.2xlarge node as a master and use t2.micro nodes as workers. Implementation is done by MPI4py (Dalcín et al., 2005) and matrices are randomly generated by NumPy. We set the verification time limit as 100 seconds and if the verification process takes more than 100 seconds, we terminate the experiment and mark it as a failure. We run four experiments and evaluate the overall processing time. Each experiment is repeated 20 times and the result is depicted in Fig. 2 . We also list stage-wise processing time (encoding (Enc), task assignment (SA), computation & return (CR), verification (Ver), decoding (Dec), and overall processing times) of experiments in Appendix F.1.

6. EXPERIMENTAL RESULTS

In Fig. 2 (a), we compare overall processing time with different number of adversarial workers l, when A ∈ F 3500×6300 q , B ∈ F 6300×3600 q , m = 5, n = 9, and E = 60. GVCC speeds up overall processing time compared to AVCC for all cases, and also shows faster overall processing than HGT when l ≥ 3. As we have claimed in Remark 3, the computational overhead for encoding the tasks can lowered by GVC, which results in reduced encoding and overall processing time compared to AVCC. Furthermore, for the verification time of HGT increases significantly with the number of workers, the verification and overall processing time of GVCC is much lower than HGT when l ≥ 3. To be specific, GVCC speeds up verification time up to ×3.34 than AVCC, and ×174 than HGT, and HGT even fails to proceed verification process in the given time limit (100s) when l = 5. In Fig. 2 (b), we show the overall processing time with stragglers in distributed computing systems, where A ∈ F 3500×6300 q , B ∈ F 6300×3600 q , m = 5, n = 9, E = 60, and l = 3. To simulate straggler effects in distributed computing, we randomly pick stragglers with probabilities p = 0, 0.02, 0.05, 0.1, and 0.2. Stragglers are forced to run background thread which slows down the execution of assigned tasks. In this experiment, we use the modified algorithm in Section 4.5 for GVCC (z = R = 63) and compare with AVCC and HGT. Since verification is done asynchronously for the received computation results in GVCC, verification time is added to computation & return time. As we can see in the Fig 2(b), GVCC outperforms AVCC and HGT for all cases. Thus, it is claimed that GVCC can efficiently handle straggler effects while leveraging a group-wise verification approach. We also provide additional experiments with a modified algorithm on another parameter setting in Appendix F.2. In Fig. 2 (c), we compare overall processing time with different shape of input matrices with three parameter settings: i) Square (A ∈ F 3600×3600 q , B ∈ F 3600×3600 q ), ii) Tall (A ∈ F 4500×1500 q , B ∈ F 1500×4500 q ), iii) Fat (A ∈ F 2700×7000 q , B ∈ F 7000×2700 q ), where m = 5, n = 9, l = 3, and E = 60. GVC speeds up overall processing time compared to AVCC and HGT, regardless of matrix shapes. The performance of GVCC with the different numbers of workers is depicted in Fig. 2(d) . We run experiments to multiply two input matrices A ∈ F 2100×4000 q and B ∈ F 4000×3861 q , with three parameter setting (E, m, n): i) (60, 5, 9), ii) (84, 6, 11), iii) (112, 7, 13) , where l = 3. GVCC achieves the lowest overall processing time in every setting. To be specific, GVCC speeds up verification time up to ×3.5 than AVCC and ×57.29 than HGT. Furthermore, GVCC speeds up overall processing time up to ×1.20 than AVCC and ×1.10 than HGT. Furthermore, it should be noted that GVC shows more benefit than AVCC as N or m gets bigger. In specific, our methods have a verification complexity of O( R m N 2 ) while AVCC has O(RN 2 ). To show the performance of GVC with different m and N , we indicate the verification time of GVC and the other schemes in table 1. We used parameter setting ((E, m, n): i) (60, 5, 9), ii) (84, 6, 11), iii) (112, 7, 13) , where l = 3 (which are same setting with experiment in Fig 2 (d) ). We indicate the verification time of HGT, AVCC, and GVC in Table 1 . We can see that the verification time of GVC remains the same value, while other methods show an increase in the verification time. This is because other methods require the verification of more results as the recovery threshold increases, but GVC has the same number of groups that needs to be verified. Moreover, to show the benefits of GVC, we conduct additional experiments. We experimented with the same parameter setting in Fig 2(a) when l = 3 and only changed the size of the matrix (m, n = 5, 9, E = 60, l = 3 and a = b = c = N ). We have changed the size of input matrix N from 900 to 3600 to show the benefit of GVC as N increases, and indicate the verification time in Table 1 . As we can see in the table, verification gap between GVCC and other scheme becomes larger as N increases. In specific, HGT shows far larger verification time than GVC. Moreover, gap between AVCC and GVC increases ×2.66 when N becomes ×2, and ×9.36 when N becomes ×4.

7. CONCLUSION

In this paper, we propose GVCC, a robust and fast distributed computing scheme for matrix multiplication that can deal with two important problems: adversarial attacks and straggler effects. By combining the verifiable computing with coded computing, GVCC allows a master to verify a group of computation results at a time, reducing verification time to find adversarial workers and provide resiliency to straggler effects simultaneously. Consequently, GVCC is shown to have significant reduction in verification and overall processing time, and experimental results demonstrate the effectiveness of GVCC in distributed computing systems.



We provide more explanation and detailed experiment settings in Appendix A. Protecting the random key from the adversarial workers can be realized by randomly generating r at the master, after receiving results from the workers. It should be noted that previous works have included stragglers or adversarial workers in the recovery threshold and in this case, the recovery threshold of GVC can be expressed as R ′ = R + S + l. The condition for f (x) was also used in squeezed polynomial codes(Hong et al., 2020), in order to reduce communication load in distributed computing. if Group-wise verification result for T t is negative then else It should be noted that group-wise verification trial has the same accuracy with the individual verification inFreivalds (1977). We demonstrate this by experiment under various adversarial attacks in Appendix D. We do not consider straggler effects here for simplicity, but provide a modified verification algorithm of GVC that considers straggler effects in Section 4.5.



Distributed computing for a matrix multiplication C = AB under adversarial workers and stragglers.

Figure 1: Training curve of neural networks under adversarial workers and system model for distributed computing

(a) Number of adversarial workers l (b) Straggler probability (c) Matrix shape (d) Number of workers Figure 2: Overall processing time of AVCC, HGT, and GVCC (proposed).

Verification time of GVC, AVCC, and HGT with different matrix size and m.

annex

Put all workers in T t to a set of possible adversarial workers T p .

