The parallel ADMM algorithm presented in [14] is shown as follows: It is clear that the number of dual variables in (2.3) is twice that in DJP-ADMM. Thus, the communication burden among agents and the storage cost for each agent in Algorithm 1 are smaller than ones in [14].

Hong et al. [16] have pointed out that ${\tilde{L}}_{-}$ is the sign Laplace matrix of the graph G.

If proximal matrices $P_i$ ($i=1,\cdots ,n$) are symmetric, then Assumption 1 can be reduced to $P+\rho \overline{A}\otimes I_m$ is a positive definite matrix. Therefore, $P=\rho D=\rho \tilde{D}\otimes I_m$ is a feasible choice, where $\tilde{D}$ is the degree matrix of the graph G. In this case, $P_i=\rho d_i{I}_m$.

By the definition of Q, the matrix Q is symmetric positive definite under Assumption 1, and thus, there exists a matrix M such that

Assume that $\left\{(x^k,{\lambda }^k)\right\}$ is the sequence produced by Algorithm 1 for the problem (2.2), where $x^k={[{(x_1^k)}^T,{(x_2^k)}^T,...,{(x_n^k)}^T]}^T$ and ${\lambda }^k=\left[{\lambda }_{e_{ij}}^k\right],$$e_{ij}\in E$. Then one has

Define $g_i(i=1,\cdots ,n):{\mathbb{R}}^m\to \mathbb{R}$ by Using the iteration of x in Algorithm 1, one can conclude that $x_i^{k+1}$ is the optimizer of $f_i+g_i^k$, i.e., Therefore, there exists a subgradient $h(x_i^{k+1})\in \partial f_i(x_i^{k+1})$ such that $h(x_i^{k+1})+\nabla g_i^k(x_i^{k+1})=0$. Then Due to the convexity of $f_i$, we have This together with (3.8) implies that Substituting the gradient $\nabla g_i^k$ of the function $g_i^k$ into the above inequality, we have From the iteration of the multipliers, one can obtain that Similarly, Hence, By the definition of the matrix A, we simplify the above inequality as follows: And then, By the definition of matrices A and D, we have In addition, where $\overline{A}$ is the adjacency matrix of the graph G. The above two relations indicate that Therefore, by the definition of the extended sign Laplace matrix $L_{-}$, one can conclude that Analogously, Besides, by the definition of matrix Q, we have Thus, recalling (3.9)-(3.12), inequality (3.7) holds. □

Assume that $\left\{(x^k,{\lambda }^k)\right\}$ is the sequence produced by Algorithm 1 for the problem (2.2), where $x^k={[{(x_1^k)}^T,{(x_2^k)}^T,...,{(x_n^k)}^T]}^T$ and ${\lambda }^k=\left[{\lambda }_{e_{ij}}^k\right],$$e_{ij}\in E$. Then under Assumption 1, one has the following equality where $Q=M^{T}M$.

To prove (3.14), we firstly claim that Indeed, by the iteration of the multiplier: ${\lambda }^{k+1}={\lambda }^k-\gamma \rho A{x}^{k+1}$, we know and Therefore, equality (3.17) and (3.18) indicate that equality (3.15) is valid. In addition, by distorting some of the terms, we obtain and Combining the above two equalities, we yield Taking into account $Q=M^{T}M$, we can get the equality (3.16). Consequently, by (3.15) and (3.16), the equality (3.14) holds. □

Assume that $\left\{(x^s,{\lambda }^s)\right\}$ is the sequence produced by Algorithm 1, where $x^s={[{(x_1^s)}^T,{(x_2^s)}^T,...,{(x_n^s)}^T]}^T$ and ${\lambda }^s=\left[{\lambda }_{e_{ij}}^s\right],e_{ij}\in E$. Let $y^k=\frac{1}{k}{\sum }_{s=0}^{k-1}{x}^{s+1}$ be the ergodic average of $x^s$ from step 1 to k. $x^{*}$ is the optimal solution of the problem (2.2). Then under Assumption 1, the following relation holds for any $k\ge 1$ and for $0<\gamma \le 2$

where $V^0$ given in (3.13) is a non-negative term. Furthermore,

It follows from the optimality of $x^{*}$ that the first inequality in (3.19) is clearly true. Let $x=x^{*}$ in inequality (3.7), then one has Take into consideration that $L_{-}=A^{T}A$ and $A{x}^{*}=0$, the above inequality can be rewritten as: By Lemma 4.2, one has and then Due to $V^k\ge 0$ for any k, the following inequality holds for $0<\gamma \le 2$ Since the function f is convex, ${\sum }_{s=0}^{k-1}f(x^{s+1})\ge kf(\frac{1}{k}{\sum }_{s=0}^{k-1}{x}^{s+1})$, and then using $y^k=\frac{1}{k}{\sum }_{s=0}^{k-1}{x}^{s+1}$, we have i.e., Therefore, inequality (3.19) stands. Furthermore, inequality (3.22) implies that On the other hand, from the optimality of $x^{*}$, we have As a result, ${lim}_{k\to +\infty }\left(f(y^k)-f(x^{*})\right)=0$ and the proof is completed. □

Theorem 1 gives the theoretical upper bound for $f(y^k)-f^{*}$, which provides the error estimates for the optimal value $f^{*}$ at each iteration k. The upper bound is consist of two additive items. Both of them approach to zero at the rate $O(1/k)$. In addition, Theorem 1 implies that $f(x^k)$ converges to the optimal value $f^{*}$ asymptotically. Furthermore, if at least one function $f_i$ is strongly convex, then the optimal solution $x^{*}$ is unique, and thus $x^k$ asymptotically approaches to $x^{*}$.

When solving the consensus optimization problem (1.1), the convergence condition of Algorithm 1 has less conservative than that in [17], wherein, the convergence of Algorithm 1 can be guaranteed if $P_i$ is symmetric and $P_i\succ \rho d_i{I}_m$ according to Remark 4.2, while algorithm in [17] requires that $P_i$ is a symmetric positive semi-definite matrix and $P_i\succ (\frac{n}{2-\gamma }-1)\rho A_i^T{A}_i=(\frac{n}{2-\gamma }-1)\rho d_i{I}_m(0<\gamma <2)$.