In Section 2, we first point out that the output of deep GCN with ReLU activation function will suffer from loss of rank problem under certain conditions and this can cause deep GCN lose expressive power. We then prove that Tanh is better at preserving the rank of the output and verify this claim with numerical tests. Then we find a way to deepen GCN in block Krylov form and propose snowball and truncated Krylov networks which perform better than state-of-the-arts (SOTA) model on semi-supervised node classification tasks on 3 benchmark datasets. Besides, we point out that finding a specifically tailored weight initialization scheme for GCNs can be an promising direction to address over-smoothing efficiently in Section 2.2. In Section 3, we first illustrate the insufficiency of the current homophily metrics and propose aggregation homophily based on a new similarity matrix. We then show the advantage of the new homophily metric over the existing ones on synthetic graph. Based on the similarity matrix, we define diversification distinguishability of a node and demonstrate why high-pass filters can help to address heterophily problem. To include both low-pass and high-pass in GNNs, we extend filterbank method and propose ACM and ACMII frameworks that can boost the performance of baseline GNNs on heterophilous graphs.

Suppose we have an undirected graph $\mathcal{G}=(\mathcal{V},\mathcal{E},A)$, where $\mathcal{V}$ is the node set with $\left|\mathcal{V}\right|=N$; $\mathcal{E}$ is the edge set without self-loop; $A\in {\mathbb{R}}^{N\times N}$ is the symmetric adjacency matrix with $A_{ij}=1$ if and only if $e_{ij}\in \mathcal{E}$, otherwise $A_{ij}=0$; D is the diagonal degree matrix, i.e.$D_{ii}={\sum }_j{A}_{ij}$ and ${\mathcal{N}}_i=\{j:e_{ij}\in \mathcal{E}\}$ is the neighborhood set of node i. A graph signal is a vector $\mathbf{x}\in {\mathbb{R}}^N$ defined on $\mathcal{V}$, where $x_i$ is defined on the node i. We also have a feature matrix $X\in {\mathbb{R}}^{N\times F}$ whose columns are graph signals and each node i has a corresponding feature vector $X_{i:}$ with dimension F, which is the i-th row of X. We denote $Z\in {\mathbb{R}}^{N\times C}$ as label encoding matrix, where $Z_{i:}$ is the one hot encoding of the label of node i and C is the total number of classes.

The (combinatorial) graph Laplacian is defined as $L=D-A$, which is a Symmetric Positive Semi-Definite (SPSD) matrix [7]. Its eigendecomposition gives $L=U\Lambda U^T$, where the columns of $U\in {\mathbb{R}}^{N\times N}$ are orthonormal eigenvectors, namely the graph Fourier basis, $\Lambda =\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_N)$ with ${\lambda }_1\le \cdots \le {\lambda }_N$, and these eigenvalues are also called frequencies. The graph Fourier transform of the graph signal $\mathbf{x}$ is defined as ${\mathbf{x}}_{\mathcal{F}}=U^{-1}\mathbf{x}=U^T\mathbf{x}={[{\mathit{u}}_1^T\mathbf{x},\dots ,{\mathit{u}}_N^T\mathbf{x}]}^T$, where ${\mathit{u}}_i^T\mathbf{x}$ is the component of $\mathbf{x}$ in the direction of ${\mathit{u}}_{\mathbf{i}}$.

Some graph Laplacian variants are commonly used, e.g. the symmetric normalized Laplacian $L_{\mathrm{sym}}=D^{-1/2}L{D}^{-1/2}=I-D^{-1/2}A{D}^{-1/2}$ and the random walk normalized Laplacian $L_{\mathrm{rw}}=D^{-1}L=I-D^{-1}A$. The eigenvalues of $L_{\mathrm{rw}}$ and $L_{\mathrm{sym}}$ are the same and are in $[0,2)$, and their corresponding eigenvectors satisfy ${\mathit{u}}_{\mathrm{rw}}^i=D^{-1/2}{\mathit{u}}_{\mathrm{sym}}^i$.

The affinity (transition) matrices can be derived from the Laplacians, e.g. $A_{\mathrm{rw}}=I-L_{\mathrm{rw}}=D^{-1}A$, $A_{\mathrm{sym}}=I-L_{\mathrm{sym}}=D^{-1/2}A{D}^{-1/2}$. Then ${\lambda }_i\left(A_{\mathrm{rw}}\right)={\lambda }_i\left(A_{\mathrm{sym}}\right)=1-{\lambda }_i\left(A_{\mathrm{sym}}\right)=1-{\lambda }_i\left(A_{\mathrm{rw}}\right)\in (-1,1]$. Renormalized affinity and Laplacian matrices are introduced in [24] as ${\widehat{A}}_{\mathrm{sym}}={\tilde{D}}^{-1/2}\tilde{A}{\tilde{D}}^{-1/2}$, ${\widehat{L}}_{\mathrm{sym}}=I-{\widehat{A}}_{\mathrm{sym}}$, where $\tilde{A}\equiv A+I,\tilde{D}\equiv D+I$ and it essentially adds a self-loop and is widely used in Graph Convolutional Network (GCN) as follows: where $W_0\in {\mathbb{R}}^{F\times F_1}$ and $W_1\in {\mathbb{R}}^{F_1\times O}$ are parameter matrices. GCN can learn by minimizing the following cross entropy loss

The random walk renormalized matrix ${\widehat{A}}_{\mathrm{rw}}={\tilde{D}}^{-1}\tilde{A}$ can also be applied to GCN and it has the same eigenvalues as ${\widehat{A}}_{\mathrm{sym}}$. The corresponding Laplacian is defined as ${\widehat{L}}_{\mathrm{rw}}=I-{\widehat{A}}_{\mathrm{rw}}$ Specifically, the nature of random walk matrix makes ${\widehat{A}}_{\mathrm{rw}}$ behaves as a mean aggregator ${\left({\widehat{A}}_{\mathrm{rw}}\mathbf{x}\right)}_i={\sum }_{j\in \{{\mathcal{N}}_i\cup i\}}{x}_j/(D_{ii}+1)$ which is applied in [17] and is important to bridge the gap between spatial- and spectral-based graph convolution methods.

Suppose we deepen GCN in the same way as [24,32], we have

For this architecture, without considering the ReLU activation function, [32] shows that $Y^{\prime }$ will converge to a space spanned by the eigenvectors of ${\widehat{A}}_{\mathrm{sym}}$ with eigenvalue 1. Taking activation function into consideration, our analyses on (3) can be summarized in the following theorems (see proof in the appendix of [44]).

Theorem 1 shows that, even considering ReLU, if we simply deepen GCN as (3), the extracted features will degrade under certain conditions, i.e. $Y^{\prime }$ only contains the stationary information of the graph structure and loses all the local information in node for being smoothed. In addition, from the proof we see that the pointwise ReLU transformation is a conspirator. Theorem 2 tells us that Tanh is better at keeping linear independence among column features. We design a numerical experiment on synthetic data to test, under a 100-layer GCN architecture, how activation functions affect the rank of the output in each hidden layer during the feedforward process. As Figure 1a shows, the rank of hidden features decreases rapidly with ReLU, while having little fluctuation under Tanh, and even the identity function performs better than ReLU. So we propose to replace ReLU by Tanh.

Besides activation function, to find a way to deepen GCN, we first show that any graph convolution with well-defined analytic spectral filter defined on ${\widehat{A}}_{\mathrm{sym}}\in {\mathbb{R}}^{N\times N}$ can be written as a product of a block Krylov matrix with a learnable parameter matrix in a specific form. Based on this, we propose snowball network and truncated Krylov network.

We take $\mathbb{S}={\mathbb{R}}^{F\times F}$. Given a set of block vectors ${\left\{X_k\right\}}_{k=1}^m\subset {\mathbb{R}}^{N\times F}$, the $\mathbb{S}$-span of ${\left\{X_k\right\}}_{k=1}^m$ is defined as ${\mathrm{span}}^{\mathbb{S}}\{X_1,\cdots ,X_m\}=\{\sum _{k=1}^m{X}_k{C}_k:C_k\in \mathbb{S}\}$. Then, the order-m block Krylov subspace with respect to the matrix $A\in {\mathbb{R}}^{N\times N}$, the block vector $B\in {\mathbb{R}}^{N\times F}$ and the vector space $\mathbb{S}$, and its corresponding block Krylov matrix are respectively defined as It is shown in [11,15] that there exists a smallest m such that for any $k\ge m$, $A^{k}B\in {\mathcal{K}}_m^{\mathbb{S}}(A,B)$, where m depends on A and B.

Let $\rho \left({\widehat{A}}_{\mathrm{sym}}\right)$ denote the spectrum radius of ${\widehat{A}}_{\mathrm{sym}}$ and suppose $\rho \left({\widehat{A}}_{\mathrm{sym}}\right)<R$ where R is the radius of convergence for a real analytic scalar function g. Based on the above definitions and conclusions, the graph convolution can be written as where ${\Gamma }_i^{\mathbb{S}}\in {\mathbb{R}}^{F\times F}$ for $i=1,\dots ,m-1$ are parameter matrix blocks and ${\Gamma }^{\mathbb{S}}\in {\mathbb{R}}^{mF\times F}$. Then, a graph convolutional layer can generally be written as where $W^{\prime }\in {\mathbb{R}}^{F\times O}$ is a parameter matrix, and $W^{\mathbb{S}}\equiv {\Gamma }^{\mathbb{S}}{W}^{\prime }\in {\mathbb{R}}^{mF\times O}$. The essential number of learnable parameters is $mF\times O$.

The block Krylov form provides an insight about why an architecture that concatenates multi-scale features in each layer will boost the expressive power of GCN. Based on this idea, we propose the snowball and truncated Block Krylov architectures [44] shown in Figure 2, where we stack multi-scale information in each layer. From the performance comparison on semi-supervised node classification tasks with different label percentage in Table 1, we can see that the proposed models consistently perform better than the state-of-the-art models, especially when there are less labeled nodes. See detailed experimental results in [44].

The metrics of homophily are defined by considering different relations between node labels and graph structures defined by adjacency matrix. There are three commonly used homophily metrics: edge homophily [1,70], node homophily [51], and class homophily [35] 4 defined as follows: where ${\left[a\right]}_{+}=max(a,0)$; $h_k$ is the class-wise homophily metric [35]. They are all in the range of $[0,1]$ and a value close to 1 corresponds to strong homophily while a value close to 0 indicates strong heterophily. $H_{\mathrm{edge}}\left(\mathcal{G}\right)$ measures the proportion of edges that connect two nodes in the same class; $H_{\mathrm{node}}\left(\mathcal{G}\right)$ evaluates the average proportion of edge-label consistency of all nodes; $H_{\mathrm{class}}\left(\mathcal{G}\right)$ tries to avoid the sensitivity to imbalanced class, which can cause $H_{\mathrm{edge}}$ misleadingly large. The above definitions are all based on the graph-label consistency and imply that the inconsistency will cause harmful effect to the performance of GNNs. With this in mind, we will show a counter example to illustrate the insufficiency of the above metrics and propose new metrics in the following subsection.

Heterophily is believed to be harmful for message-passing based GNNs [6,51,70] because intuitively features of nodes in different classes will be falsely mixed and this will lead nodes indistinguishable [70]. Nevertheless, it is not always the case, e.g. the bipartite graph shown in Figure 3 is highly heterophilous according to the homophily metrics in (20), but after mean aggregation, the nodes in classes 1 and 2 only exchange colors and are still distinguishable. Authors in [6] also point out the insufficiency of $H_{\mathrm{node}}$ by examples to show that different graph typologies with the same $H_{\mathrm{node}}$ can carry different label information.

To analyze to what extent the graph structure can affect the output of a GNN, we first simplify the GCN by removing its nonlinearity as [60]. Let $\widehat{A}\in {\mathbb{R}}^{N\times N}$ denote a general aggregation operator. Then, Equation (1) can be simplified as, After each gradient decent step $\Delta W=\gamma \frac{d\mathcal{L}}{dW}$, where $\gamma$ is the learning rate, the update of $Y^{\prime }$ will be, where $S(\widehat{A},X)\equiv \widehat{A}X{\left(\widehat{A}X\right)}^T$ is a post-aggregation node similarity matrix, $Z-Y$ is the prediction error matrix. The update direction of node i is essentially a weighted sum of the prediction error, i.e.$\Delta {\left(Y^{\prime }\right)}_{i,:}={\sum }_{j\in \mathcal{V}}{\left[S(\widehat{A},X)\right]}_{i,j}{(Z-Y)}_{j,:}$.

To study the effect of heterophily, we first define the aggregation similarity score as follows.

$S_{\mathrm{agg}}\left(S(\widehat{A},X)\right)$ measures the proportion of nodes $v\in \mathcal{V}$ that will put relatively larger similarity weights on nodes in the same class than in other classes after aggregation. It is easy to see that $S_{\mathrm{agg}}\left(S(\widehat{A},X)\right)\in [0,1]$. But in practice, we observe that in most datasets, we will have $S_{\mathrm{agg}}\left(S(\widehat{A},X)\right)\ge 0.5$. Based on this observation, we rescale (23) to the following modified aggregation similarity for practical usage,

In order to measure the consistency between labels and graph structures without considering node features and make a fair comparison with the existing homophily metrics in (20), we define the graph ($\mathcal{G}$) aggregation ($\widehat{A}$) homophily and its modified version as In practice, we will only check $H_{\mathrm{agg}}\left(\mathcal{G}\right)$ when $H_{\mathrm{agg}}^M\left(\mathcal{G}\right)=0$. As Figure 3 shows, when $\widehat{A}={\widehat{A}}_{\mathrm{rw}}$, $H_{\mathrm{agg}}\left(\mathcal{G}\right)=H_{\mathrm{agg}}^M\left(\mathcal{G}\right)=1$. Thus, this new metric reflects the fact that nodes in classes 1 and 2 are still highly distinguishable after aggregation, while other metrics mentioned before fail to capture the information and misleadingly give value 0. This shows the advantage of $H_{\mathrm{agg}}\left(\mathcal{G}\right)$ and $H_{\mathrm{agg}}^M\left(\mathcal{G}\right)$ by additionally considering information from aggregation operator $\widehat{A}$ and the similarity matrix.

We first consider the example shown in Figure 5. From $S(\widehat{A},X)$, nodes 1,3 assign relatively large positive weights to nodes in class 2 after aggregation, which will make node 1,3 hard to be distinguished from nodes in class 2. Despite the fact, we can still distinguish between nodes 1,3 and 4,5,6,7 by considering their neighborhood difference: nodes 1,3 are different from most of their neighbors while nodes 4,5,6,7 are similar to most of their neighbors. This indicates, in some cases, although some nodes become similar after aggregation, they are still distinguishable via their surrounding dissimilarities. This leads us to use diversification operation, i.e. high-pass (HP) filter $I-\widehat{A}$[10] (will be introduced in the next subsection) to extract the information of neighborhood differences and address harmful heterophily. As $S(I-\widehat{A},X)$ in Figure 5 shows, nodes 1,3 will assign negative weights to nodes 4,5,6,7 after diversification operation, i.e. nodes 1,3 treat nodes 4,5,6,7 as negative samples and will move away from them during backpropagation. Base on this example, we first propose diversification distinguishability as follows to measures the proportion of nodes that diversification operation is potentially helpful for,

We can see that ${\mathrm{DD}}_{\widehat{A},X}\left(\mathcal{G}\right)\in [0,1]$ . The effectiveness of diversification operation can be proved for binary classification problems under certain conditions based on definition 2, leading us to:

We discuss relevant work of GNNs on addressing heterophily challenge in this part. Authors in [1] acknowledge the difficulty of learning on graphs with weak homophily and propose MixHop to extract features from multi-hop neighborhood to get more information. Geom-GCN [51] precomputes unsupervised node embeddings and uses graph structure defined by geometric relationships in the embedding space to define the bi-level aggregation process. Authors in [20] propose measurements based on feature smoothness and label smoothness that are potentially helpful to guide GNNs on dealing with heterophilous graphs. H${}_2$GCN [70] combines 3 key designs to address heterophily: (1) ego- and neighbor-embedding separation; (2) higher-order neighborhoods; (3) combination of intermediate representations. CPGNN [69] models label correlations by the compatibility matrix, which is beneficial for heterophily settings, and propagates a prior belief estimation into GNNs by the compatibility matrix. FBGNN [47] first proposes to use filterbank to address heterophily problem, but it does not fully explain the insights behind HP filters and does not contain identity channel and node-wise channel mixing mechanism. FAGCN [2] learns edge-level aggregation weights as GAT [58] but allows the weights to be negative which enables the network to capture the high-frequency components in graph signals. GPRGNN [6] uses learnable weights that can be both positive and negative for feature propagation, it allows GRPGNN to adapt heterophily structure of graph and is able to handle both high- and low-frequency parts of the graph signals.

MDP is a framework to model the learning process that the agent learns from the interaction with the environment[56,67,68]. The interaction happens in discrete time steps, $t=0,1,2,3,\cdots$. At step t, given a state $S_t=s_t\in \mathcal{S}$, the agent picks an action $a_t\in \mathcal{A}\left(s_t\right)$ according to a policy $\pi (·|s_t)$, which is a rule of choosing actions given a state. Then, at time $t+1$, the environmental dynamics $p:\mathcal{S}\times \mathcal{R}\times \mathcal{A}\times \mathcal{S}\to [0,1]$ take the agent to a new state $S_{t+1}=s_{t+1}\in \mathcal{S}$ and provide a numerical reward $R_{t+1}=r_{t+1}(s_t,a_t,s_{t+1})\in \mathbb{R}$. Such a sequence of interaction gives us a trajectory $\tau =\{S_0,A_0,R_1,S_1,A_1,R_2,S_2,A_2,R_3,\cdots \}$. The objective is to find an optimal policy to maximize the expected long-term discounted cumulative reward $V_{\pi }\left(s\right)=E_{\pi }\left[\sum _{k=0}^{\infty }{\gamma }^k{R}_{t+k+1}\right|S_t=s]$ for each state s, where $\gamma$ is the discount factor.

For a given policy $\pi$, solving its value function ${\mathbf{V}}_{\pi }$ is equivalent to solving the following linear system, where ${\mathbf{V}}_{\pi }={\left[V_{\pi }\left(s\right)\right]}_{s\in \mathcal{S}}^T\in {\mathbb{R}}^{\left|\mathcal{S}\right|},{\mathbf{r}}_{\pi }={\left[r_{\pi }\left(s\right)\right]}_{s\in \mathcal{S}}^T\in {\mathbb{R}}^{\left|\mathcal{S}\right|},P_{\pi }={\left[P_{\pi }\left(s^{\prime }\right|s)\right]}_{s^{\prime },s\in \mathcal{S}}\in {\mathbb{R}}^{\left|\mathcal{S}\right|\times \left|\mathcal{S}\right|}$. The state transition matrix $P_{\pi }$ essentially defines a graph structure over states and the reward vector ${\mathbf{r}}_{\pi }$ is a signal defines on graph. Thus, solving value function can be considered as a (supervised or semi-supervised) node regression tasks over graph. Besides solving ${\mathbf{V}}_{\pi }$, the graph structure can also be used for reward propagation and representation learning in Reinforcement Learning (RL) [26,27,28].

Treating MDP as a graph is an old but never outdated idea. Traditional methods use graph Laplacian for a fixed policy to estimate ${\mathbf{V}}_{\pi }$, e.g. proto-value function [49]. In addition to value function estimation, [28] proposes to use GCN to learn potential-based reward shaping, which can accelerate the learning process of the agent.

The above methods both construct the graph from the sampled trajectory data. With modern Graph Representation Learning (GRL) methods e.g. node embedding methods [3], link prediction methods [52,54,57], we can learn to reconstruct the underlying graph (adjacency matrix) from sampled data more efficiently. And label propagation [53], which is a commonly used algorithm for graph semi-supervised learning, can be helpful for efficient reward propagation. In Section 4.3, we will introduce the potential of using GRL for reward propagation and representation learning in reinforcement learning.

In this section, we will draw how to represent Markov Decision Process (MDP) with graph and introduce two possible ways of using graph representation learning to address the problems defined on MDP.

Each state can be treated as a node on a graph, the transition probability between each pair of nodes (an element in state transition matrix) can be represented by the edge (or weight) between them and value function is a function defined on each node of the graph. The details (for finite MDP) are introduced in matrix form as follows [38,59]: