1. Introduction

2. Materials and Methods

2.3. Methodology

In the Figure 3 We now explain our proposed method for the classification of ASD. The proposed model consists of three modules which we call as Features Module , Data Augment Module and a Classification Module. The general flow of the proposed approach is as follows , Firstly , the connectivity features will be calculated for both class of subjects . Secondly, The connectivity features will be used to train a Generative Adversarial Network to generate synthetic features and class per subjects so that the problem of over-fitting while training the DL model be reduced . Finally , A classification module will be used for the classification of subjects to ASD and HC using the Multi-Head attention module.

In the following subsections , We briefly describe each the three modules that were shown in the Figure 3.

2.3.1. Features Module

The purpose of the features module as shown in Figure 4 is to convert the time series data on the brain regions across subjects to a fixed length dimension that can be processed by the Data Augment Module and subsequently by the Classification Module. The time series data on the 116 brain regions vary sometimes within site and also across sites and tackling the varying time series data is a significant challenge because taking a fixed size window for the time series across subjects not only violates the inherent structure of the pre-processed fMRI but also limit generalization over the subjects. To tackle this challenge we have used a Pearson’s correlation "r" as given in Eq 1 to measure the functional connectivity between any two brain regions "x" and "y". Since we are using the AAL-116 template in the study where each subject has 116 brain regions so the correlation will result in a symmetric 116x116 matrix from which we extracted the upper diagonal "6670" values excluding the diagonal because the diagonal is always 1 as the correlation between the two same brain regions will always be 1. Therefore for each of the subjects whether ASD or HC we will have a "6670" dimensional connectivity features vector which will be used subsequently in the rest of the modules.

$$r={\displaystyle \frac{{\sum }_{i=1}^n(x_i-\overline{x})(y_i-\overline{y})}{\sqrt{{\sum }_{i=1}^n{(x_i-\overline{x})}^2{\sum }_{i=1}^n{(y_i-\overline{y})}^2}}}$$

(1)

The motivations of using the functional connectivity features is deeply rooted in how the brain actually performs a task as it has been shown with significant evidence that brain regions when doing a task work in pairs and functional connectivity has proved to be associated with cognition and motor control in human beings [29,30].

2.3.2. Data Augment Module

The purpose of the Data Augment Module is to generate synthetic 6670 dimensional features across ASD and HC because the size of dataset corresponding to the features dimension is very less leading to the over-fitting of the DL models. We have used Generative Adversarial Networks (GAN) [31] based approach to generated synthetic features across subjects and since we want not only features but also want to condition the features across the subjects so we conditioned our GANs based on the subject condition that is either belonging to ASD or HC. The objective function of the Conditional GAN (CGAN) as explained in [32] is given in Eq 2 . Where "G" and "D" are the Generator and Discriminator , "x" is the features vector , "y" is the class of the subject and "z" is the random vector . The main purpose of the GAN minmax approach is to fool the discriminator over time by the generator by generating subjects as close to the real data. .

$$\underset{G}{min}\underset{D}{max}V(D,G)={\mathbb{E}}_{x\sim p_{data}\left(x\right)}[logD\left(x\right|y)]+{\mathbb{E}}_{z\sim p_z\left(z\right)}[log(1-D\left(G\left(z\right|y)\right))]$$

(2)

Figure 5. Data Augment Module.

Figure 5. Data Augment Module.

The motivations of using the CGAN based approach is to overcome the problem of using a fixed size subject time series data and to minimize the problem of over-fitting that is faced when training the DL model.

2.3.3. Classification Module

In the Classification Module in Figure 6 we have shown that how the final objective of the classification will be achieved. Here first we will have features and associated labels for the original and synthetic ASD and HC subjects and then a Multi-Head attention [33] based module will be used so that focus is on the relevant features followed by a fully connected layer and a classification head.The Multi-Head attention based module is designed using the Eq 3 where "Q", "K" and "V" are called the query, key and value and "d" is normalization constant.

$$Attention(Q,K,V)=softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(3)

2.3.4. Algorithm

In the following section we explain the proposed approach using an algorithm as written in the Algorithm 1 so that the reader has a clear mind map of the step-wise working of the proposed approach.

Algorithm 1: Proposed GAN Powered Multi-Head Attention Approach’s Algorithm

1: Input ← D , Output ← {} , ganEpochs, synthEpochs , classEpochs   2: X,Y ← D (X ← Features Matrix , Y = ASD/HC)   3: $X_{train}$, $X_{test}$, $Y_{train}$, $Y_{test}$ ← Train-Test-Split(X,Y)   4: $F_{train}$ ← {}, $F_{test}$ ← {}   5: (Features Module):   6: for$x_{train}$, $x_{test}$, $y_{train}$, $y_{test}$ in ($X_{train}$,$X_{test}$,$T_{train}$,$Y_{test}$) do   7:     $subjectTrai{n}_{116xT}$ =$x_{train}$, $subjectTes{t}_{116xT}$ =$x_{test}$   8:     $Trai{n}_{116x116}$ ← PearsonCorr($subjectTrai{n}_{116xT}$)   9:     $Tes{t}_{116x116}$ ← PearsonCorr($subjectTes{t}_{116xT}$)   10:     $Trai{n}_{6670}$ ← UpperTria($Trai{n}_{116x116}$)   11:     $Tes{t}_{6670}$ ← UpperTria($Tes{t}_{116x116}$)   12:     $F_{train}$ ← $F_{train}$U { $Trai{n}_{6670}$ ,$y_{train}$ }   13:     $F_{test}$ ← $F_{test}$U { $Tes{t}_{6670}$ ,$y_{test}$ }   14: end for  15: (Data Augment Module):   16: $F_{synthetic}$ ← {}   17: for epoch in ganEpochs do   18:     $Dat{a}_{random}$,$labe{l}_{random}$ ←  Random()   19:      $F_{real}$,$Y_{real}$ ← Generator($Dat{a}_{random}$,$labe{l}_{random}$)   20:     $F_{real}$,$Y_{real}$ ← Subset($F_{train}$)   21:     Discriminator($F_{fake}$,$Y_{fake}$)   22:     Discriminator($F_{real}$,$Y_{real}$)   23: end for  24: for epoch in synthEpochs do   25:      $D_{synthetic}$,$labe{l}_{synthetic}$ ←  Generator()   26:     $F_{synthetic}$ ← $F_{synthetic}$U { $D_{synthetic}$ ,$labe{l}_{synthetic}$ }   27: end for  28: (Classification Module):   29: for epoch in classEpochs do   30:      $F_1$ ← MultiHead($F_{train}$,$F_{synthetic}$)   31:      $F_2$ ← FC1($F_1$)   32:      $F_3$ ← FC2($F_2$)   33:      $F_4$ ← FC3($F_3$)   34:      $Mode{l}_{trained}$ ← Class($F_4$)   35: end for  36: Output ← $Mode{l}_{trained}$($F_{test}$)

The proposed algorithm works by taking an input as the subject data and associated label and output the accuracy of the model on the testing dataset. Algorithm is divided into the three sub modules which we cal as Features Module , Data Augment Module and the Classification Module. The function "Train-Test-Split" as described in the line 3 is used to partition the dataset into the training and testing dataset by considering stratification. The function "PearsonCorr" is the correlation coefficient as mentioned in Eq 1 and the function "UpperTria" is used to extract the unique values from the symmetric 116 x 116 connectivity matrix. The function "Generator" and "Discriminator" are the neural networks used to train the CGAN in the adversarial manner. The function "MultiHead" , "FC’s" and "Class" are the multi-head based attention layer , fully connected layers and the classification layer of the Classification Module.

3. Results

3.2. Experiments

There are two kinds of experiments that we conducted for the current work , An overall accuracy comparison where the whole dataset was first divided into the training and testing split using stratification and then the model got trained on the combined training data and then tested on the combined testing dataset. A site wise comparison where we used the largest site NYU for the training , trained the model on this site and then tested on the remaining sites but excluding the site CMU because it contained too few subjects to compare.

3.2.1. Overall Comparison

In the first experiment we compared the results of the proposed approach with the state-of-the-art approaches as shown in Table 2. It can be seen that our proposed approach accuracy , precision and recall have outperformed the other approaches which validates the effectiveness of the proposed approach on the whole data.

3.2.2. Site Wise Comparison

In the second comparison we used the "NYU" training total dataset as the training dataset and tested the trained model on the rest of the 15 sites excluding the "CMU" site as done in [21] . We have shown the results of the proposed approach with the state-of-the-art approaches and it can be seen that our proposed approach has outperformed the other approaches in 10 out of the 15 sites validating the effectiveness of the proposed approach.

Table 3. Site wise dataset comparison with the State-of-the-Art(%).

Table 3. Site wise dataset comparison with the State-of-the-Art(%).

	Site	Size	MHSA[21]	MVES[22]	ASD-GANNet
1	CALTECH	37	64.60	71.40	71.60
2	KKI	39	79.60	75.00	75.40
3	LEUVEN	61	70.40	72.10	73.20
4	MaxMun	42	66.40	61.50	66.90
5	OUSH	23	66.00	-	60.00
6	Olin	25	76.00	73.50	76.00
7	Pitt	45	65.80	74.50	73.00
8	SBL	26	89.30	64.30	65.00
9	SDSU	33	75.70	79.30	79.60
10	Stanford	36	77.50	69.20	70.00
11	Trinity	44	73.10	73.30	74.00
12	UCLA	75	69.30	77.60	78.50
13	USM	113	76.90	87.30	88.10
14	UM	61	74.90	75.70	75.90
15	Yale	48	75.10	80.00	81.10

4. Discussion

5. Conclusions

Abbreviations

References

1. American Psychiatric Association, D.; Association, A.P.; others. Diagnostic and statistical manual of mental disorders: DSM-5; Vol. 5, American psychiatric association Washington, DC, 2013.
2. Lord, C.; Elsabbagh, M.; Baird, G.; Veenstra-Vanderweele, J. Autism spectrum disorder. The lancet 2018, 392, 508–520. [Google Scholar]
3. Hirota, T.; King, B.H. Autism spectrum disorder: A review. Jama 2023, 329, 157–168. [Google Scholar]
4. Gabbay-Dizdar, N.; Ilan, M.; Meiri, G.; Faroy, M.; Michaelovski, A.; Flusser, H.; Menashe, I.; Koller, J.; Zachor, D.A.; Dinstein, I. Early diagnosis of autism in the community is associated with marked improvement in social symptoms within 1–2 years. Autism 2022, 26, 1353–1363. [Google Scholar]
5. Iyama-Kurtycz, T.; Iyama-Kurtycz, T. Obstacles to diagnosing autism spectrum disorder and myths about autism. Diagnosing and Caring for the Child with Autism Spectrum Disorder: A Practical Guide for the Primary Care Provider 2020, pp. 9–17.
6. McCarty, P.; Frye, R.E. Early detection and diagnosis of autism spectrum disorder: Why is it so difficult? Seminars in Pediatric Neurology. Elsevier, 2020, Vol. 35, p. 100831.
7. Soares, J.M.; Magalhães, R.; Moreira, P.S.; Sousa, A.; Ganz, E.; Sampaio, A.; Alves, V.; Marques, P.; Sousa, N. A Hitchhiker’s guide to functional magnetic resonance imaging. Frontiers in neuroscience 2016, 10, 218898. [Google Scholar]
8. Chow, M.S.; Wu, S.L.; Webb, S.E.; Gluskin, K.; Yew, D. Functional magnetic resonance imaging and the brain: A brief review. World journal of radiology 2017, 9, 5. [Google Scholar]
9. Lajoie, I.; Nugent, S.; Debacker, C.; Dyson, K.; Tancredi, F.B.; Badhwar, A.; Belleville, S.; Deschaintre, Y.; Bellec, P.; Doyon, J.; others. Application of calibrated fMRI in Alzheimer’s disease. NeuroImage: Clinical 2017, 15, 348–358. [Google Scholar]
10. Yap, K.H.; Manan, H.A.; Sharip, S. Heterogeneity in brain functional changes of cognitive processing in ADHD across age: a systematic review of task-based fMRI studies. Behavioural Brain Research 2021, 397, 112888. [Google Scholar]
11. Santana, C.P.; de Carvalho, E.A.; Rodrigues, I.D.; Bastos, G.S.; de Souza, A.D.; de Brito, L.L. rs-fMRI and machine learning for ASD diagnosis: A systematic review and meta-analysis. Scientific reports 2022, 12, 6030. [Google Scholar]
12. Guo, L.; Bai, G.; Zhang, H.; Lu, D.; Zheng, J.; Xu, G. Cognitive functioning in temporal lobe epilepsy: a BOLD-fMRI study. Molecular neurobiology 2017, 54, 8361–8369. [Google Scholar]
13. Casteràs, D.P.; Cano, M.; Camprodon, J.; Loo, C.; Palao, D.; Soriano-Mas, C.; Cardoner, N. A multimetric systematic review of fMRI findings in patients with MDD receiving ECT. Eur Neuropsychopharmacol 2021, 44, S47. [Google Scholar] [CrossRef]
14. Wolters, A.F.; van de Weijer, S.C.; Leentjens, A.F.; Duits, A.A.; Jacobs, H.I.; Kuijf, M.L. “Resting-state fMRI in Parkinson’s disease patients with cognitive impairment: A meta-analysis”: Answer to Wang and colleagues. Parkinsonism & Related Disorders 2019, 66, 253–254. [Google Scholar]
15. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
16. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Communications of the ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
17. Nazir, M.; Shakil, S.; Khurshid, K. Role of deep learning in brain tumor detection and classification (2015 to 2020): A review. Computerized medical imaging and graphics 2021, 91, 101940. [Google Scholar]
18. Lavecchia, A. Deep learning in drug discovery: opportunities, challenges and future prospects. Drug discovery today 2019, 24, 2017–2032. [Google Scholar] [CrossRef]
19. Lauriola, I.; Lavelli, A.; Aiolli, F. An introduction to deep learning in natural language processing: Models, techniques, and tools. Neurocomputing 2022, 470, 443–456. [Google Scholar]
20. Yang, J.; Hu, M.; Hu, Y.; Zhang, Z.; Zhong, J. Diagnosis of Autism Spectrum Disorder (ASD) Using Recursive Feature Elimination–Graph Neural Network (RFE–GNN) and Phenotypic Feature Extractor (PFE). Sensors 2023, 23, 9647. [Google Scholar] [CrossRef] [PubMed]
21. Zhao, F.; Feng, F.; Ye, S.; Mao, Y.; Chen, X.; Li, Y.; Ning, M.; Zhang, M. Multi-head self-attention mechanism-based global feature learning model for ASD diagnosis. Biomedical Signal Processing and Control 2024, 91, 106090. [Google Scholar]
22. Kang, L.; Chen, J.; Huang, J.; Jiang, J. Autism spectrum disorder recognition based on multi-view ensemble learning with multi-site fMRI. Cognitive Neurodynamics 2023, 17, 345–355. [Google Scholar] [CrossRef]
23. Zhang, F.; Wei, Y.; Liu, J.; Wang, Y.; Xi, W.; Pan, Y. Identification of Autism spectrum disorder based on a novel feature selection method and Variational Autoencoder. Computers in Biology and Medicine 2022, 148, 105854. [Google Scholar] [CrossRef]
24. Wang, N.; Yao, D.; Ma, L.; Liu, M. Multi-site clustering and nested feature extraction for identifying autism spectrum disorder with resting-state fMRI. Medical image analysis 2022, 75, 102279. [Google Scholar]
25. Cao, M.; Yang, M.; Qin, C.; Zhu, X.; Chen, Y.; Wang, J.; Liu, T. Using DeepGCN to identify the autism spectrum disorder from multi-site resting-state data. Biomedical Signal Processing and Control 2021, 70, 103015. [Google Scholar]
26. Di Martino, A.; Yan, C.G.; Li, Q.; Denio, E.; Castellanos, F.X.; Alaerts, K.; Anderson, J.S.; Assaf, M.; Bookheimer, S.Y.; Dapretto, M.; others. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Molecular psychiatry 2014, 19, 659–667. [Google Scholar]
27. Craddock, C.; Benhajali, Y.; Chu, C.; Chouinard, F.; Evans, A.; Jakab, A.; Khundrakpam, B.S.; Lewis, J.D.; Li, Q.; Milham, M.; others. The neuro bureau preprocessing initiative: open sharing of preprocessed neuroimaging data and derivatives. Frontiers in Neuroinformatics 2013, 7, 5. [Google Scholar]
28. Korhonen, O.; Saarimäki, H.; Glerean, E.; Sams, M.; Saramäki, J. Consistency of regions of interest as nodes of fMRI functional brain networks. Network Neuroscience 2017, 1, 254–274. [Google Scholar]
29. Lips, M.A.; Wijngaarden, M.A.; van der Grond, J.; van Buchem, M.A.; de Groot, G.H.; Rombouts, S.A.; Pijl, H.; Veer, I.M. Resting-state functional connectivity of brain regions involved in cognitive control, motivation, and reward is enhanced in obese females. The American journal of clinical nutrition 2014, 100, 524–531. [Google Scholar]
30. Bijsterbosch, J.D.; Woolrich, M.W.; Glasser, M.F.; Robinson, E.C.; Beckmann, C.F.; Van Essen, D.C.; Harrison, S.J.; Smith, S.M. The relationship between spatial configuration and functional connectivity of brain regions. elife 2018, 7, e32992. [Google Scholar]
31. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial networks. Communications of the ACM 2020, 63, 139–144. [Google Scholar] [CrossRef]
32. Mirza, M.; Osindero, S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, arXiv:1411.1784 2014.
33. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Advances in neural information processing systems 2017, 30. [Google Scholar]