1. Introduction

2. Literature Survey

3. Methodology

3.3. Feature Extraction

The effectiveness of our spoken Kashmiri recognition system is grounded in the advanced data processing techniques employed, particularly the dual feature extraction and spectrogram augmentation. High-quality and diverse datasets are crucial for developing robust models that generalize to new data. Our approach to optimizing the dataset includes carefully crafted steps such as dual feature extraction, spectrogram augmentation, and standard scaling to enhance model performance.

3.3.1. Existing Methods:

The use of Mel-Frequency Cepstral Coefficients (MFCCs) is well-established in the field of speech recognition due to their effectiveness in capturing the power spectrum of audio signals [10]. However, despite their popularity, MFCCs exhibit certain limitations, particularly in noisy environments. Their focus on short-term power spectra can result in a loss of temporal dynamics and transient characteristics, which are crucial for accurate speech recognition in languages with complex phonetic structures, such as Kashmiri [34,35]. Moreover, MFCCs’ susceptibility to background noise often degrades the performance of speech recognition models, necessitating the exploration of alternative or supplementary feature extraction methods.

Recent advancements have addressed these limitations by combining MFCCs with other feature extraction techniques. For example, the integration of Principal Component Analysis (PCA) with MFCCs has enhanced accuracy and reduced data dimensionality in Indonesian speech recognition systems [34]. Similarly, the use of Mel-Spectrograms, which provide a detailed representation of both spectral and temporal variations, has gained traction in conjunction with deep learning models like Convolutional Neural Networks (CNNs) [36]. The combination of Constant-Q Transform (CQT) with Mel-Spectrograms has also been proposed to capture more intricate spectral features, demonstrating potential improvements over traditional MFCC methods [37].

3.3.2. Challenges and Limitations:

Despite these innovations, several challenges persist. The hybrid approaches that combine MFCCs with deep learning models, such as CNNs, often require significant computational resources, limiting their applicability in real-time or resource-constrained environments [38]. Additionally, methods that rely solely on Mel-Spectrograms or similar features may offer rich spectral information but often at the cost of computational efficiency and temporal precision. This trade-off highlights the need for a balanced approach that leverages the strengths of both MFCCs and spectrogram-based features while mitigating their respective weaknesses.

3.3.3. Rationale for Our Approach:

Given the challenges identified in existing methodologies, we propose a dual feature extraction approach that combines MFCCs with partial Mel-Spectrograms. Our approach is specifically designed to balance the spectral richness provided by Mel-Spectrograms with the temporal precision of MFCCs, while also addressing the computational efficiency concerns. Unlike traditional Mel-Spectrogram techniques that utilize the full frequency range, our method focuses on specific Mel bands most relevant to the phonetic characteristics of the Kashmiri language. This targeted extraction reduces dimensionality and computational load, capturing the most critical spectral information without compromising efficiency.

Our experiments tested multiple Mel frequency bands for the partial Mel-Spectrogram, including several combinations from low to high Mel bins. These experiments involved testing ranges like 10-50 Mel, 30-60 Mel, and 50-120 Mel bins. Among these, the range of 20-60 Mel bins consistently provided the highest accuracy. This particular range proved optimal for capturing Kashmiri’s critical phonetic elements, leading to an accuracy rate of 86% in our Random Forest classifier. This performance indicates the effectiveness of our dual feature extraction approach, balancing the spectral richness of the Mel-Spectrogram with the temporal precision of MFCCs.

$$\mathrm{Partial}\mathrm{Mel-Spectrogram}(t,f)=20{log}_{10}\left({\left|\mathrm{STFT}\left(x\right(t\left)\right)·M\left(f\right)\right|}^2\right)[f_{\mathrm{start}}:f_{\mathrm{end}}]$$

(1)

where $\mathrm{STFT}\left(x\right(t\left)\right)$ is the Short-Time Fourier Transform of the signal $x\left(t\right)$ and $M\left(f\right)$ represents the Mel filter bank. This partial representation is critical for understanding speech’s tonal and dynamic elements, particularly in capturing the subtle phonetic nuances of Kashmiri [39]. The choice of the 20-60 Mel bin range was informed by extensive empirical testing, demonstrating that this range effectively captures the tonal variations distinctive to the Kashmiri language. While MFCCs offer a strong foundation for capturing the cepstral characteristics of speech, their susceptibility to noise and loss of temporal detail are well-documented concerns [9]. By combining MFCCs with partial Mel-Spectrograms, we aim to balance these limitations with the strengths of Mel-Spectrograms, resulting in a more robust feature set. The final feature vector is constructed by stacking the partial Mel-Spectrogram and MFCC features vertically, as shown below:

$${\mathbf{F}}_{\mathrm{combined}}=\left[\begin{array}{c}{\mathbf{F}}_{\mathrm{Partial}\mathrm{Mel-Spectrogram}}\\ {\mathbf{F}}_{\mathrm{MFCC}}\end{array}\right]$$

(2)

This vertical stacking effectively captures both spectral and temporal variations by treating them as sequential layers, significantly improving the robustness and accuracy of the speech recognition system. By integrating the temporal precision of MFCCs with the spectral richness provided by the partial Mel-Spectrogram, our approach ensures that the most critical acoustic features are preserved and emphasized. This layered representation allows the model to leverage complementary strengths from both feature types: MFCCs contribute to capturing the fine-grained temporal dynamics, which are essential for distinguishing between phonemes that may be temporally close but spectrally different, while the partial Mel-Spectrogram provides a detailed view of the harmonic structure and formants, which are crucial for recognizing vowel quality and tonal variations.

Figure 3 presents the distinct visual patterns captured by the partial Mel-Spectrogram and MFCC features. The partial Mel-Spectrogram highlights detailed spectral properties over a specific Mel bin range, crucial for distinguishing nuanced phonetic elements. This focused frequency analysis is particularly effective in capturing the complex tonal variations inherent in the Kashmiri language. To ensure that our approach was effective and appropriate, we conducted a comparative analysis of different feature extraction techniques, including full-range Mel-Spectrograms and the combination of MFCCs and full Mel-Spectrograms. The results of this analysis indicate that while these alternative methods provided valuable insights, the partial Mel-Spectrogram combined with MFCCs offered superior performance in terms of both accuracy and computational efficiency, particularly for the phonetic challenges presented by the Kashmiri language. The effectiveness of such hybrid approaches in various domains has been well-documented, further validating our methodology [40].

Table 3. Performance comparison of different feature sets and models using various classifiers.

Table 3. Performance comparison of different feature sets and models using various classifiers.

Feature Set	Accuracy	Precision	Recall
MFCC + Partial Mel-Spectrogram (Random Forest)	0.86	0.87	0.87
MFCC + Partial Mel-Spectrogram (Ridge Classifier)	0.80	0.83	0.82
MFCC + Partial Mel-Spectrogram (K-Nearest Neighbors)	0.61	0.61	0.62
MFCC + Partial Mel-Spectrogram (Decision Tree)	0.77	0.77	0.78
MFCC only (Random Forest)	0.79	0.80	0.80
Mel-Spectrogram only (Random Forest)	0.75	0.77	0.77

The complementary nature of MFCC and partial Mel-Spectrogram features is underscored by their respective entropy values, with the partial Mel-Spectrogram capturing more detailed and diverse spectral information [41]. This dual feature extraction approach not only improves the accuracy of the recognition system but also enhances its robustness, enabling the model to generalize effectively across different speakers and conditions. We combined MFCC and partial Mel-Spectrogram features with spectrogram augmentation techniques, such as Gaussian noise and time-stretching, to enhance model robustness. These augmentations introduced variability, allowing the model to generalize better across different acoustic environments and addressing dataset limitations [42?, 45]. After augmentation, standard scaling was applied to normalize feature distributions, ensuring comparability across feature types and improving the stability and accuracy of the model during training [42,43]. By leveraging the strengths of both MFCC and partial Mel-Spectrogram features, our approach offers a significant advancement in the accuracy and generalization of the Kashmiri speech recognition model, providing a solid foundation for further research and development in this field.

3.4. Deep Learning Models

3.4.1. Convolutional Neural networks (CNN)

A meticulously organized array of acoustic feature values represented mathematically as $\mathbf{Z}\in {\mathbb{R}}^{c\times b\times f}$, encompasses an intricate configuration characterized by a specific number of channels denoted as c, an expansive frequency bandwidth articulated as b, and a determined time length expressed as f. Within this framework, the convolutional layer embarks on a transformative journey by convolving the aforementioned $\mathbf{Z}$ with an array of k distinct filters, denoted collectively as ${\left\{{\mathbf{W}}_i\right\}}_k$, where each individual filter ${\mathbf{W}}_i$ is formulated as a $3D$ tensor residing in the realm of ${\mathbb{R}}^{c\times m\times n}$, showcasing a width along the frequency dimension that is precisely m and a length along the frame dimension that is accurately n. The outcome of this convolutional endeavour results in k preactivation feature maps that merge into a singular 3D tensor, denoted as $\mathbf{C}\in {\mathbb{R}}^{k\times b_C\times f_C}$, with each feature map ${\mathbf{C}}_i$ being meticulously computed through a specific mathematical operation, where the symbol ∗ serves as the conduit for the convolution operation, and $b_i$ signifies a bias parameter that nuances the output.

$${\mathbf{C}}_i={\mathbf{W}}_i\ast \mathbf{Z}+b_i,i=1,\cdots ,k$$

(3)

It is essential to highlight three pivotal points that warrant attention: first, the sequence length $f_C$ of the tensor C post-convolution is meticulously ensured to be congruent with the sequence length f of the input matrix Z by implementing a strategic zero-padding technique along the frame axis before the commencement of each convolution operation; second, in our model, the convolution stride is deliberately selected to be 1, ensuring a consistent and fluid convolutional process across all operations; and third, we consciously opt against the application of limited weight sharing, which typically partitions frequency bands into discrete groups of restricted bandwidths, instead favouring a holistic convolutional approach that traverses $\mathbf{Z}$ along both the frequency axis and the time axis, thereby culminating in a straightforward 2D convolution that is widely recognized and utilized in the domain of computer vision.

$${\tilde{\mathbf{C}}}_i=max\left(0,{\mathbf{C}}_i\right)$$

(4)

The pre-activation feature maps, denoted as $\mathbf{C}$, undergo a transformative process where they are subjected to a variety of nonlinear activation functions that significantly alter their values and representation. In the subsequent sections, we will introduce a trio of distinct activation functions, elucidating their unique functionalities within the context of a convolutional layer, and it is imperative to highlight that all the operations we will describe below are executed on an element-wise basis for clarity and precision. The Rectifier Linear Unit, commonly referred to as ReLU, is a piecewise linear activation function characterized by its behaviour of yielding a value of zero whenever the input falls below zero while simultaneously returning the input itself when it is non-negative. Formally, if we consider a single feature map represented as ${\mathbf{C}}_i$, the definition of a ReLU function can be articulated as follows:

$${\left[{\widehat{\mathbf{C}}}_i\right]}_{r,\ell }=\underset{j=1}{\overset{p}{max}}\left\{{\left[{\tilde{\mathbf{C}}}_i\right]}_{r\times s+j,t}\right\}$$

(5)

Following the intricate process of applying these element-wise nonlinearities, the resulting features are then directed into a max-pooling layer, which is responsible for extracting the maximum value from a set of p adjacent units, thereby summarizing the most salient information. We focus our pooling operations exclusively along the frequency axis, as this approach has been shown to effectively mitigate spectral variations within the same speaker’s output and in comparisons between the outputs of different speakers, as supported by previous research findings.

Notably, pooling along the temporal axis has been demonstrated to yield less significant benefits, indicating a preference for frequency-based pooling methods. In particular, let us denote the i-th feature map prior to pooling as ${\mathbf{C}}_i$ and its post-pooling counterpart as ${\widehat{\mathbf{C}}}_i$, then the value at position $(r,t)$ in the pooled feature map, denoted as ${\left[{\widehat{\mathbf{C}}}_i\right]}_{r,t}$, is computed based on the following criteria: where the variable s refers to the step size utilized in the pooling operation, and p represents the size of the pooling window, ensuring that all the values ${\left[{\tilde{\mathbf{C}}}_i\right]}_{r\times s+j,t}$ involved in the max calculation share the same temporal index t. Consequently, it is essential to note that the feature maps that emerge from the pooling process maintain identical sequence lengths when compared to their precursory versions before the pooling operation was applied, thereby preserving the overall structure of the data while enhancing its representative quality.

3.4.2. Gated Multi-Layer Perceptrons (gMLPs)

Liu et al. have unveiled a groundbreaking network architecture known as gMLP [44], which is ingeniously built upon Multi-Layer Perceptrons (MLPs) principles while ingeniously integrating sophisticated gating mechanisms that enhance its functionality and adaptability. Their extensive research and experimentation compellingly illustrated that the performance of gMLP can stand shoulder to shoulder with that of the widely recognized Transformers when applied to vital language and vision tasks that hold significant importance in the field of artificial intelligence. Interestingly, the results of their empirical analyses have brought to light the fascinating notion that self-attention, a hallmark feature of Vision Transformers, may not actually be a critical requirement for achieving high performance, as gMLP manages to reach comparable levels of accuracy without it. In a direct comparison with the BERT model, the gMLP architecture not only matches the performance of Transformers in terms of pretraining perplexity but also demonstrates superior capabilities in certain downstream assignments related to NLP [45], showcasing its versatility and effectiveness. Overall, the empirical findings derived from their comprehensive studies strongly suggest that gMLP possesses a remarkable scalability that is comparable to that of Transformers, especially when considering the utilization of expanded datasets and enhanced computational resources that are available in modern research environments.

Figure 4. A comprehensive depiction of the gMLP architecture incorporating the Spatial Gating Unit (SGU). The architecture is comprised of a series of L blocks characterized by uniform structure and dimensions. All projection operations within the model are linear in nature, and " ⊙ " denotes element-wise multiplication (linear gating).

Figure 4. A comprehensive depiction of the gMLP architecture incorporating the Spatial Gating Unit (SGU). The architecture is comprised of a series of L blocks characterized by uniform structure and dimensions. All projection operations within the model are linear in nature, and " ⊙ " denotes element-wise multiplication (linear gating).

The architecture of gMLP [44], is elegantly constructed from a series of L blocks, all of which share identical dimensions and configurations, resulting in a harmonious and cohesive design. In this context, denote $X\in {\mathbb{R}}^{n\times d}$ as the representations of tokens, where the sequence length is denoted by n, and the dimensionality is represented by d, encapsulating the essence of each token’s information and attributes. Each block is meticulously defined in the manner outlined below. Here, $\sigma$ represents an activation function (GeLU) known for its effectiveness in neural networks. The matrices U and V serve the purpose of establishing linear projections along the channel dimension, akin to the mechanisms utilized in the FFNs found within the architecture of Transformers.

$$\zeta =\sigma \left(XU\right),\tilde{\zeta }=s\left(\zeta \right),Y=\tilde{\zeta }V$$

(6)

An essential component of the previously described formulation is the layer, denoted as $s(·)$, which plays a crucial role in capturing the intricate spatial interactions among the tokens in the scenario where s functions as an identity mapping, the transformation described above simplifies to that of a conventional Feed-Forward Network, wherein individual tokens undergo processing in isolation, devoid of any inter-token communication that could enhance their contextual understanding. Consequently, one of the primary objectives is to ingeniously devise an effective s that is adept at capturing the complex and multifaceted spatial interactions between different tokens.

$$f_{W,\beta }\left(\zeta \right)=W\zeta +\beta$$

(7)

The overall architecture of each block draws inspiration from the concept of inverted bottlenecks, whereby $s(·)$ is defined explicitly as a spatial depthwise convolution, which is instrumental in promoting efficient information flow. It is worth noting that, in contrast to the Transformer architectures, our model elegantly circumvents the necessity for position embeddings, as the requisite positional information is inherently captured within the operations of $s(·)$.

$$s\left(\zeta \right)=\zeta \odot f_{W,\beta }\left(\zeta \right)$$

(8)

In the context of the discussion, where the symbol ⊙ signifies the operation of element-wise multiplication, we have come to recognize that for the purpose of maintaining stability throughout the training process, it is of utmost importance to set the initial values of the weights, denoted by W, to be extremely close to zero while assigning the biases, represented by $\beta$, to the value of one; this specific configuration leads to the outcome that $f_{W,\beta }\left(\zeta \right)$ is approximately equal to the vector of ones, denoted as $\mathbf{1}$, which in turn implies that at the very inception of training, the function $s\left(\zeta \right)$ closely resembles the input $\zeta$. By implementing this careful initialization strategy, we effectively guarantee that each gMLP block operates in a manner akin to a standard feedforward neural network, or FFN, during the initial phases of training; at this juncture, every individual token is treated in isolation, processed independently without any immediate interaction with others, and only as the learning progresses does the model begin to gradually introduce spatial information that interconnects and influences the tokens over the period of training.

3.4.3. Hybrid CNN-gMLP Model

The hybrid CNN-gMLP model presented in this study builds upon a rich history of advancements in both Convolutional Neural Networks (CNNs) and Gated Multi-Layer Perceptrons (gMLPs), particularly in the domain of speech recognition. Over the past decade, CNNs have established themselves as a powerful tool for local feature extraction, especially in tasks requiring structured data analysis, such as images and spectrograms. Their ability to capture spatial hierarchies in data has made them a natural choice for tasks like speech recognition, where both temporal and spectral features are critical [46,47]. CNNs have been effectively used in various audio processing tasks, demonstrating robust performance in extracting meaningful features from raw audio signals and improving the accuracy of speech recognition systems [48,49]. The introduction of gMLPs has added a new dimension to modelling sequential data. Unlike traditional MLPs, which primarily focus on individual inputs independently, gMLPs are designed to capture global dependencies across input sequences. This capability makes them particularly well-suited for tasks such as language modelling and, more recently, speech recognition [50,51]. The gMLP model has demonstrated competitive results in tasks that require the integration of context across long sequences, outperforming traditional models in scenarios that demand a deep understanding of the global structure of the data [52,53]. The decision to combine CNNs and gMLPs in this study was driven by the need to leverage the strengths of both models while mitigating their limitations. CNNs excel at capturing localized patterns within the data, such as the fine-grained temporal and spectral features of speech signals. Still, they often struggle with capturing long-range dependencies [47]. Conversely, gMLPs are adept at modeling these long-range dependencies but may not capture local features with the same level of granularity as CNNs [50]. By integrating these two approaches, the hybrid model aims to provide a more comprehensive understanding of the speech signal, capturing both local and global features essential for accurate speech recognition. A key innovation of our approach is the use of a dual feature extraction technique, combining Mel-Frequency Cepstral Coefficients (MFCCs) with partial Mel-Spectrograms. MFCCs are widely recognized for their ability to capture the power spectrum of audio signals, offering a detailed representation of the temporal aspects of speech. However, their performance can be limited in noisy environments, and they may miss important spectral details. To address this, we complement MFCCs with partial Mel-Spectrograms, which provide a more comprehensive view of the spectral content by focusing on specific Mel frequency bands relevant to the phonetic characteristics of the Kashmiri language. This dual feature extraction method enhances the robustness of the hybrid CNN-gMLP model by capturing both the spectral richness and the temporal precision required for accurate recognition. The dual feature extraction technique employed in this study serves as a robust input for the CNN, which processes these features to extract detailed local patterns. The output of the CNN is then passed through the gMLP layers, which are responsible for capturing the broader contextual information across the entire input sequence. This combination is particularly effective in addressing the phonetic challenges of Kashmiri, as it allows the model to balance the need for detailed spectral analysis with the requirement for global context understanding [6,11].

In designing this hybrid architecture, we implemented three convolutional blocks with increasing filter sizes to capture progressively more abstract features from the input data. The first block, with 32 filters, captures basic spectral patterns, while the subsequent blocks, each with 64 filters, extract more complex features. Pooling layers follow these convolutional layers to reduce the dimensionality of the feature maps, thereby enhancing computational efficiency and focusing the model’s attention on the most salient features. After the CNN layers, the extracted features are flattened and passed through a dense layer, which serves as a bridge to the gMLP layers. The gMLP component, consisting of four layers, employs layer normalization and gating mechanisms to effectively model the dependencies across the input sequence, ensuring that the model can generalize well across different speakers and dialects.

Figure 5. Architecture of the Hybrid CNN-gMLP Model for Spoken Kashmiri Recognition.

Figure 5. Architecture of the Hybrid CNN-gMLP Model for Spoken Kashmiri Recognition.

4. Results and Discussion

Table 4. Hyper-parameters for training Hybrid CNN-gMLP Model

5. Conclusion

References

1. Team, G.R. Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages. arXiv, arXiv:2303.01037 2023. [CrossRef]
2. Singh, A.; Kadyan, V.; Kumar, M.; Bassan, N. A comprehensive survey on automatic speech recognition using neural networks. Multimedia Tools and Applications 2020, 79, 3673–3704. [Google Scholar]
3. Koul, O.N. Kashmiri: A Descriptive Grammar; Routledge, 2009.
4. Wali, K.; Koul, O.N. Kashmiri: A Cognitive-Descriptive Grammar; Routledge, 1997.
5. Besacier, L.; Barnard, E.; Karpov, A.; Schultz, T. Automatic Speech Recognition for Under-Resourced Languages: A Survey. Speech Communication 2014, 56, 85–100. [Google Scholar] [CrossRef]
6. Liu, H.; Dai, Z.; So, D.R.; Le, Q.V. Pay Attention to MLPs. Advances in Neural Information Processing Systems 2021, 34, 9204–9215. [Google Scholar]
7. O’Neill, J.; Carson-Berndsen, J. Challenges in ASR Systems for Underrepresented Dialects. Proceedings of the Annual Conference on Computational Linguistics, 2023, pp. 233–241.
8. Adams, O.; Neubig, G.; Cohn, T.; Bird, S. Learning a Lexicon and Translation Model from Phoneme Lattices. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 2379–2389. [CrossRef]
9. Hannun, A.; Case, C.; Casper, J.; Catanzaro, B.; Diamos, G.; Elsen, E.; Prenger, R. Deep Speech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567, arXiv:1412.5567 2014. [CrossRef]
10. Aggarwal, A.; Srivastava, A.; Agarwal, A.; Chahal, N.; Singh, D.; Alnuaim, A.; Alhadlaq, A.; Lee, H.N. Two-Way Feature Extraction for Speech Emotion Recognition Using Deep Learning. Sensors 2022, 22, 2378. [Google Scholar] [CrossRef] [PubMed]
11. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; Houlsby, N. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. International Conference on Learning Representations (ICLR), 2020. [CrossRef]
12. Zhao, J.; Shi, G.X.; Wang, G.B.; Zhang, W. Automatic Speech Recognition for Low-Resource Languages: The Thuee Systems for the IARPA Openasr20 Evaluation. Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2021, pp. 113–119. [CrossRef]
13. Kipyatkova, I.; Kagirov, I. Analytical Review of Methods for Solving Data Scarcity Issues Regarding Elaboration of Automatic Speech Recognition Systems for Low-Resource Languages. Information Technologies and Computing Systems 2022, 21, 2–15. [Google Scholar] [CrossRef]
14. Zhao, J.; Zhang, W. Improving Automatic Speech Recognition Performance for Low-Resource Languages with Self-Supervised Models. IEEE Journal of Selected Topics in Signal Processing 2022, 16, 1105–1115. [Google Scholar] [CrossRef]
15. Min, Z.; Ge, Q.; Li, Z.; Weinan, E. MAC: A Unified Framework Boosting Low Resource Automatic Speech Recognition. arXiv, arXiv:2302.03498 2023.
16. Bartelds, M.; San, N.; McDonnell, B.; Jurafsky, D.; Wieling, M.B. Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation. arXiv preprint arXiv:2305.10951, arXiv:2305.10951 2023.
17. Yeo, J.H.; Kim, M.; Watanabe, S.; Ro, Y. Visual Speech Recognition for Languages with Limited Labeled Data Using Automatic Labels from Whisper. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023, pp. 6650–6654. [CrossRef]
18. Pratama, R.S.A.; Amrullah, A. Analysis of Whisper Automatic Speech Recognition Performance on Low Resource Language. Pilar Nusa Mandiri Journal of Computer and Information Science 2024, 20, 1–8. [Google Scholar] [CrossRef]
19. Bekarystankyzy, A.; Mamyrbayev, O.; Anarbekova, T. Integrated End-to-End Automatic Speech Recognition for Agglutinative Languages. Proceedings of the ACM on Human-Computer Interaction 2024, 8, 1–19. [Google Scholar] [CrossRef]
20. Sullivan, T.; Harding, E. Domain Shift in Speech Dialect Identification: Challenges and Solutions. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023, pp. 510–517.
21. Bhatt, R.M. The Kashmiri Language; Vol. 46, Studies in Natural Language and Linguistic Theory, Springer, 1999. [CrossRef]
22. Koka, N.A. Social Distribution of Linguistic Variants in Kashmiri Speech. Academy Publication 2016. [Google Scholar]
23. Zampieri, M.; Nakov, P. Phonetic Variation in Dialects; Cambridge University Press, 2021. [CrossRef]
24. Rather, S.N.; Singh, N. Phonetic and Phonological Variations in Kashmiri Language Dialects. International Journal of Linguistics 2017, 9, 12–20. [Google Scholar]
25. Kachru, B.B. Kashmiri. In The Indo-Aryan Languages; Cardona, G.; Jain, D., Eds.; Routledge, 2003; pp. 895–930.
26. Bhat, M.A.; Hassan, S. Prosodic Features of Kashmiri Dialect of Maraaz: A Comparative Study. IRE Journals 2020. [Google Scholar]
27. Open Speech Corpus Contributors. Open Speech Corpus Tool. https://github.com/open-speech-corpus/open-speech-corpus-tool, 2023. Accessed: 2024-08-12.
28. Rabiner, L.; Juang, B.H. Fundamentals of Speech Recognition; Prentice-Hall, Inc.: Englewood Cliffs, NJ, USA, 1993. [Google Scholar]
29. Journals, I. Zero Crossing Rate and Energy of the Speech Signal of Devanagari Script. IOSR Journals 2015. [Google Scholar]
30. Shen, J.L.; Hung, J.W.; Lee, L.S. Robust entropy-based endpoint detection for speech recognition in noisy environments. Proc. 5th International Conference on Spoken Language Processing (ICSLP 1998), 1998, p. paper 0232. [CrossRef]
31. Jurafsky, D.; Martin, J.H. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, 2nd ed.; Prentice-Hall, Inc.: Upper Saddle River, NJ, USA, 2009. [Google Scholar]
32. Wang, D.; Brown, G.J. Computational Auditory Scene Analysis: Principles, Algorithms, and Applications; IEEE Press/Wiley, 2016.
33. Ganie, H.A.; Hasnain, S.K. Computational Linguistics and the Kashmiri Language: Issues and Challenges. International Journal of Computer Applications 2015, 128, 1–6. [Google Scholar] [CrossRef]
34. Winursito, A.; Hidayat, R.; Bejo, A. Improvement of MFCC feature extraction accuracy using PCA in Indonesian speech recognition. 2018 International Conference on Information and Communications Technology (ICOIACT), 2018. [CrossRef]
35. Hidayat, R. Frequency Domain Analysis of MFCC Feature Extraction in Children’s Speech Recognition System. Infotel Journal 2022, 14, 28–36. [Google Scholar] [CrossRef]
36. Setianingrum, A.; Hulliyah, K.; Amrilla, M.F. Speech Recognition of Sundanese Dialect Using Convolutional Neural Network Method with Mel-Spectrogram Feature Extraction. 2023 8th International Conference on Information Technology, Information Systems and Mechatronics (CITSM), 2023. [CrossRef]
37. Permana, S.D.H.; Rahman, T.K.A. Improved Feature Extraction for Sound Recognition Using Combined Constant-Q Transform (CQT) and Mel Spectrogram for CNN Input. 2023 International Conference on Mechatronics, Robotics, and Automation (ICMERALDA), 2023. [CrossRef]
38. Li, Q.; Yang, Y.; Lan, T.; Zhu, H.; Wei, Q.; Qiao, F.; Liu, X.; Yang, H. MSP-MFCC: Energy-Efficient MFCC Feature Extraction Method With Mixed-Signal Processing Architecture for Wearable Speech Recognition Applications. IEEE Access 2020. [Google Scholar] [CrossRef]
39. Amodei, D.; Ananthanarayanan, S.; Anubhai, R.; Bai, J.; Battenberg, E.; Case, C.; Walker, J.; Zhu, Z. Deep Speech 2: End-to-End Speech Recognition in English and Mandarin. Proceedings of The 33rd International Conference on Machine Learning, 2016, pp. 173–182.
40. Joshi, D.; Pareek, J.; Ambatkar, P. Comparative Study of MFCC and Mel Spectrogram for Raga Classification Using CNN. Indian Journal of Science and Technology, 2023. [CrossRef]
41. Hafiz, N.F.; Mashohor, S.; Shazril, M.H.S.E.M.A.; Rasid, M.F.A.; Ali, A. Comparison of Mel Frequency Cepstral Coefficient (MFCC) and Mel Spectrogram Techniques to Classify Industrial Machine Sound. Proceedings of the 16th International Conference on Software, Knowledge, Information Management, and Applications (SKIMA). [CrossRef]
42. Abayomi-Alli, O.O.; Damaševičius, R.; Qazi, A.; Adedoyin-Olowe, M.; Misra, S. Data Augmentation and Deep Learning Methods in Sound Classification: A Systematic Review. Electronics 2022, 11, 3795. [Google Scholar] [CrossRef]
43. Qazi, A.; Damaševičius, R.; Abayomi-Alli, O.O. Combining Transformer, Convolutional Neural Network, and Long Short-Term Memory Architectures: A Novel Ensemble Learning Technique That Leverages Multi-Acoustic Features for Speech Emotion Recognition in Distance Education Classrooms. Applied Sciences 2022, 12, 1–21. [Google Scholar] [CrossRef]
44. Liu, H.; Dai, Z.; So, D.; Le, Q.V. Pay attention to mlps. Advances in neural information processing systems 2021, 34, 9204–9215. [Google Scholar]
45. Yu, P.; Artetxe, M.; Ott, M.; Shleifer, S.; Gong, H.; Stoyanov, V.; Li, X. Efficient language modeling with sparse all-mlp. arXiv, arXiv:2203.06850 2022.
46. Passricha, V.; Aggarwal, R. A Hybrid of Deep CNN and Bidirectional LSTM for Automatic Speech Recognition. Journal of Intelligent Systems 2018, 27, 555–563. [Google Scholar] [CrossRef]
47. Wubet, Y.A.; Lian, K.Y. Voice Conversion Based Augmentation and a Hybrid CNN-LSTM Model for Improving Speaker-Independent Keyword Recognition on Limited Datasets. IEEE Access 2022, 10, 114222–114234. [Google Scholar] [CrossRef]
48. Atila, O.; Şengür, A. Attention Guided 3D CNN-LSTM Model for Accurate Speech-Based Emotion Recognition. Applied Acoustics 2021, 175, 108260. [Google Scholar] [CrossRef]
49. John Lorenzo Bautista, Y. Lee, H.S. Speech Emotion Recognition Based on Parallel CNN-Attention Networks with Multi-Fold Data Augmentation. Electronics 2023, 11, 3935. [Google Scholar] [CrossRef]
50. Hu, K.; Sainath, T.N.; Pang, R.; Prabhavalkar, R. Deliberation Model Based Two-Pass End-To-End Speech Recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing 2020, 28, 123–134. [Google Scholar] [CrossRef]
51. Li, G.; Sun, Z.; Hu, W.; Cheng, G.; Qu, Y. Position-aware relational transformer for knowledge graph embedding. IEEE Transactions on Neural Networks and Learning Systems 2023. [Google Scholar] [CrossRef] [PubMed]
52. Yang, C.H.H.; Gu, Y.; Liu, Y.C.; Ghosh, S.; Bulyko, I.; Stolcke, A. Generative Speech Recognition Error Correction With Large Language Models and Task-Activating Prompting. IEEE/ACM Transactions on Audio, Speech, and Language Processing 2023, 31, 234–245. [Google Scholar] [CrossRef]
53. Bai, Y.; Yi, J.; Tao, J.; Tian, Z.; Wen, Z.; Zhang, S. Fast End-to-End Speech Recognition Via Non-Autoregressive Models and Cross-Modal Knowledge Transferring From BERT. IEEE/ACM Transactions on Audio, Speech, and Language Processing 2021, 29, 456–467. [Google Scholar] [CrossRef]
54. Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv, arXiv:1412.6980 2014.
55. Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 2014, 15, 1929–1958. [Google Scholar]
56. Powers, D.M. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061, arXiv:2010.16061 2020.
57. Goutte, C.; Gaussier, E. A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation. Advances in Information Retrieval. ECIR 2005. Lecture Notes in Computer Science, vol 3408. Springer, Berlin, Heidelberg, 2005, pp. 345–359. [CrossRef]
58. Baevski, A.; Schneider, S.; Auli, M. Wav2Vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. arXiv preprint arXiv:2006.11477, arXiv:2006.11477 2020. [CrossRef]
59. Chan, W.; Jaitly, N.; Le, Q.; Vinyals, O. Listen, Attend and Spell: A Neural Network for Large Vocabulary Conversational Speech Recognition. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 4960–4964. [CrossRef]