1. Introduction

2. Literature Review

2.2. Burmese Language

The Burmese language is so complex and consists of Tones, Stress, Vowels, and Consonants. Where the tones are referred to as the lexical meaning of the syllables. There are two groups of tones consists of static and dynamic. Where the static has three different categories low, middle, and high. Also two different categories in dynamic as falling and rising. The stress is like louder, longer, and higher speech that is opposite to normal speech. The Myanmar alphabet consists of 33 letters, 12 vowels, and ten numerals and is written like other languages, from left to right. Unlike English, Burmese sentences do not need any space in sentences but for beauty and cleanliness, sometimes space is used (Wikipedia, 2023). Figure 2.1 shows the Myanmar basic consonants, also the Myanmar vowels, and the Myanmar basic numerals.

Figure 2. 1. Burmese language (Sann Su Su Yee, 2010).

Figure 2. 1. Burmese language (Sann Su Su Yee, 2010).

3. Methodology

3.1. Overview

Before anything is started, the first thing to do is research the related project ideas and methodology. In order that this project will be done quickly and efficiently. However, researching is not an easy task because there are a ton of papers and articles that discuss this idea, and most of them are written with the same principles and ideas and some are not that effective. The same methodology and same techniques are applied in most of the research papers. In addition, there are quite a few papers that really stand out from the others. Therefore, reading one paper after another is really tough because of the analysis. The issue is analyzing every paper to know whether these papers are useful for this project or not. Some of the papers used completely the same method, which is why it is a waste of time. After all the hard work, there are very few papers that really can be helpful for the project.

Figure 3. 1. Methodology Structural Outline.

Figure 3. 1. Methodology Structural Outline.

After researching and analyzing, another thing is to collect the data sample, in this case, speech data samples. As an advantage of research, there are some websites that provided free, already clean, speech data samples. Additionally, another source is the YouTube Burmese podcasts and audiobooks. The audiobooks are really long and need to be clean for usage but they are really effective data samples because they speak in the clearest as possible to make others people understand. In this way, these audiobooks are excellent materials for data samples.

Data processing is the process of cleaning the data samples to maximize the output of the algorithm. For this case, the audiobooks need to clean because they will be downloaded in MPEG Audio Layer 3 (mp3) format and they need to be converted into Waveform Audio File Format (wav). Another thing to do is trim the data samples because there are times that people don’t speak for a few seconds not only in conversation but also in audiobooks. Then, trim and cut the timeframes to reduce the size of the dataset to boost the algorithm run time.

The modeling stage is the most important and difficult part of the project. There are tons of deep learning models that can help with ASR for any language. But the only problem is that it takes a lot of time to write lines by lines of code which is very confusing. Because these models need to be finely tuned depending on the language that is going to be predicted. To overcome this problem, by using the pre-built models. For this project, Wav2vec2 by MetaAI (Meta AI, 2020). Due to a self-supervised training method, which is a relatively novel idea in this industry, Wav2Vec 2.0 is one of the most advanced models for automatic speech recognition currently available. With this method of training, we may pre-train a model using unlabeled data, which is always easier to acquire. The model may then be adjusted for a particular purpose on a given dataset (Sus, 2021). This model used Connectionist Temporal Classification (CTC) and is pre-trained for multiple languages and can be implemented in any ASR project for training with new datasets by users (Platen, 2021). Training the model takes a lot of time and it depends on how much training data will be used also. After training, evaluate the model performance, and train again for more accuracy with more training data. If the model is giving out a decent result then the model is ready for prototyping.

After the modeling stage, now it is time to prototype the system.

To prototype the system, the best way to do it is to upload the trained model to the server and reuse it anytime when needed it. By doing this, the model stays on the server with the latest training performance and it makes the model test in various ways. This way it will be easier to implement in any project or even easily implement on the website later.

After all the hard work, there will be another hard work and the confusing stage is only left which is to implement the system into the web interface, a website. Furthermore, making and hosting that website on the internet in order to access by the whole world.

The last stage of the project is to evaluate the prototype through the website. Adjusting the appearance of the website added more data samples for better accuracy, trained the model for longer, and optimized the models in the modeling stage. These actions will be repeated to maintain the prototype and website.

3.2. Hardware Requirements

Table 3. 1. Hardware Specifications.

Table 3. 1. Hardware Specifications.

Component	Specifications
Laptop or Desktop	Operating System	Windows 10 Home
	Central Processing Unit	AMD Ryzen 5
	Memory	16 GB
	Hard Disk Drive	1TB
	Solid-state Drive	256 GB

3.3. Software Requirements

Table 3. 2. Software Specifications.

Table 3. 2. Software Specifications.

Software	Version
Anaconda	2021.11
Microsoft Visual Studio	1.74.2
cmd	10.0.19045.2546

3.4. Programming Languages

Table 3. 3. Programming Language.

Table 3. 3. Programming Language.

Programming Language	Usage
Python	Main programming language use from start to finish, model building, training, evaluation, and deployment, all are done in a single language.

3.5. Project Timeline

Table 3. 4. Timeline.

Table 3. 4. Timeline.

Week	DESCRIPTION
Week 1	Article Collection
Week 2	Article Analysis
Week 3	Literature Review
Week 4	Methodology
Week 5	Proposal Submission
Week 6	Project Pitching
Week 7	Report Submission
Week 8	Speech Dataset Collection
Week 9	Speech Dataset Collection
Week 10	Data Pre-processing
Week 11	Data Pre-processing
Week 12	Modelling
Week 13	Modelling
Week 14	Prototyping
Week 15	Website Creation
Week 16	Prototype Implementation into Website
Week 17	Prototype Implementation into Website
Week 18	Evaluation
Week 19	Project Pitching
Week 20	Overall Refining
Week 21	Final Report Submission

3.6. Project Development

3.6.1. Data Collection

This Burmese ASR project is using the dataset called SLR80 from the popular dataset provider OpenSLR (Oo et al., 2020). This dataset contains a total of 2530 clean speech samples by 20 speakers and all the speakers are female. The dataset also provides a clean transcription of each speech sample. The combination of all speech data samples will be around 5 to 6 hours. For an ASR model, this amount of hours is not ideal but considering the computer specifications, this amount is good enough to at least train and get some output. As the dataset is small, the training time will be more for better accuracy of prediction.

3.6.2. Data Processing

Before model training, data processing will set up the data as needed. Divide the dataset into two groups, train, and test, as the initial step in the data processing. The remaining 2024 samples, or 80% of the 2530 samples, are utilized for training, while 506 samples, or 20%, are used for testing.

Figure 3. 2. Train and Test Split.

Figure 3. 2. Train and Test Split.

As this model is supervised by label data, it is required to delete some characters that do not really have a sound, such as special characters and spaces, after separating the dataset into train and test. Another step is to provide all the unique characters from the Burmese language to the CTC algorithm. From the transcription of datasets, the total unique characters in the transcription are 60 characters, and also manually added two more characters that are unknown characters and padding to the start and end of the transcription. These two characters are specially for the CTC algorithm to improve performance. Eventually, the total of unique characters is 63 characters. After that, load all the characters into Wav2Vec2CTCTokenizer, this will tokenize the transcription sentence into characters. Now, preprocessing for text data is done and now it is time for audio data. For the audio data, the data set is already trimmed and short but the sampling rate is 48KHz. The sampling rate required by the CTC algorithm is 16KHz. So, change the sampling rate of the whole dataset.

3.6.3. Modelling

For the modelling, there are a few steps needed to prepare before starting the training process. First of all, step up the Wav2Vec2FeatureExtractor, this feature extractor will do the jobs such as pre-processing audio files to Log-Mel Spectrogram features, padding, normalization, and conversion to PyTorch tensors. After that, set up the Wav2Vec2Processor, it is a wrapper of tokenizers and feature extractors.

As the most important process for almost all deep learning models, the Word Error Rate (WER) function is defined and created. Finally setting up the Wav2Vec2ForCTC model, this model is fined tuned and pre-trained by MetaAI. Since this model is so big that trained with a large amount of data from multiple languages, for this project downscaling the parameters is an important thing to do. In downscaling, the layer drop parameter is set to zero since the dataset for training is so small. And assign the Wav2Vec2Processor to the algorithm.

It's time for training now that the model has been defined. There are a few factors that must be specified before training. The first configuration involves dividing the entire training data by the batch size and setting it to 100. Instead of using epochs, the Wav2Vec2ForCTC model uses steps. The epoch number must still be included, though. After setting up the parameters, call the train function and it will start the training process. The training process will take a huge amount of hours even if the dataset is small. After 2 days worth of training the model, the model is start to recognize the speech and give some output text. The final result of the training process is shown in Table 4.1.

Table 4. 1. Training Result.

Table 4. 1. Training Result.

Epoch	Training Loss	Learning Rate	Evaluation Loss	WER	Training Time
1.58	10.3152	6.00E-05	4.660822	1	8 - 9 hours
3.17	3.9002	0.00012	3.500513	1	8 - 9 hours
4.76	3.4987	0.00018	3.450787	1	8 - 9 hours
6.35	3.4524	0.00024	3.372131	1	8 - 9 hours
7.93	3.128	0.0003	1.946949	1	8 - 9 hours
9.52	1.3368	0.000233	0.684924	1	8 - 9 hours

3.6.4. Prototyping

After the modelling is done, the next step is to prototype the model so that the model is useable to do the prediction. In order to do that, there is a good and reliable website called Hugging Face that pretty much looks like a GitHub. Upload the model there and anyone can download the model, not the code, by downloading the model, it does not need to train the model again, it can be used instantly after downloading the model from the local directory and only take a few lines of codes to predict the speech.

Download the model and create a function to predict the speech. To predict the speech, the speech file needs to be put inside one folder then call the path of that speech file and change the sampling rate of the speech file. And changing the speech file into PyTorch tensors and then finally prediction is taken place and as a result, the text will be returned.

3.6.5. Deployment

Now everything is ready and the only thing left to do is to make the prototype useable through the website. Implementing the Python code and function into the website is a bit confusing and there are a ton of programming languages that do that job but for this project, Streamlit is used. Streamlit is a python library that makes the python code into a web interface but the downside is that Streamlit does not have that much of a visually appealing function. So, the website will be very simple visual.

4. Result and Discussion

4.1. Overview

In Deep learning or neural networks, supervised learning is better than unsupervised learning because of its controllability. But at the same time, the model needs detailed parameters to handle all the processes. A very good ASR needs thousands of hours of speech data with good transcription. This is a downside of ASR because these data are so big and to get a clean transcription of each speech data is a great effort that can not be done in a given time. Even after that, to handle these data and model training, a very powerful computer is needed because the training process of ASR models takes days or weeks. A normal personal computer cannot perform these powerful processes.

After long hours and harsh process of training, the overall training result is gathered as shown in Table 4.1. As the table shows, there is only a total of six training and evaluation epochs, however, to get that six epochs, the computer is going through a very powerful process for more than 48 hours. Even though the dataset is small and the training time is only 48 hours, the training loss, evaluation loss, and learning rate are getting higher and higher results. But the only thing that did not change throughout the epochs is WER, it is consistently staying at one.

Figure 4. 1. Training Loss VS Evaluation Loss.

Figure 4. 1. Training Loss VS Evaluation Loss.

As from the figure above, at the first epochs of training, the loss during training is really high which is above 10, but at the same epoch, the loss during the evaluation is only over four. That is because the training samples are four times higher than the testing samples. Overall, as the epochs go higher, the loss for both training and evaluation is getting decreased. At the last epoch of six, both training loss and evaluation loss are under two.

Figure 4. 2. Learning Rate by Epochs.

Figure 4. 2. Learning Rate by Epochs.

The learning rate of the models is also a great factor that determines the accuracy of the model. In this case, the learning rate is not very significant but slowly it gets a good result as the epochs go higher. The learning rate is very small that is under zero, it is because the small dataset is getting at the very big model, Wav2Vec2ForCTC. The learning rate is rather unstable compared to the training loss and evaluation loss since at epoch six, the learning rate is going down in values compared to the values at epoch five.

Figure 4. 3. Training Time Vs WER.

Figure 4. 3. Training Time Vs WER.

After training the model for more than 48 hours, the model is getting in good shape from every aspect except the WER. The WER is staying at one consistently through the epochs. The training time required between eight to nine hours to finish one epoch, it is a very time-consuming and very powerful huge task for a computer to handle. However, as the fact from table 4.1, it is good enough to assume that the model is going to a be good model but slowly.

4.2. Website Deployment

After all the processing, modelling, training and result analysis, the last step and the best way to showcase all the work that has been done along the way is through the website deployment. Abstract of the website deployment is all about creating the website and integrating the model and maintaining that website over time. It is mostly making the website work faster, perform well, and give the user a seamless experience. To create such kind of website through the Python programming language, the streamlit is the best and easy tool to use. Streamlit is a Python library that provides easy solutions and implementation for creating websites.

Streamlit has many advantages using as ease of implementation, detailed documentation and a huge community but there are also disadvantages such as limited features and poor visuals. The basic streamlit visual main interface is easy to create according to the need of the user and Figure 4.4 shows the simple and basic user interface for this project.

Figure 4. 4. Basic User Interface.

Figure 4. 4. Basic User Interface.

According to the Figure 4.4, there are two main options that are available to the user, the first one is to upload the speech file as needed and the second one is to see the examples. The first one is dedicated to the main function of the website, there, users can upload the speech file and listen to the uploaded speech file and an option to extract the text from the uploaded speech file as shown in Figure 4.5.

Figure 4. 5. Uploaded Speech File and Extracted Text.

Figure 4. 5. Uploaded Speech File and Extracted Text.

When the users upload the speech file for extracting, the uploaded speech will go through the data pre-processing step basically changing the uploaded speech file sampling rate to 16000 Hz because the model is trained on the speech samples of 16000 Hz. There is another condition to change the uploaded speech file that is the speech file must be stored in a temporary folder on the local computer. This local temporary folder will be stored just for a few seconds, as soon as the processing and extracting of the uploaded speech file is complete, this local temporary folder will be deleted by itself.

As Figure 4.4 shows, there is another option to choose from if the users do not want to upload the speech file, they can view the pre-existed speech files and their transcriptions. Users can click the “Show Sample” checkbox and three samples will show on the screen. Users can choose one of the samples and view the transcriptions. This process is shown in Figure 4.6.

Figure 4. 6. Pre-existed Samples.

Figure 4. 6. Pre-existed Samples.

5. Conclusion

References

1. Acharjya, D. P., Mitra, A., & Zaman, N. (2022). Deep learning in data analytics. Springer International Publishing.
2. Baevski, A., Zhou, Y., Mohamed, A., & Auli, M. (2020). wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33, 12449-12460.
3. Beg, A., & Hasnain, S. K. (2008). A speech recognition system for Urdu language. Wireless Networks, Information Processing and Systems, 118-126. [CrossRef]
4. Chit, K. M., & Lin, L. L. (2021). Exploring CTC based end-to-end techniques for Myanmar speech recognition. Advances in Intelligent Systems and Computing, 1038-1046. [CrossRef]
5. Fatima-tuz-Zahra, N. Jhanjhi, S. N. Brohi, N. A. Malik and M. Humayun, "Proposing a Hybrid RPL Protocol for Rank and Wormhole Attack Mitigation using Machine Learning," 2020 2nd International Conference on Computer and Information Sciences (ICCIS), Sakaka, Saudi Arabia, 2020, pp. 1-6. [CrossRef]
6. Gaikwad, S. K., Gawali, B. W., & Yannawar, P. (2010). A review on speech recognition technique. International Journal of Computer Applications, 10(3), 16-24.
7. Gopi, R., Sathiyamoorthi, V., Selvakumar, S., Manikandan, R., Chatterjee, P., Jhanjhi, N. Z., & Luhach, A. K. (2022). Enhanced method of ANN based model for detection of DDoS attacks on multimedia internet of things. Multimedia Tools and Applications, 1-19.
8. Gouda W, Almurafeh M, Humayun M, Jhanjhi NZ. Detection of COVID-19 Based on Chest X-rays Using Deep Learning. Healthcare. 2022; 10(2):343. [CrossRef]
9. Gu, N., Lee, K., Basha, M., Ram, S. K., You, G., & Hahnloser, R. H. (2024, April). Positive Transfer of the Whisper Speech Transformer to Human and Animal Voice Activity Detection. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 7505-7509). IEEE.
10. Humayun M, Ashfaq F, Jhanjhi NZ, Alsadun MK. Traffic Management: Multi-Scale Vehicle Detection in Varying Weather Conditions Using YOLOv4 and Spatial Pyramid Pooling Network. Electronics. 2022; 11(17):2748. [CrossRef]
11. Humayun M, Sujatha R, Almuayqil SN, Jhanjhi NZ. A Transfer Learning Approach with a Convolutional Neural Network for the Classification of Lung Carcinoma. Healthcare. 2022; 10(6):1058. [CrossRef]
12. Hussain, K., Rahmatyar, A. R., Riskhan, B., Sheikh, M. A. U., & Sindiramutty, S. R. (2024, January). Threats and Vulnerabilities of Wireless Networks in the Internet of Things (IoT). In 2024 IEEE 1st Karachi Section Humanitarian Technology Conference (KHI-HTC) (pp. 1-8). IEEE.
13. Hussain, S. J., Irfan, M., Jhanjhi, N. Z., et al. (2021). Performance enhancement in wireless body area networks with secure communication. Wireless Personal Communications, 116(1), 1–22. [CrossRef]
14. Johnson, M., Lapkin, S., Long, V., Sanchez, P., Suominen, H., Basilakis, J., & Dawson, L. (2014). A systematic review of speech recognition technology in health care. BMC medical informatics and decision making, 14, 1-14.
15. Kamath, U., Liu, J., & Whitaker, J. (2019). Deep learning for NLP and speech recognition (Vol. 84). Cham, Switzerland: Springer.
16. Khaing, I., & Linn, K. Z. (2013). Myanmar continuous speech recognition system based on DTW and HMM. International Journal of Innovations in Engineering and Technology (IJIET), 2(1), 78-83.
17. Lim, M., Abdullah, A., Jhanjhi, N. Z., Khan, M. K., & Supramaniam, M. (2019). Link prediction in time-evolving criminal network with deep reinforcement learning technique. IEEE Access, 7, 184797-184807.
18. Mallick, C., Bhoi, S. K., Singh, T., Hussain, K., Riskhan, B., & Sahoo, K. S. (2023). Cost Minimization of Airline Crew Scheduling Problem Using Assignment Technique. International Journal of Intelligent Systems and Applications in Engineering, 11(7s), 285-298.
19. Majid, M., Hayat, M. F., Khan, F. Z., Ahmad, M., Jhanjhi, N. Z., Bhuiyan, M. A. S.,... & AlZain, M. A. (2021). Ontology-Based System for Educational Program Counseling. Intelligent Automation & Soft Computing, 30(1).
20. Mallick, C., Bhoi, S. K., Singh, T., Swain, P., Ruskhan, B., Hussain, K., & Sahoo, K. S. (2023). Transportation Problem Solver for Drug Delivery in Pharmaceutical Companies using Steppingstone Method. International Journal of Intelligent Systems and Applications in Engineering, 11(5s), 343-352.
21. Minn, K. H., & Soe, K. M. (2019). Myanmar word stemming and part-of-speech tagging using rule based approach (Doctoral dissertation, MERAL Portal).
22. Mon, A. N., Pa Pa, W., & Thu, Y. K. (2018). Improving Myanmar automatic speech recognition with optimization of convolutional neural network parameters. International Journal on Natural Language Computing (IJNLC) Vol, 7.
23. Mon, A. N., Pa, W. P., & Ye, K. T. (2019). UCSY-SC1: A Myanmar speech corpus for automatic speech recognition. International Journal of Electrical and Computer Engineering, 9(4), 3194.
24. Naing, H. M. S., Hlaing, A. M., Pa, W. P., Hu, X., Thu, Y. K., Hori, C., & Kawai, H. (2015, December). A Myanmar large vocabulary continuous speech recognition system. In 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) (pp. 320-327). IEEE.
25. Naing, H. M. S., Hlaing, A. M., Pa, W. P., Hu, X., Thu, Y. K., Hori, C., & Kawai, H. (2015, December). A Myanmar large vocabulary continuous speech recognition system. In 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) (pp. 320-327). IEEE.
26. Oo, Y. M., Wattanavekin, T., Li, C., De Silva, P., Sarin, S., Pipatsrisawat, K.,... & Gutkin, A. (2020, May). Burmese speech corpus, finite-state text normalization and pronunciation grammars with an application to text-to-speech. In Proceedings of the Twelfth Language Resources and Evaluation Conference (pp. 6328-6339).
27. Rabiner, L. R., Wilpon, J. G., & Soong, F. K. (1989). High performance connected digit recognition using hidden Markov models. IEEE Transactions on Acoustics, Speech, and Signal Processing, 37(8), 1214-1225.
28. Saeed, S., Jhanjhi, N. Z., Naqvi, M., Malik, N. A., & Humayun, M. (2019). Disparage the barriers of journal citation reports (JCR). International Journal of Computer Science and Network Security, 19(5), 156-175.
29. Shah, S. A. A., Bukhari, S. S. A., Humayun, M., Jhanjhi, N. Z., & Abbas, S. F. (2019, April). Test case generation using unified modeling language. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1-6). IEEE.
30. Soe, W., & Theins, Y. (2015). Syllable-based Myanmar language model for speech recognition. IEEE Xplore. Retrieved from https://ieeexplore.ieee.org/document/7166608.
31. Sowjanya, A. M. (2021). Self-Supervised Model for Speech Tasks with Hugging Face Transformers. Turkish Online Journal of Qualitative Inquiry, 12(10).
32. Tabbakh, A. (2021). Bankruptcy prediction using robust machine learning model. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(10), 3060-3073.
33. Thein, Y. (2010). High accuracy Myanmar handwritten character recognition using hybrid approach through MICR and neural network. International Journal of Computer Science Issues (IJCSI), 7(6), 22.
34. Thu, Y. K. (2021, April 17). myG2P. GitHub. Retrieved from https://github.com/ye-kyaw-thu/myG2P.
35. Tucci, L., Laskowski, N., & Burns, E. (2019). A guide to Artificial intelligence in the enterprise.
36. V. Singhal et al., "Artificial Intelligence Enabled Road Vehicle-Train Collision Risk Assessment Framework for Unmanned Railway Level Crossings," in IEEE Access, vol. 8, pp. 113790-113806, 2020. [CrossRef]
37. Willyard, W. (2022, January 28). What role does an acoustic model play in speech recognition? Rev blog. Rev Blog. https://www.rev.com/blog/resources/what-is-an-acoustic-model-in-speech-recognition.
38. Xu, S., Liu, X., Ma, K., Dong, F., Riskhan, B., Xiang, S., & Bing, C. (2023). Rumor detection on social media using hierarchically aggregated feature via graph neural networks. Applied Intelligence, 53(3), 3136-3149.
39. Yong, C. T., Hao, C. V., Riskhan, B., Lim, S. K. Y., Boon, T. G., Wei, T. S., Balakrishnan, S., & Shah, I. A. (2023). An implementation of efficient smart street lights with crime and accident monitoring: A review. Journal of Survey in Fisheries Sciences.
40. Yu, D., & Deng, L. (2015). Automatic speech recognition. Signals and Communication Technology. [CrossRef]
41. Zaman, G., Mahdin, H., Hussain, K., Atta-Ur-Rahman, Abawajy, J., & Mostafa, S. A. (2021). An ontological framework for information extraction from diverse scientific sources. IEEE Access, 9, 42111-42124. [CrossRef]
42. Zhong, G., Wang, L., Ling, X., & Dong, J. (2016). An overview on data representation learning: From traditional feature learning to recent deep learning. The Journal of Finance and Data Science, 2(4), 265-278. [CrossRef]