Abstract

Bangla is used by over 300 million people, but its automated speech-to-text (STT) technology is still one of the least mature. Factors such as lack of high-quality conversational datasets, usage of models trained mostly on scripted speech, and insufficient coverage of dialectal diversity and demographics hinder development. Consequently, current Bangla STT models are only highly accurate under certain conditions and become less accurate in natural conversational setting. In order to overcome this limitation, this work presents ConvBangla-115, a 115.9 hours Bangla conversational speech dataset that consists of spontaneous utterances made by speakers aged from 10 to 50 years old in regions like Dhaka, Rajshahi, Rangpur, and Khulna under diverse acoustic conditions with recorded transcription verifications. Using this dataset, five variations of OpenAI’s Whisper model are fine-tuned on training subsets of 5, 10, 50, and 115.9 hours and the results are investigated across three angles: the effect of fine-tuning, amount of training data, and the model size. The fine-tuning yields positive results with all configurations; Whisper large-v3 (Turbo) achieves WER of 19.60% ± 0.3 with a relative improvement of 60.8% from zero-shot WER of 50.00%, where all observed differences are statistically significant (p < 0.001). Besides the main indicator, the results show that WER follows the power law of dependence on the number of training hours; the value of the exponent rises as the model becomes larger (from 0.08 in Tiny to 0.25 in Medium); the first 5 hours cover 49% of the total gains for Tiny but only 17% for Medium; and the model which is of one size larger and fine-tuned for 50 hours performs not worse or better than the model of smaller size trained on 115.9 hours. The paper offers a speech corpus in Bangla, a fine-tuning study in various model sizes and amounts of training data, and recommendations for speech recognition in low-resource scenarios.

References

• U. Aiman, M. N. Islam, H. Sultan Chowdhury, M. S. Rahman, M. T. Habib, and M. Hasan. BRADS: a multipurpose audio dataset for Bangla regional word detection, March 2025.
• R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Henretty, R. Morais, L. Saunders, F. Tyers, and G. Weber. Common Voice: a massively-multilingual speech corpus. In Proc. 12th Language Resources and Evaluation Conf., pages 4218–4222, May 2020.
• Bengali.AI. Open source datasets. https://www.bengali. ai/datasets, 2023.
• D. Bhadra, M. Hosain, and F. Alam. Deep feed-forward neural network for bangla isolated speech recognition. Journal Of Nazrul University, 10(1-2):85–96, 2025.
• M. M. Billah, M. L. Sarker, and M. A. H. Akhand. KBES: a dataset for realistic Bangla speech emotion recognition with intensity level. Data in Brief, 51:109741, December 2023.
• S. K. Chatterji. Origin and Development of the Bengali Language, Pt. 1. 1926.
• S. Guchhait, A. S. A. Hans, and J. Augustine. Automatic speech recognition of Bengali using Kaldi. In Proc. 2nd Int. Conf. Sustainable Expert Systems, pages 153–166, Singapore, 2022. Springer Nature.
• M. Hasan. Transformer-based Whisper Bangla ASR model. https://github.com/menon92/BangalASR, 2022.
• A. Hasnat, J. Mowla, and M. Khan. Isolated and continuous Bangla speech recognition: implementation, performance and application perspective. https://www.semanticscholar.org/paper/ e6697a3bb8601ab533012473b6844b2121c7d7bd, 2007.
• F. Hassan, M. R. A. Kotwal, and M. N. Huda. Bangla ASR design by suppressing gender factor with gender-independent and gender-based HMM classifiers. In Proc. World Congress on Information and Communication Technologies, pages 1276–1281, December 2011.
• G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82–97, November 2012.
• D. Jurafsky and J. H. Martin. Speech and language processing, 2023. 3rd ed. draft.
• M. A.-A.-K. Kafi, R. Moni, and D. M. Raza. Bangla voice dataset: simple, complex, and compound structures, September 2025.
• M. R. Karim, S. Islam, and M. M. Rahman. Multilingual speech-to-text framework for Bangla-English code-switching scenarios. In Proc. Int. Conf. Natural Language Processing (ICNLP), pages 112–120. IEEE, 2024.
• S. Kibria, A. M. Samin, M. H. Kobir, M. S. Rahman, M. R. Selim, and M. Z. Iqbal. Bangladeshi Bangla speech corpus for automatic speech recognition research. Speech Communication, 136:84–97, January 2022.
• P. Lamere, P. Kwok, E. Gouvea, B. Raj, R. Singh, W. Walker, M. K. Warmuth, and P. Wolf. The CMU Sphinx-4 speech recognition system. https://www.semanticscholar.org/paper/ 5064c602c3a57f4e6f1e4c8f8fb137384c5d41a7, 2001.
• Mehraj Hossain Mahi, Anzir Rahman Khan, and Mayen Uddin Mojumdar. Bangladial: A merged and imbalanced text dataset for bengali regional dialect analysis. Data in Brief, 63:112200, 2025.
• S. Mandal, B. Das, and P. Mitra. Shruti-ii: A vernacular speech recognition system in bengali and an application for visually impaired community. In 2010 IEEE Students Technology Symposium (TechSym), pages 229–233, 2010.
• S. Mandal, S. Yadav, and A. Rai. End-to-end Bengali speech recognition. arXiv, https://www.semanticscholar.org/ paper/463c98d2fd4954923db2366786480339c95f9dfb, September 2020.
• M. F. Mridha, A. Q. Ohi, M. A. Hamid, and M. M. Monowar. A study on the challenges and opportunities of speech recognition for Bengali language. Artificial Intelligence Review, 55(4):3431–3455, April 2022.
• G. Muhammad, Y. A. Alotaibi, and M. N. Huda. Automatic speech recognition for Bangla digits. In Proc. 12th Int. Conf. Computers and Information Technology (ICCIT), pages 379–383, December 2009.
• A. U. Nasib, H. Kabir, R. Ahmed, and J. Uddin. A real-time speech-to-text conversion technique for the Bengali language. In Proc. Int. Conf. Computer, Communication, Chemical, Material and Electronic Engineering (IC4ME2), pages 1–4. IEEE, 2018.
• OpenAI. Robust speech recognition via large-scale weak supervision. https://github.com/openai/whisper, 2022.
• L. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–286, February 1989.
• T. Rahman, F. Ahmed, and F. Chowdhury. Data augmentation for bangla speech recognition using synthetic audio generation. arXiv preprint arXiv:2501.08765, 2025.
• F. R. Rakib, S. S. Dip, S. Alam, N. Tasnim, M. I. H. Shihab, M. N. Ansary, S. M. Hossen, M. H. Meghla, M. Mamun, F. Sadeque, S. S. Chowdhury, T. Reasat, A. Sushmit, and A. I. Humayun. OOD-Speech: a large Bengali speech recognition dataset for out-of-distribution benchmarking. arXiv:2305.09688, May 2023.
• M. Rakib, M. I. Hossain, N. Mohammed, and F. Rahman. Bangla-Wave: improving Bangla automatic speech recognition utilizing n-gram language models. In Proc. 12th Int. Conf. Software and Computer Applications (ICSCA ’23), pages 297–301, June 2023.
• M. S. Islam Ridoy, S. Akter, and M. A. Rahman. Adaptability of ASR models on low-resource language: a comparative study of Whisper and Wav2Vec-BERT on Bangla. In Proc. 2nd Int. Conf. Next-Generation Computing, IoT and Machine Learning (NCIM), pages 1–6, June 2025.
• A. M. Samin, M. H. Kobir, S. Kibria, and M. S. Rahman. Deep learning based large vocabulary continuous speech recognition of an under-resourced language Bangladeshi Bangla. Acoustical Science and Technology, 42(5):252–260, 2021.
• A. M. Samin, M. H. Kobir, M. M. S. Rafee, M. F. Ahmed, M. Hasan, P. Ghosh, S. Kibria, and M. S. Rahman. BanSpeech: a multi-domain Bangla speech recognition benchmark toward robust performance in challenging conditions. IEEE Access, 12:34527–34538, 2024.
• J. R. Saurav, S. Amin, S. Kibria, and M. S. Rahman. Bangla speech recognition for voice search. In Proc. International Conference on Bangla Speech and Language Processing (ICBSLP), pages 1–4. IEEE, 2018.
• S. Sultana, M. A. H. Akhand, P. K. Das, and M. M. Hafizur Rahman. Bangla speech-to-text conversion using SAPI. In Proc. Int. Conf. Computer and Communication Engineering (ICCCE), pages 385–390, July 2012.
• S. Hosain Sumit, T. Al Muntasir, M. Arefin Zaman, R. Nath Nandi, and T. Sourov. Noise robust end-to-end speech recognition for Bangla language. In Proc. Int. Conf. Bangla Speech and Language Processing (ICBSLP), pages 1–5, September 2018.
• O. Tilk and T. Alum¨ae. Bidirectional recurrent neural network with attention mechanism for punctuation restoration. In Proc. Interspeech 2016, pages 3047–3051, September 2016.
• A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017.