Speech Technologies in Indian Languages

The project titled “Speech Technologies in Indian Languages”(SP/2122/1960/CSMEIT/003119) is a sub-project under the National Language Translation Mission: Bhashini, funded by MeiTY, GoI, of which IIT Madras along with IIIT Hyderabad is a Consortium Leader. Under the mandate of this project, a team of faculties, research associates and interns have been working to produce a standard corpus of spontaneous speech in Indian Languages. This is to improve the results of automated speech transcriptions and translations that can be implemented to optimize Automated Speech Recognition(ASR) and Text-To-Speech(TTS) systems for various purposes.

Keeping in mind this broad objective, we present the collective efforts of the team dedicated to improving the automated speech transcripts and translations of the National Programme on Technology Enhanced Learning (NPTEL) technical lectures. This team, consisting of four interns, project associates, and several distinguished faculty members from IIT Madras and SNU Chennai, worked across three languages (Bangla, Malayalam, Telugu). We have attempted to provide parallel corpora for academic lectures and materials using the available database of NPTEL course lectures, many of which have already been translated and dubbed from English into regional Indian languages. The idea has been to identify and list out semantic, grammatical and contextual errors in these NPTEL courses and work towards fixing these issues, contributing to the curation of better machine language translations of technical lectures, ensuring the lectures are lucid and accessible to every learner, regardless of language background or linguistic competence.

In our experience with indigenous and open-source models, we have found significant shortcomings in the translation of these lectures. Notably, the translation of spontaneous speech must account for disfluencies, corrections, repetitions, code-mixing, code-switching, and the use of simpler expressions that are common in spoken language but rare in written texts. As language models are primarily trained on open-source written texts and frequently for major high-resource languages, their translated outputs often compromise the semantic and syntactic integrity of the original languages (Ramesh et al., 2022; Siripragada et al., 2020). Furthermore, we aim to retain the domain-specific terms within each lecture because our goal is to familiarize learners with English terminologies, enabling them to refer to original sources. Over-translation of these domain terms not only diverges from this goal but also fragments the overall semantic unity of the lecture (Ramesh et al., 2022).We have endeavored to produce corpora that consist of the original audio subtitles, the translation provided by a language model (here, BhashaVerse, IIIT Hyderabad), and our proposed translations, including a glossary of terms for which we have provided alternatives as needed. Through our informal field trials, we have identified many archaic and incorrect terms and expressions used in these lectures, which we have strived to correct in our translations. We hope to receive valuable feedback from our reviewers to further colloquialize these translations, maintain the dialectal styles inherent in various spontaneous speech corpora, and leverage overall translation accuracy (Siripragada et al., 2020).