Mangaluru: A study conducted by researchers on the Tulu language found significant scope for improvement in Google’s Tulu translation feature. The research team included two final-year MCA students from the Manipal Institute of Technology (MIT). Tulu language was included in Google translation feature, from June 27, 2024. Prajna Devadiga, who conducted the research along with Kavyashree, told TOI that the introduction of Tulu language to Google Translate saw widespread usage, including contributors uploading articles on Wikipedia. “This prompted us to study how accurate these translations actually are,” she said.The team evaluated Google’s Tulu translation using four established metrics. These included the BLEU (Bilingual Evaluation Understudy) score, which measures similarity between machine-generated and human translations; chrF++ (Character F-score plus plus), commonly used to assess translation quality in low-resource languages; Translation Error Rate (TER), which calculates the number of edits required to match a human reference; and Comet (Crosslingual Optimised Metric for Evaluation of Translation), a neural-based evaluation metric.Prajna said that, using these metrics, the researchers analysed how Tulu sentences should ideally be structured and compared them with Google’s translated outputs. A dataset was created by first manually translating select English sentences into accurate Tulu, and then comparing these reference translations with Google’s output.The study revealed significant scope for improvement. Since Tulu is a low-resource language, the system struggles with word-level meaning and context, often substituting Kannada words instead. The researchers noted that enhancing Google translator corpus and providing more focused training data for Tulu are crucial for accurate translation, and wider digital adoption of the language, said UB Pavanaja, director of Vishwa Kannada Foundation, who served as the external guide.The study was conducted under the guidance of Adarsh Rag, Ashwath, and Musica Supriya from the computer engineering department. The study is expected to be published soon.
