Linguistic Analysis of Machine Translation Quality
“Usually what you hear about when people complain about machine translation is that it used the wrong word or didn't quite match the appropriate tone of a message, but looking at gendered languages vs non-gendered languages, verb tenses and aspects that aren't equivalent between languages, or transitivity of verbs are also important aspects of translation that people need to be focusing on for machine translators.”
- Jackson Wolf, LING 1000 TA and student mentor, Spring 2023
Machine translation (MT), a language technology that takes text or speech in one language and transforms it into another language, can help people communicate in multilingual settings. Whether you're reading a website, chatting with someone online, or traveling to a foreign country, machine translation can assist you in understanding content or conversations that would otherwise be challenging due to language barriers. However, current commercial MT systems are not always able to handle factors inherently encoded in language, such as gender of referents (Savoldi et al., TACL 2021) and pronoun use (Lauscher et al., ACL 2023).
Lauren Strobl, Lyosha Genzel, Michelle Hu, and Sanchi Gupta cataloged issues with machine translation by comparing the accuracies between typological diverse languages (English <> Punjabi, Russian <> Arabic, and Kazakh <> German). Explore their work below to discover the issues they uncovered!
Cite this project as: Strobl, L., Genzel, L., Hu, M., & Gupta, S. P4: Linguistic Analysis of Machine Translation (MT) Quality. Under the supervision of professor Sue Lorenson and teaching assistants Lauren Levine and Evelyn Diaz-Iturriaga. LING 1000: Introduction to Language, Georgetown University. Fall 2023.
"Let It Go" according to Google Translate
The linguistic "awkwardness" resulting from machine translation can be quite humorous! Watch the video on the left from Twisted Translations, a YouTube channel dedicated to performing musical parodies based on mistranslation mishaps!
Click on the links below to see other students' work on this topic
Angiono, G., Chen, M., Darensburg, S., Filippova, M., Harriman, B., Lee-Caracci, K., Seymour, A., & Zabelski, A. Machine Translation Quality: Machine Analysis. Under the supervision of professor Lara Bryfonski and teaching assistant Jackson Wolf. LING 1000: Introduction to Language, Georgetown University. Spring 2024.
For further information, we direct you to the following resources:
Lauscher, A., Nozza, D., Miltersen, E., Crowley, A., & Hovy, D. (2023). What about “em”? How Commercial Machine Translation Fails to Handle (Neo-)Pronouns. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 377–392, Toronto, Canada. Association for Computational Linguistics.
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021). Gender Bias in Machine Translation. Transactions of the Association for Computational Linguistics, 9:845–874.