MACHINE TRANSLATION OF METALLURGICAL SCIENTIFIC TEXTS: A COMPARATIVE STUDY OF CHATGPT AND DEEPL TRANSLATOR

Abstract

The article analyzes the accuracy of written translation of a scientific text in the field of metallurgy using a chatbot and virtual assistant powered by generative artificial intelligence (ChatGPT) and the neural machine translation service DeepL Translator. For the experimental study, an original article from a category “A” scientific journal was used. It was rich in metallurgical terminology and also included terminology from physics and chemistry. The text volume exceeded 30,000 characters (excluding spaces). Within this study, the accuracy of the translated text in conveying the main content was measured by calculating the number of errors. To assess the quality of domainspecific terminology reproduction, a bilingual glossary of 100 terminological units was compiled. As a result of analyzing approaches to evaluating the translated text by both humans and automatic systems, the author identified three key types of errors. They include those that change the meaning of the source text; those that may allow ambiguous interpretation; and those that do not affect the meaning of the source text. Each type of error is illustrated with specific examples. The results of the quantitative analysis of empirical data are presented in tables, followed by their interpretation, which served as the basis for drawing conclusions. The experiment confirmed the hypothesis put forward at the initial stage of the study: different machine translation systems indeed provide translations of the same scientific text in metallurgy with varying quality. According to the defined criteria, the chatbot and virtual assistant with generative AI, ChatGPT, demonstrated better performance, as its output text was more coherent and easier to comprehend. However, neither ChatGPT nor DeepL Translator was able to provide a fully correct and finalized translation that would be unambiguously clear to a specialist in metallurgy without human post-editing. In conclusion, the use of machine translation systems in the field of metallurgy can be useful when the purpose of translation is general acquaintance with a scientific text.