Artificial intelligence in foreign language teaching: Evaluating the reliability of large language models with a focus on Serbian as a foreign language
In foreign language teaching, language competence implies, among other things, understanding dialectisms, phraseology, idioms, proverbs, and gnomes. The aim of this paper is to explore the possibilities of using artificial intelligence at higher levels of foreign language teaching, using Serbian as...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
University of Belgrade, Teacher Education Faculty
2025-07-01
|
| Series: | Inovacije u Nastavi |
| Subjects: | |
| Online Access: | http://www.inovacijeunastavi.rs/en/38-2-7en/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | In foreign language teaching, language competence implies, among other things, understanding dialectisms, phraseology, idioms, proverbs, and gnomes. The aim of this paper is to explore the possibilities of using artificial intelligence at higher levels of foreign language teaching, using Serbian as an exemplary foreign language. Three large language models (Claude, ChatGPT, and Gemini) were used for the purposes of this research. Three texts of a different level of difficulty, and written in different Serbian language dialects, were entered into the three language models, and the language models were asked to translate the texts into the standard Serbian language. The research sample consisted of 113 respondents – non-philologists, foreign language teachers, and teachers of Serbian as a mother tongue or a foreign language. An anonimous questionnaire, created on Google Forms, was the research instrument and it was distributed to the evaluators via email, i.e., to their channels and profiles on social networks. Based on a predefined list of criteria, the research participants had to evaluate the quality of the texts, without knowing that the texts were translated by artificial intelligence. Descriptive statistics techniques were used to process the obtained statistical data, as well as the one-way ANOVA with post-hoc tests (Bonferroni test). The research results indicate that Claude and ChatGPT can be considered reliable in translating texts written in dialects. When it comes to the lexical accuracy of the texts, the texts generated using these two tools were evaluated as very reliable, making it easier for students to understand dialects, phraseology, idioms, and proverbs in Serbian language. Based on the obtained results, the paper offers didactical implications for using artificial intelligence in foreign language teaching. |
|---|---|
| ISSN: | 0352-2334 2335-0806 |