Assessing GPT-4’s Reliability for Japanese Kanji Instruction: Accuracy, Prompt Control, and Contextual Sentences

Abstract

Example sentences and native-language translations are often seen as beneficial for vocabulary learning. However, manually creating such resources in multiple languages is labor-intensive. GPT-4 offers a promising solution by generating both example sentences and translations. This study evaluates GPT-4’s performance in an adaptive Kanji learning tool, focusing on translation accuracy, semantic adequacy, and the quality of example sentences. A systematic analysis of 50 Kanji—representing about 4 % of the commonly used characters—their most frequent compound words, and 185 example sentences reveals high translation accuracy and mostly coherent contextual sentences, though occasional hallucinations and inappropriate expressions occur. Strict formatting and content guidelines reduce such deviations but increase prompt complexity. The results indicate that GPT-4 Turbo can support learning effectively, provided careful review are ensured.