Tracing Interactional Change in Japanese EFL Learners’ Interactions with a Teacher-Designed AI Voice Chatbot: A Pilot Study
DOI:
https://doi.org/10.54855/ijaile.26331Keywords:
conversation partner, teacher-designed chatbot, EFL learnersAbstract
Although AI-mediated conversation practice is increasingly common in EFL teaching, qualitative analysis of learner–AI interaction has been insufficient. This exploratory pilot study examined Japanese EFL learners’ interactional change. Ten university students completed three five-minute sessions with a teacher-designed chatbot on Google AI Studio. Logs were analyzed quantitatively and qualitatively. Effect sizes suggested a two-phase pattern of observed change: large effects (d = 0.88–1.13) from Sessions 1 to 2 marked an initial adjustment phase, while medium effects (d = 0.59–0.66) from Sessions 2 to 3 indicated slowed quantitative growth alongside continued qualitative elaboration, particularly in topic development. Qualitative analysis identified both constraints (vocabulary gaps, abstract questions) and affordances (explanatory expansions, analytical reflections, self-directed topic drift). Comparison with peer-to-peer data highlighted distinctive features: longer turns, near absence of backchannels, and more frequent language-related episodes. These preliminary findings offer initial guidance for teachers developing AI conversation tools using accessible platforms.
References
Alenezi, A., & Alenezi, A. (2025). Evaluating the effectiveness of chatbot-assisted learning in enhancing English conversational skills among secondary school students. Education Sciences, 15(9), 1136. https://doi.org/10.3390/educsci15091136 DOI: https://doi.org/10.3390/educsci15091136
Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: Research synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning, 32(8), 827–877. https://doi.org/10.1080/09588221.2018.1535508 DOI: https://doi.org/10.1080/09588221.2018.1535508
Chapelle, C. A. (2001). Computer applications in second language acquisition. Cambridge University Press. https://doi.org/10.1017/CBO9781139524681 DOI: https://doi.org/10.1017/CBO9781139524681
Chapelle, C. A. (2009). The relationship between second language acquisition theory and computer-assisted language learning. The Modern Language Journal, 93, 741–753. https://doi.org/10.1111/j.1540-4781.2009.00970.x DOI: https://doi.org/10.1111/j.1540-4781.2009.00970.x
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). SAGE.
Ellis, R. (1994). The study of second language acquisition. Oxford University Press.
Fryer, L. K., & Carpenter, R. (2006). Bots as language learning tools. Language Learning and Technology, 10(3), 8–14. https://doi.org/10.64152/10125/44068 DOI: https://doi.org/10.64152/10125/44068
Gass, S. M., & Varonis, E. M. (1994). Input, interaction, and second language production. Studies in Second Language Acquisition, 16(3), 283–302. https://doi.org/10.1017/S0272263100013097 DOI: https://doi.org/10.1017/S0272263100013097
Horwitz, E. K., Horwitz, M. B., & Cope, J. (1986). Foreign language classroom anxiety. The Modern Language Journal, 70(2), 125–132. https://doi.org/10.1111/j.1540-4781.1986.tb05256.x DOI: https://doi.org/10.1111/j.1540-4781.1986.tb05256.x
Jeon, J. (2024). Exploring AI chatbot affordances in the EFL classroom: Young learners’ experiences and perspectives. Computer Assisted Language Learning, 37(1–2), 1–26. https://doi.org/10.1080/09588221.2021.2021241 DOI: https://doi.org/10.1080/09588221.2021.2021241
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868 DOI: https://doi.org/10.1177/00336882231162868
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012 DOI: https://doi.org/10.1016/j.jcm.2016.02.012
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863 DOI: https://doi.org/10.3389/fpsyg.2013.00863
Levy, M., & Stockwell, G. (2006). CALL dimensions: Options and issues in computer-assisted language learning. Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203708200 DOI: https://doi.org/10.4324/9780203708200
Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. C. Ritchie & T. K. Bhatia (Eds.), Handbook of second language acquisition (pp. 413–468). Academic Press. https://doi.org/10.1016/B978-012589042-7/50015-3 DOI: https://doi.org/10.1016/B978-012589042-7/50015-3
Markee, N. (2000). Conversation analysis. Lawrence Erlbaum Associates. https://doi.org/10.4324/9781410606471 DOI: https://doi.org/10.4324/9781410606471
Matiienko-Silnytska, A., Mikava, N., Savranchuk, I., Tkhor, N., & Poliakova, H. (2025). Conversational analysis of learner–AI chatbot interactions in developing spoken fluency. Arab World English Journal, 16(4), 224–238. https://doi.org/10.24093/awej/vol16no4.13 DOI: https://doi.org/10.24093/awej/vol16no4.13
Nadeem, S., Iftikhar, S., & Rasheed, A. (2025). Addressing the fear of speaking: The role of AI in reducing speaking anxiety in ESL learners. Journal of Applied Linguistics and TESOL, 8(2), 825–833. https://jalt.com.pk/index.php/jalt/article/view/693
Nguyen, C. V. (2025). Employing TALKPAL.AI to enhance speaking proficiency for Vietnamese adult learners: A literature review. International Journal of AI in Language Education, 2(2). 40–54. https://doi.org/10.54855/ijaile.25223 DOI: https://doi.org/10.54855/ijaile.25223
Nguyen, H. A. (2024). Harnessing AI-based tools for enhancing English speaking proficiency: Impacts, challenges, and long-term engagement. International Journal of AI in Language Education, 1(2), 18–29. https://doi.org/10.54855/ijaile.24122 DOI: https://doi.org/10.54855/ijaile.24122
Nguyen, L. A. D., & Le, T. T. P. (2025). Exploring the effects of an AI chatbot on emotional engagement in English speaking lessons: Insights from Call Annie. International Journal of AI in Language Education, 2(2), 79–99. https://doi.org/10.54855/ijaile.25225 DOI: https://doi.org/10.54855/ijaile.25225
Pica, T., Holliday, L., Lewis, N., & Morgenthaler, L. (1989). Comprehensible output as an outcome of linguistic demands on the learner. Studies in Second Language Acquisition, 11(1), 63–90. https://doi.org/10.1017/S027226310000783X DOI: https://doi.org/10.1017/S027226310000783X
Seedhouse, P. (2004). The interactional architecture of the language classroom: A conversation analysis perspective. Blackwell Publishing.
Sok, S., & Shin, H. W. (2025). Do interactions with ChatGPT influence L2 learners’ oral speaking ability, summarization ability, and perceptions of generative AI tasks? TESOL Quarterly, 59, S19–S51. https://doi.org/10.1002/tesq.70001 DOI: https://doi.org/10.1002/tesq.70001
Sullivan, G. M., & Feinn, R. (2012). Using effect size—or why the p value is not enough. Journal of Graduate Medical Education, 4(3), 279–282. https://doi.org/10.4300/JGME-D-12-00156.1 DOI: https://doi.org/10.4300/JGME-D-12-00156.1
Swain, M. (1985). Communicative competence: Some roles of comprehensible input and comprehensible output. In S. Gass & C. Madden (Eds.), Input in second language acquisition (pp. 235–253). Newbury House.
Swain, M. (2000). The output hypothesis and beyond: Mediating acquisition through collaborative dialogue. In J. P. Lantolf (Ed.), Sociocultural theory and second language learning (pp. 97–114). Oxford University Press.
Swain, M., & Lapkin, S. (1995). Problems in output and the cognitive processes they generate: A step towards second language learning. Applied Linguistics, 16(3), 371–391. https://doi.org/10.1093/applin/16.3.371 DOI: https://doi.org/10.1093/applin/16.3.371
Swain, M., & Lapkin, S. (1998). Interaction and second language learning: Two adolescent French immersion students working together. The Modern Language Journal, 82(3), 320–337. https://doi.org/10.1111/j.1540-4781.1998.tb01209.x DOI: https://doi.org/10.1111/j.1540-4781.1998.tb01209.x
Tarone, E. (1981). Some thoughts on the notion of communication strategy. TESOL Quarterly, 15(3), 285–295. https://doi.org/10.2307/3586754 DOI: https://doi.org/10.2307/3586754
Varonis, E. M., & Gass, S. M. (1985). Non-native/non-native conversations: A model for negotiation of meaning. Applied Linguistics, 6(1), 71–90. https://doi.org/10.1093/applin/6.1.71 DOI: https://doi.org/10.1093/applin/6.1.71
Wang, C., Zou, B., Du, Y., & Wang, Z. (2024). The impact of different conversational generative AI chatbots on EFL learners: An analysis of willingness to communicate, foreign language speaking anxiety, and self-perceived communicative competence. System, 127, 103533. https://doi.org/10.1016/j.system.2024.103533 DOI: https://doi.org/10.1016/j.system.2024.103533
Yin, Q., & Satar, M. (2020). English as a foreign language learner interaction with chatbots: Negotiation for meaning. International Online Journal of Education and Teaching, 7(2), 390–410. https://www.iojet.org/index.php/IOJET/article/view/707
Zhang, D., Wu, J. G., & Fu, Z. (2024). From shy to fly: Facilitating EFL learners’ willingness to communicate with an AI chatbot and an intelligent tutoring system. System, 127, 103501. https://doi.org/10.1016/j.system.2024.103501 DOI: https://doi.org/10.1016/j.system.2024.103501
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Ken’ichi Yamazaki (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
