Effectiveness of ChatGPT Feedback on Low-Level English Students’ Writing Proficiency and Their Perceptions: An Experimental Study

Authors

DOI:

https://doi.org/10.54855/ijaile.26324

Keywords:

ChatGPT, Writing skills, Low-level students

Abstract

The purpose of this study is to explore the effectiveness of ChatGPT’s feedback on the writing of non-English major university students and their perceptions of this implementation. Participants in this study had a low level of English (A2). Researchers employed an experimental research design to achieve the research objectives. There were 42 students in total, including 21 students in the experimental group (EG) and 21 in the control group (CG). EG received ChatGPT feedback after writing draft 1, whereas CG received peer feedback and general instructions from teachers.  Data were collected through pre- and post-tests, and semi-structured interviews. Paired sample t-tests revealed a significant improvement in mean scores between pre-and post-tests of EG (p=.00<.05), in contrast to CG’s. While students complimented ChatGPT on its immediacy, dialogic, and personalized responses, the tool's prompting difficulties and blind acceptance were their major concerns. Findings suggest the need for clear AI literacy training for students and a ‘human-in-the-loop’ approach in which ChatGPT feedback incorporates teachers’ guidance.

Author Biographies

  • Vu Quynh Hoa, Faculty of Foreign Languages, Hanoi University of Science and Technology

    Ms. Vu Quynh Hoa has been working as the English lecturer of the Faculty of Foreign Languages, Hanoi University of Science and Technology (HUST) since 2019. She holds an MA in TESOL from the University of Languages and International Studies – Vietnam National University, Vietnam; an MA in Business Management in Norwich Business school – University of East Anglia, UK and a BA in English Teacher Education from the University of Languages and International Studies – Vietnam National University, Vietnam, all earned with distinction. She participated in the teaching procedure for the STEM-focused EnglishWorks program co-organized by Hanoi University of Science and Technology, U.S Embassy and SEAMEO RETRAC in 2024. Her research in ELT primarily focuses on enhancing students’ engagement and blended learning. Her studies have been published in nationally recognized journals, including the NUAE Journal of Art Education and the Journal of Education and Society.

  • Nguyen Kim Phuc, Faculty of Foreign Languages, Hanoi University of Science and Technology

    Ms. Nguyễn Kim Phúc has been working as the English lecturer of the Faculty of Foreign Languages, Hanoi University of Science and Technology (HUST) since 2019. She is a certified VSTEP Speaking and Writing Assessor, accredited by the University of Languages and International Studies.  She holds an MA in Applied Linguistics TESOL from the University of Queensland, Australia, and a BA in English (Honors Program) from Vietnam National University, both earned with distinction. Her research in ELT primarily focuses on enhancing students’ productive skills. Her studies have been published in nationally recognized journals, including the NUAE Journal of Art Education and the Journal of Education and Society. She also presented at the Conference on “Improving online teaching quality” of the Academy of Finance in 2022 and co-presented at VietTESOL International Convention 2024.

References

Abdi, M. M., Ananta, B. D. B, & Nurbatra, L. H. (2024). Exploring the use ChatGPT in improving students' paragraph composition skills. Focus on ELT Journal, 6 (4), 17-36. https://doi.org/10.14744/felt.6.4.2

Aldabbus, S., & Almansouri, E. (2022). Academic writing difficulties encountered by University EFL Learners. British Journal of English Linguistics, 10(3), 1–11. https://doi.org/10.37745/bjel.2013/vol10n3111

Ardito, C. G. (2024). Generative AI detection in higher education assessments. New Dir Teach Learn, n/a(n/a). https://doi.org/10.1002/tl.20624

Asadi, M., Ebadi, S., & Mohammadi, L. (2025). The impact of integrating ChatGPT with teachers’ feedback on EFL writing skills. Thinking Skills and Creativity, 56, 101766. https://doi.org/10.1016/j.tsc.2025.101766

Chan, C., & Hu, W. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20(43), 1–18. https://doi.org/10.1186/s41239-023-00411-8.

Chae, Y., & Davidson, T. (2025). Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning. Sociological Methods & Research, 0(0). https://doi.org/10.1177/00491241251325243

Chun, H. L., Lee, S. M., & Park, I. E. (2021). A systematic review of AI technology use in English education. Multimedia-Assisted Language Learning, 24(1), 87–103. https://doi.org/10.15702/mall.2021.24.1.87

Firkins, A., Forey, G., & Sengupta, S. (2007). Teaching writing to low proficiency EFL students. ELT Journal, 61(4), 341–352. https://doi.org/10.1093/elt/ccm052

Fleckenstein, J., Liebenow, L. W., & Meyer, J. (2023). Automated feedback and writing: A multi-level meta-analysis of effects on students' performance. Frontiers in Artificial Intelligence, 6. https://doi.org/10.3389/frai.2023.1162454

Google for Educators. (2024b). Generative. Mountain View, CA: AI for educators. Google LLC. https://skillshop.exceedlms.com/student/path/1176018.

Graham, S., Harris, K., & Hebert, M. (2011). Informing writing: The benefits of formative assessment: A report from Carnegie Corporation of New York. New York: Carnegie Corporation.

Grimes, D. & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. The Journal of Technology, Learning and Assessment, 8(6). https://www.ejournals.bc.edu/index.php/jtla/article/view/1625/1469

Hasanah, U., & Safitri, W. (2024). Student perception toward ChatGPT in writing skill in English study program Universitas Muhammadiyah of Bengkulu. Teaching and Learning English in the Language Education (Telle), 4(2). https://doi.org/10.36085/telle.v4i2.6937

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Ho, N. N., Tra, T. C. T., Pham, T. Q. & Tran, P. T. (2026). Discrepancies between ChatGPT and Vietnamese EFL teachers in writing assessment. International Journal of AI in Language Education, 3(1), 1–16. https://doi.org/10.54855/ijaile.26311

Jacobs, H. L., Zinkgraf, S. A., Wormouth, D.R., Hartfiel, V. F., & Hughey, J. B. (1981). Testing ESL composition: A practical approach. Newbury House. ERIC ED217708. https://eric.ed.gov/?id=ED217708

Kurt, G., & Kurt, Y. (2024). Enhancing L2 writing skills: ChatGPT as an automated feedback tool. Journal of Information Technology Education: Research, 23, Article 24. https://doi.org/10.28945/5370

Lại, T. K. L. & Trịnh, H. L. (2025). TÁC ĐỘNG CỦA CHATGPT ĐỐI VỚI NĂNG LỰC VIẾT HỌC THUẬT CỦA SINH VIÊN CHUYÊN NGÀNH TIẾNG ANH TẠI MỘT TRƯỜNG ĐẠI HỌC. Tạp chí Nghiên cứu nước ngoài, 41(3), 92-109. https://doi.org/10.63023/2525-2445/jfs.ulis.5514

Lam, R. (2018). Feedback in writing portfolio assessment. In R. Lam (Ed.), Portfolio assessment for the teaching and learning of writing, (pp. 59–72). Springer. https://doi.org/10.1007/978-981-13-1174-1_5

Ljubojević, D. (2024). Student feedback on the effectiveness of ChatGPT for essay writing. In Proceedings of the 15th International Conference on e-Learning 2024 (pp. 116-126). CEUR Workshop Proceedings. https://ceur-ws.org/Vol-3938/Paper_12.pdf

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410

Mahapatra, S. (2024). Impact of ChatGPT on ESL students’ academic writing skills: A mixed methods intervention study. Smart Learning Environments, 11(9). https://doi.org/10.1186/s40561-024-00295-9

Meniado, J. C., Duong, T. T. H., Panyadilokpong, N., & Lertkomolwit, P. (2024). Using ChatGPT for second language writing: Experiences and perceptions of EFL learners in Thailand and Vietnam. Computers and Education: Artificial Intelligence, 7, 100313. https://doi.org/10.1016/j.caeai.2024.100313

Nguyen, D. M. Q. (2026). The role of ChatGPT in providing written corrective feedback for EFL academic writing: A literature review. International Journal of AI in Language Education, 3(1), 35–47. https://doi.org/10.54855/ijaile.26313

Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144. https://doi.org/10.1016/j.edurev.2013.01.002

Poláková, P., & Ivenz, P. (2024). The impact of ChatGPT feedback on the development of EFL students’ writing skills. Cogent Education, 11, 2410101. https://doi.org/10.1080/2331186X.2024.2410101

Rahman, N., Zulkornain, H., & Hamzah, H. (2022). Exploring artificial intelligence using automated writing evaluation for writing skills. Environment-Behaviour Proceedings Journal, 7(SI9), 547–553. https://doi.org/10.21834/ebpj.v7iSI9.4304

Setyowati, L., Sukmawan, S., & El-Sulukiyyah, A. A. (2020). Exploring the use of ESL composition profile for college writing in the Indonesian context. International Journal of Language Education, 4(2), 171–182. https://doi.org/10.26858/ijole.v4i2.13662

Stillman, J. (2025, February 25). How to write the perfect AI prompt, according to OpenAI President Greg Brockman. Inc.com. https://www.inc.com/jessica-stillman/how-to-write-perfect-ai-prompt-openai-greg-brockman.html

Shermis, M. & Wilson, J. (Eds). (2024). Routledge international handbook of automatd essay evaluation. Routledge (in press).

Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students’ writing. Learning and Instruction, 91, 101894. https://doi.org/10.1016/j.learninstruc.2024.101894

Texas A&M University Writing Center. (2026). Formative feedback. Texas A&M University. https://writingcenter.tamu.edu/faculty/resources/formative-feedback.html

Wang, M., & Guo, W. (2023. The potential impact of ChatGPT on education: Using history as a rearview mirror. ECNU Review Education. https://doi.org/10.1177/20965311231189826.

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 24824–24837. https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf

Wilson, J., Myers, M. C., & Potter, A. (2022). Investigating the promise of automated writing evaluation for supporting formative writing assessment at scale. Assessment in Education: Principles, Policy & Practice, 1–17. https://doi.org/10.1080/0969594X.2022.2025762

Woo, D. J., Guo, K., & Susanto, H. (2023). Cases of EFL secondary students' prompt engineering pathways to complete a writing task with ChatGPT. arXiv. https://doi.org/10.48550/arXiv.2307.05493

Zhang, R. & Chen, J. (2026). The effects of ChatGPT on EFL students’ writing skills: A meta-analysis. System, 140, 104043. https://doi.org/10.1016/j.system.2026.104043

Zhang, Z. (V.), & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36, 90–102. https://doi.org/10.1016/j.asw.2018.02.004

Downloads

Published

08/31/2026

Issue

Vol. 3 No. 2 (2026): AI in Language Education

Section

Research Articles

License

Copyright (c) 2026 Vu Quynh Hoa, Nguyen Kim Phuc (Author)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Vu, Q. H., & Nguyen, K. P. (2026). Effectiveness of ChatGPT Feedback on Low-Level English Students’ Writing Proficiency and Their Perceptions: An Experimental Study. International Journal of AI in Language Education, 3(2), 55-70. https://doi.org/10.54855/ijaile.26324

Similar Articles

21-30 of 36

You may also start an advanced similarity search for this article.