Citation :

Md. Saiful Alam, A H M Ohidujjaman, Md. Nurullah Patwary, Humayra Akhter , 2026. "GenAI for Revolutionising Writing Assessment: A Case Study on the Comparative Advantages of ChatGPT 3.5, ChatGPT Edu, and Human Teachers in Assessing Essays" International Journal of Community Empowerment & Society Administration [IJCESA]  Volume 3, Issue 1: 01-11.

Abstract :

The historical normativity is that education and AI make a parallel of reciprocal effect systems, where innovations in one field help the other progress. One of the recent developments in scientific innovations relating to writing pedagogy is the emergence of large language models such as ChatGPT. This has created novel prospects for transforming educational practices, particularly in the realm of writing assessment. This development has prompted the researchers to examine how ChatGPT Edu performs over its predecessor, ChatGPT 3.5, in the framework of Automated Essay Scoring (AES). This study specifically evaluates the performance of ChatGPT Edu and ChatGPT 3.5 in comparison to human teachers in assessing student writing. It applies a qualitative case study research design to address the research problems. To collect the required amount of data, the study uses a variety of data sources, such as EFL students’ handwritten essays, ChatGPT-produced grades, comments, evaluations, as well as thorough evaluations from an experienced EFL writing teacher. The study applies the summative content analysis approach to analyze the qualitative data. The findings reveal that ChatGPT Edu is superior to ChatGPT 3.5 in all three assessment dimensions, and it can contribute to a higher degree of reliability and pedagogical value in second language writing assessment. The findings also demonstrate that ChatGPT Edu has a stronger capacity to score that closely matches teacher-assigned scores. These findings suggest that the cutting-edge AI tools can be of great support to EFL teachers in assessing writing. Furthermore, this study also contributes to the ongoing discussion on how successfully AI-supported instruments can be integrated into EFL education.

References :

[1] Abramski Katherine, Citraro Salvatore, Lombardi Luigi, et al., Cognitive network science reveals bias in GPT-3, GPT-3.5 Turbo, and GPT-4 mirroring math anxiety in high-school students, Big Data and Cognitive Computing. 7(3) (2023) 124. https://doi.org/10.3390/bdcc7030124.

[2] Adeshola Ibrahim and Adepoju Adeola Praise, The opportunities and challenges of ChatGPT in education, Interactive Learning Environments. 32(10) (2023) 1-14. https://doi.org/10.1080/10494820.2023.2253858.

[3] Ajay Helen B., Tillett Paul  I., and Page Ellis Betten, The analysis of essays by computer (AEC II): Final report (Technical Report No. 8 0102), U. S. Department of Health, Education, and Welfare, Office of Education, National Center for Educational Research and Development. (1973)

[4] Altamimi Ahmed B., Effectiveness of ChatGPT in essay auto-grading. In: Proceedings of the 2023 International Conference on Computing, Electronics & Communications Engineering (iCCECE); Aug 14-16, 2023; Swansea, UK. (2023)102–106. https://doi.org/10.1109/iCCECE59400.2023.10238541.

[5] Ali Kamran, Barhom Noha, Tamimi Faleh and Duggal, Monty, ChatGPT—A double‐edged sword for healthcare education? Implications for assessments of dental students, European Journal of Dental Education. 28(1) (2023) 206-211. https://doi.org/10.1111/eje.12937

[6] Amineh Roya Jafari, and Asl Hanieh Davatgari, Review of constructivism and social constructivism, Journal of social sciences, literature and languages. 1(1) (2015) 9-16. https://www.blue-ap.com/J/List/4/iss/volume%2001%20(2015)/issue%2001/2.pdf

[7] Asmawi Adelina and Alam M. Saiful, Qualitative research: Understanding its underlying philosophies. Forum for Education Studies. 2 (2) (2024) 1320.https://doi.org/10.59400/fes.v2i2.1320

[8] Bahroun Zineb, Anane Chakib, Ahmed Véronique et al., The potential of ChatGPT in education: A systematic bibliometric review, Sustainability. 15(17) (2023) 12983. https://doi.org/10.3390/su151712983

[9] Božić Velibor and Poola, Indrasen, ChatGPT and education [preprint]. (2023) [cited 2025 Aug 13]; Available from: https://doi.org/10.13140/RG.2.2.18837.40168

[10] Bouziane Karima and Bouziane Abdelmounim, AI versus human effectiveness in essay evaluation, Discover Education. 3(1) (2024) 201. https://doi.org/10.1007/s44217-024-00320-6

[11] Brown Terry, Smith Rebecca and Lee Alan, How close is ChatGPT to human graders in assessing student essays?, Int J Artif Intell Educ. 34(1) (2024) 112-128. https://doi.org/10.1007/s40593-023-00312-9

[12] Bukowski Marcin and Tokowicz Nora, Automated essay scoring: A review of the literature, Behavior Research Methods. 53 (2021) 2090–2113. https://doi.org/10.3758/s13428-020-01509-1.

[13] Chang Li Hsin and Ginter Filip, Automatic short answer grading for Finnish with ChatGPT. In: Proceedings of the AAAI Conference on Artificial Intelligence. 38(21), (Mar 2024) 23173–23181. Menlo Park (CA): AAAI Press (2024). https://doi.org/10.1609/aaai.v38i21.30363.

[14] Chen Shen, Li Yingya, Lu Sheng, et al., Evaluating the ChatGPT family of models for biomedical reasoning and classification, Journal of the American Medical Informatics Association. 31(4) (2024) 940-948. https://doi.org/10.1093/jamia/ocad256.

[15] Crompton Helen and Burke Diane, Artificial intelligence in higher education: the state of the field, International Journal of Educational Technology in Higher Education. 20(22) (2023) 1-22. https://doi.org/10.1186/s41239-023-00392-8

[16] Devitt Michael, Realism and Truth. Princeton, NJ, USA: Princeton University Press. (1997)

[17] deWinter  Joost C.F., Dodou Dimitra and Stienen Arno H.A., ChatGPT in Education: Empowering Educators through Methods for Recognition and Assessment, Informatics. 10(4) (2023) 87. https://doi.org/10.3390/informatics10040087.

[18] Dong Fei, Zhang Yue and Yang Jie, Attention-based recurrent convolutional neural network for automatic essay scoring. In: Levy, R., Specia, L. editors. Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017); 2017 Aug (3-4); Vancouver, Canada. Association for Computational Linguistics. (2024) 153–162. https://doi.org/10.18653/v1/K17-1017

[19] Dwivedi Yogesh, K., Kshetri Nir., Hughes Laurie, et al., “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges, and implications of generative conversational AI for research, practice, and policy, International Journal of Information Management. 71 (2023) 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642.

[20] Felix Catherine V., Ed., The role of the teacher and AI in education. In International perspectives on the role of technology in humanizing higher education. West Yorkshire, United Kingdom: Emerald Publishing Limited. (2020) 33-48.

[21] Firat Mehmet, How ChatGPT can transform autodidactic experiences and open education? [preprint], Center for Open Science. (2023). [cited 2025 Aug 4]. Available from: https://doi.org/10.31219/osf.io/9ge8m

[22] Flyvbjerg Bent, (Ed.), Case study. In: Denzin NK, Lincoln YS, editors. The Sage Handbook of Qualitative Research. (4th ed.). Thousand Oaks (CA), USA: Sage Publications. (2011) 301–316.

[23] Foltz Peter W., Laham David and Landauer Thomas K., Automatic essay assessment, Assessment in Education: Principles, Policy & Practice. 10(3) (2003) 295–308. https://doi.org/10.1080/0969594032000148154.

[24] Gardner John, O'Leary Michael, and Yuan Li, Artificial intelligence in educational assessment: ‘Breakthrough? or buncombe and ballyhoo?, Journal of Computer Assisted Learning. 37(5) (2021) 1207-1216. https://doi.org/10.1111/jcal.12577

[25] Harré Romano and Krausz Michael, Varieties of Relativism. Oxford, UK: Blackwell. (1996)

[26] Herbold Steffen, Hautli Janisz Annette, Heuer Utte, et al., A large-scale comparison of human-written versus ChatGPT-generated essays, Sci. Rep. 13(1) (2023)18617. https://doi.org/10.1038/s41598-023-45644-9.

[27] Jackaria Potching M., Hajan Bonjovi H., and Mastul Al-Rashiff H., A comparative analysis of the rating of college students’ essays by ChatGPT versus human raters, International Journal of Learning, Teaching and Educational Research. 23(2) (2024) 478-492. https://doi.org/10.26803/ijlter.23.2.23

[28] Javaid Mohd, Haleem Abid and Singh Ravi Pratap et al., Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system, BenchCouncil Transactions on Benchmarks, Standards and Evaluations. 3(2) (2023) 100115. https://doi.org/10.1016/j.tbench.2023.100115

[29] Jukiewicz Marcin, The future of grading programming assignments in education: The role of ChatGPT in automating the assessment and feedback process, Thinking Skills and Creativity. 52 (2024) 101522. https://doi.org/10.1016/j.tsc.2024.101522

[30] Kayyali Mustafa, Ed., Future possibilities and challenges of AI in education. In: Sharma RC, Bozkurt A, editors, Transforming education with generative AI: prompt engineering and synthetic content creation. 1st ed. Pennsylvania, United States, Hershey: IGI Global. (2024) 118–37.

[31] Kınık Busra and Çetin Husein, Human vs. AI: The use of ChatGPT in writing assessment, Advances in Educational Technologies and Instructional Design Book Series. Hershey, Pennsylvania (PA), United States: IGI Global. (2023) 194–215. https://doi.org/10.4018/979-8-3693-0353-5.ch009

[32] Koubaa Anis, GPT 4 vs. GPT 3.5: A concise showdown [preprint], Preprints.org; 2023 Mar 24 [cited 2025 Aug 4]. Available from: https://doi.org/10.20944/preprints202303.0422.v1

[33] Lampropoulos Georgios and Papadakis Stamatios, (Ed.), The Educational Value of Artificial Intelligence and Social Robots. In: Lampropoulos, G., Papadakis, S., Social Robots in Education. Studies in Computational Intelligence. Cham, Canton of Zug, Switzerland: Springer. (2025) 3-15.

[34] Latif Ehan and Zhai Xiaoming, Fine-tuning ChatGPT for automatic scoring, Computers and Education: Artificial Intelligence. 6 (2024) 100210. https://doi.org/10.1016/j.caeai.2024.100210

[35] Michel-Villarreal Rosario, Vilalta-Perdomo Eliseo, Salinas-Navarro David Ernesto, et al., Challenges and opportunities of generative AI for higher education as explained by ChatGPT. Education Sciences. 13(9) (2023) 856. https://doi.org/10.3390/educsci13090856

[36] Mizumoto Atushi and Eguchi Masaki, Exploring the potential of using an AI language model for automated essay scoring, Research Methods in Applied Linguistics. 2(2) (2023) 100050. https://doi.org/10.1016/j.rmal.2023.100050.

[37] Okunlaya Rifqah Olufunmilayo, Syed Abdullah Norris and Alinda Alias Rose, Artificial intelligence (AI) library services innovative conceptual framework for the digital transformation of university education, Library Hi Tech. 40(6) (2022) 1869-1892. https://doi.org/10.1108/LHT-07-2021-0242

[38] Page Ellis Betten, The use of the computer in analyzing student essays, International Review of Education. 14(2) (1968) 210–225. http://www.jstor.org/stable/3442515.

[39] Page Ellis Betten, The imminence of... grading essays by computer, Phi Delta Kappan. 47(5) (1966) 238–243. Available from: https://www.jstor.org/stable/20371545

[40] Page Ellis Betten, Ed., Project Essay Grade: PEG. In Shermis, M. D., Jill B., Automated essay scoring: A cross-disciplinary perspective. Mahwah, New Jersey, USA: Lawrence Erlbaum Associates. (2003) 43-45.

[41] Polverini Guilia and Gregorcic Bor, How understanding large language models can inform the use of ChatGPT in physics education, European Journal of Physics. 45(2) (2025) 025701. https://:10.1088/1361-6404/ad1420

[42] Potchong, M., Hajan, Bonjovi H., and Mastul, Al-Rashiff H., A comparative analysis of the rating of college students’ essays by ChatGPT versus human raters, Int’l Journal of Learning, Teaching and Educational Research. 23(2) (2024) 478–92. https://doi.org/10.26803/ijlter.23.2.23

[43] Ramesh Dadi and Sanampudi Suresh Kumar, An automated essay scoring system: a systematic literature review, Artificial Intelligence Review. 55(3) (2022) 2495-2527. https://doi.org/10.1007/s10462-021-10068-2

[44] Reddit community post, OpenAI introduces ChatGPT Edu for universities [Online forum post], Reddit. (May 30, 2024). https://www.reddit.com/r/OpenAI/comments/1d4df2e/

[45] Rospigliosi Pricles Asher, Artificial intelligence in teaching and learning: what questions should we ask of ChatGPT?, Interactive Learning Environments. 31(1) (2023) 1-3. https://doi.org/10.1080/10494820.2023.2180191

[46] Rudner Lawrence M., and Liang Tahung, Automated essay scoring using Bayes’ theorem, The Journal of Technology, Learning and Assessment. 1(2) (2002). [cited 2025 Aug 13]; Available from: https://ejournals.bc.edu/index.php/jtla/article/view/1668

[47] Shafik Wasswaa, Introduction to ChatGPT. In Advanced Applications of Generative AI and Natural Language Processing Models. Hershey, Pennsylvania, United States: IGI Global. (2024) 1-25.

[48] Scheschenja Michael, Viniol Simon, Bastian Moritz B., et al., Feasibility of GPT-3 and GPT-4 for in-depth patient education prior to interventional radiological procedures: a comparative analysis, Cardiovascular and interventional radiology. 47(2) (2024) 245-250. https://doi.org/10.1007/s00270-023-03563-2.

[49] Schiff Daniel, 2022. Education for AI, not AI for education: The role of education and ethics in national AI policy strategies, International Journal of Artificial Intelligence in Education. 32(3) (2022) 527-563. https://doi.org/10.1007/s40593-021-00270-2

[50] Shermis Mark D., Mzumara Howard, Olson Jenifar, et al., On line grading of student essays: PEG goes on the World Wide Web. Assessment & Evaluation in Higher Education. 26(3) (2001) 247–258. https://doi.org/10.1080/02602930120052404.

[51] Slinkard Karen and Singleton Vernon L., Total phenol analysis: automation and comparison with manual methods, American Journal of Enology and Viticulture. 28(1) (1997) 49-55. https://doi.org/10.5344/ajev.1974.28.1.49.

[52] Steiss Jacob, Tate Tamara, Graham Steve, et al., Comparing the quality of human and ChatGPT feedback of students’ writing, Learning and Instruction. Jun 1 (2024) 91:101894–4. https://doi.org/10.1016/j.learninstruc.2024.101894

[53] Su Jiahong and Yang Weipemg, Unlocking the power of ChatGPT: A framework for applying generative AI in education, ECNU Review of Education. 6(3) (2023) 355-366. https://doi.org/10.1177/20965311231168423

[54] Tahiru Fati, AI in education: A systematic literature review, Journal of Cases on Information Technology (JCIT). 23(1) (2021) 1- 20. https://doi.org/10.4018/JCIT.2021010101

[55] Yusuf Abdullah, Pervin Nasrin and Román González Marcos, Generative AI and the future of higher education: a threat to academic integrity or reformation? Evidence from multicultural perspectives, Int J Educ Technol High Educ. 21(1) (2024) 21. https://doi.org/10.1186/s41239-024-00453-6

[56] Zhang Liang and Wang Hao, Automated essay scoring using generative AI models: An empirical study with ChatGPT, Comput Educ. (2023) 180:104524. https://doi.org/10.1016/j.compedu.2022.104524.

[57] Zhai Xiaoming and Nehm Ross H., AI and formative assessment: The train has left the station, Journal of Research in Science Teaching. 60(6) (2023) 1390-1398. https://doi.org/10.1002/tea.21885

[58] Zhai Xuesong, Xiaoyan Chu, Ching Sing Chai, et al., A Review of Artificial Intelligence (AI) in Education from 2010 to 2020, Complexity. (1) (2021) 8812542. https://doi.org/10.1155/2021/8812542

Keywords :

ChatGPT, ChatGPT Edu, Language Assessment, Human Assessment.