Enhancing Short Essay Question Quality in MBBS Course: A Comparative Study of ChatGPT and Human Collaboration
DOI:
https://doi.org/10.37018/JFJMU/5054Keywords:
ChatGPT, short essay question, artificial intelligence, teacher collaborationAbstract
Background: The short essay questions quality plays an important role in assessing students' knowledge and understanding in educational settings. This study aims to enhance the quality of short essay questions by using ChatGPT and teacher collaboration. Objective of the study is to evaluate the short essay questions for a college-level MBBS course made by ChatGPT and human source.
Materials and methods: This qualitative exploratory study aimed to develop and evaluate essay questions for a college-level MBBS course at Fatima Memorial Hospital. The study involved selecting course learning objectives and developing questions with ChatGPT 3.5 and specialists. Both groups created 20 questions, which were reviewed by independent experts using a checklist with five components: clarity, problem inclusion, structure, English composition, and appropriate length. Questions were rated on a Likert scale from 1 to 5. SPSS version 25 was used for statistical analysis, including t-tests to compare ratings. The study found differences in quality and effectiveness between AI and human-generated questions.
Results: Data analysis was done which showed the mean scores given by human evaluators and AI. Human superseded AI in clarity with the mean score given by human evaluators was 3.51, while AI systems had 3.41. AI accomplished better in structuring the short essay question with the imply rating of 3.63. (p=0.557). AI was advanced in demonstrating the problem inclusion then the human with the score of 3.13 (p=0.774). SEQS made with the aid of the human had good English composition compared to AI (p=0.466). Appropriate length of question became the best factor in which the human and AI each completed same (p=0.917).
Conclusion: This study provides a comprehensive analysis by comparing human and ChatGPT in the quality of short essay questions. The results indicate that AI has the capability to replicate human judgment in certain aspects of question.
Downloads
Published
How to Cite
Issue
Section
License
The Journal of Fatima Jinnah Medical University follows the Attribution Creative Commons-Non commercial (CC BY-NC) license which allows the users to copy and redistribute the material in any medium or format, remix, transform and build upon the material. The users must give credit to the source and indicate, provide a link to the license, and indicate if changes were made. However, the CC By-NC license restricts the use of material for commercial purposes. For further details about the license please check the Creative Commons website. The editorial board of JFJMU strives hard for the authenticity and accuracy of the material published in the journal. However, findings and statements are views of the authors and do not necessarily represent views of the Editorial Board.
