ChartGPT-3.5's Performance on Introduction to Medical Laboratory Sciences Test: AI Triumphs in Theory but Stumbles in Diagrams

Abstract

The inclusion of Artificial Intelligence (AI) in health education has seen a sizable uptake among universities and medical students, with a growing focus on enhancing AI tools to facilitate learning and decision-making. This study evaluates the performance of AI on a formal test introducing medical science concepts. A 90-mark test, written by 60 students, was sent to the ChatGPT3.5 window for answering. AI completed 5 questions, obtaining 72.2% (65/90) on the theory part, leaving 3 diagram questions (27.8% [25/90]) with reasons. ChatGPT3.5 achieved perfect scores on theory questions; however, it struggled to interpret questions with diagrams. Thus, this study explores the implications of AI in medical education and the areas that require further development. A more advanced ChatGPT could have resulted in a better performance if it were upgraded to analyse the diagram. In essence, while AI contributes valuable insights and efficiency in certain domains of teaching and learning, it should be viewed as a complementary tool rather than a replacement for the irreplaceable human element. The relationship between AI and human expertise holds the potential to enhance teaching and learning in healthcare settings, combining the strengths of both for more comprehensive and reliable patient care, which is necessary for the development of students' competencies. ChatGPT-4.5 generally has higher cost implications compared to ChatGPT-4 and ChatGPT-3.5 due to its advanced capabilities and improved performance. In some countries and institutions with limited financial resources, it's important to leverage free access to ChatGPT for educational purposes, while being mindful that advanced versions may incur costs. Integrating ChatGPT as a teaching tool may require financial investment to access higher-tier services and ensure effective implementation.