Analyzing Student Interactions with an AI Tutor in Undergraduate Chemistry

Computer based tutoring in the form of Intelligent Tutoring Systems (ITS) have been in development since the 1970s, but the implementation of generative AI into these systems is far more recent with large publishers only releasing their systems a few years ago. As this technology is new, little is known about how students are using genAI tutors. Using the established literature of ITS and human tutoring in science education as a foundation, this work is focused on exploring how students use these new tools with the goal of helping inform instructors of how students use the tools provided to them, but to also situate this new technology in the broader study of tutoring in chemistry education.  

Our work is currently focused on characterizing the types of messages between students and the AI tutor to categorize the prevalent interaction types of students. We developed a codebook that describes the function of student messages and a codebook that describes the facilitation move put forth from the AI. Through the Office of Teaching, Learning, and Technology, PIPER, a machine learning algorithm, is being developed to code student messages at a scale unrealistic for human abilities. This will allow for a more complete picture of student work and make identifying common student interaction patterns easier.  

ongoing work, currently in progress by Rhiannon Davids