Empowering Educators with AI: Utilizing Small Language Models for Classifying Educational Content and Understanding Curricula in Restorative Dentistry and Endodontics

Abstract

Context: Advances in artificial intelligence (AI) have increasingly influenced medical education through large language models (LLMs) like ChatGPT. These models promise to help students comprehend and navigate educational content. However, their effectiveness in specialized fields like dentistry needs thorough evaluation. This project aims to assess the accuracy and applicability of LLMs in dental education by fine-tuning them with specialized dental content.

Methods: The project includes the following steps:

- Dataset Constitution and Annotation: Expand and enhance a dental dataset, including MCQs, lectures, glossaries, and reference chapters. The    content will be chunked into small pieces and annotated with competencies, knowledge levels, and topics. The chunks will be embedded using   CamemBERT-bio, and similarity searches will be performed using the Facebook AI Similarity Search (Faiss) library.

- Development of the LLM: Develop a Retrieval-Augmented Generation (RAG) approach or fine-tune low and medium LLMs for the dental field.

- Performance Evaluation: Assess the LLMs' accuracy in answering MCQs from the Collège National des Enseignants en Odontologie Conservatrice (CNEOC) and classifying content based on European guidelines. Compare global LLMs like GPT-4 with smaller models such as CamemBERT and Mistral fine-tuned with dental educational content. Evaluate classification accuracy using metrics like F1 score, precision, and    trueness.

- Data Sharing and Communication: Share datasets and code on GitHub and disseminate findings through scientific publications and conferences.

Expected Results: We hypothesize that smaller LLMs fine-tuned on dental educational content will have better classification performance than global models like ChatGPT-4. This approach will help educators train and develop their models using their materials, maintaining data privacy and intellectual property.

Perspectives: This project aims to aid students in exam preparation and help educators in curriculum design. Sharing an educational database and open-source algorithms will foster collaboration and innovation in dental curricula.

 

Short CV

2022-2023  Lyon Faculty of Medicine - Diploma in Pedagogy
2018-2021  Laboratory of Mechanics and Structures LaMCoS, (INSA/CNRS) – PhD (Biomechanics)
2018           Toulouse Dental Faculty – Certificate in Endodontic Microsurgery
2013-2021  Lyon Dental Faculty – Doctor of Dental Surgery & Post Graduate Diploma in Oral Medicine (3yrs full time)
2007-2013  Engineering school Ecole Nationale Supérieure des Mines de Saint-Etienne
2007           Bachelor S with highest honour (Le Mans).

Practice & Teaching
Since 2022 Assistant Professor at the Dpt Restorative Dentistry and Endodontics (Lyon Dental Hospital) - Supervision of dental students (initial and post graduate diplomas) – Lectures for Certificate in Endodontics
Since 2021 Part time Private Practice in Endodontics - Belleville sur Saone (France)
2013           Eindhoven University of Technology (Netherlands): 6 months Internship in the Dpt of Soft Tissue Biomechanics & Tissue Engineering (Cees Oomens)