Enhancing Chatbot-Assisted Study Program Orientation

Abstract

As university dropout rates increase, implementing innovative solutions is crucial to reduce attrition. Aligning students’ interests with their study programs enhances academic success, satisfaction, and retention. This paper presents a novel approach using open-source Large Language Models (LLM) and Retrieval-Augmented Generation (RAG) to develop a semi-open-domain knowledge chatbot. The chatbot generates informed responses and recommendations to diverse student queries by retrieving relevant data while maintaining ethical standards and avoiding biased responses. When testing five model combinations on 70 prompts partially from real study advisors, results demonstrate that the RAG approach with the Mixtral LLM and RoBERTa embedding model offers superior performance. Our method for handling critical user prompts further indicates a significantly improved response quality. These findings advance service-oriented chatbots in education, aiming to reduce student attrition through accurate and helpful program recommendations.