Optimizing Automated Conversational Large Language Models for Higher Educational Institution

Abstract

Higher education institutions need to improve their query systems in order to improve communication, response times, and information access. This study investigates a novel method of combining large language models (LLMs) with knowledge bases created especially for higher education institutions in order to meet this critical requirement. We adhered with all higher education institution regulations and gathered data in both structured and unstructured formats from Sheth L. U. J. College of Arts and Sir M. V. College of Science and Commerce in Mumbai, India. Our system, employing the Llama-Index framework for text embedding and the two primary large language models, Google Gemini 1.0 and OpenAI GPT-3.5, for response generation, achieved an impressive average response time of 5 seconds for both models. Additionally, it attained an average relevancy score of 0.96 for Google Gemini 1.0 and 0.93 for OpenAI GPT-3.5 across diverse query categories. These findings clearly show how LLM-powered systems can improve communication and offer incredibly helpful, relevant information in higher education settings. We strongly encourage the deployment of these systems in order to greatly improve communication and the user experience in the context of higher education. In order to further enhance the efficiency and coverage of these systems, we plan to expand our knowledge sources and host them on cloud platforms, making them easily accessible.