Evaluating Explainability, Premise Rejection, and Confidence Calibration in LLMs for Science Q&A

Abstract

Large language models (LLMs) are increasingly integrated into educational environments, scientific information retrieval, and decision-support systems due to their ability to generate human-like responses and explanations. Despite their growing popularity, concerns remain regarding the reliability and trustworthiness of their outputs. LLMs can produce responses that appear coherent, persuasive, and scientifically grounded while containing factual inaccuracies, incomplete reasoning, unsupported claims, or hallucinated information. Moreover, these errors are often accompanied by high levels of expressed confidence, potentially increasing the risk of user overreliance and misinformation. As a result, evaluating trust in LLM-generated explanations requires going beyond simple measures of answer correctness. This study investigates the trustworthiness of LLM responses under a range of challenging science question-answering (Q&A) conditions. Specifically, we examine four dimensions of performance: (1) factual accuracy of answers, (2) quality and correctness of explanations, (3) ability to identify and reject misleading or false premises embedded within questions, and (4) confidence calibration, defined as the degree to which expressed confidence aligns with actual response reliability. Four widely accessible LLM platforms—ChatGPT, Grok, Gemini, and DeepSeek—were evaluated using a common set of prompts, standardized response constraints, and archived raw outputs to ensure comparability across systems. To approximate realistic classroom and educational-support scenarios, all models were instructed to provide concise responses of approximately 300 characters. Data collection was conducted between December 7, 2025, and January 10, 2026, and therefore represents a time-specific snapshot of model behavior rather than a permanent ranking of systems.