Evaluating Factual Grounding Strategies in Large Language Models

Abstract

[Abstract] Large Language Models (LLMs) often generate non-factual, or “hallucinated,” content, limiting their reliability in knowledge-intensive tasks. This challenge is particularly critical in multi-hop question answering (MHQA), where models must integrate and reason over multiple pieces of evidence. In this paper, we present an empirical study of prompting strategies aimed at improving the factual grounding of LLMs. Using the Llama-3.1 (8B) model on the HotpotQA benchmark, we evaluate five prompting techniques along three main design points: shot count (zero-shot vs. few-shot), context integration (with vs. without supporting documents), and output constraints (free-form vs. structured responses requiring evidence). We assess both answer accuracy and the precision of supporting fact identification, allowing us to analyze correctness from evidential grounding. Our results reveal different trade-offs: while few-shot prompting improves reasoning consistency, the gains diminish without high-quality supporting context. Similarly, structured outputs reduce variance and improve factual alignment, but their benefits depend critically on how evidence is presented and constrained. Compared to prior studies that focus primarily on accuracy, our analysis highlights the importance of balancing answer quality with verifiable evidence. These findings provide actionable guidance for the design of prompts in multi-hop QA and inform broader efforts to mitigate hallucinations in LLMs across retrieval-augmented and reasoning-intensive applications.