Reputation-Aware Truth Discovery for Robust Evidence Fusion in RAG Pipelines
Date:
Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by grounding responses in external evidence, but the reliability of retrieved information often varies across heterogeneous sources, leading to conflicting evidence and reduced trustworthiness. This challenge is particularly acute in information-sparse settings, where robust decision-making depends on principled mechanisms for aggregating uncertain and potentially unreliable inputs. This paper presents Skald, a reputation-aware truth discovery method for robust evidence fusion in RAG pipelines. Skald combines a truth discovery module for heterogeneous data with a reputation mechanism that preserves and updates source reliability across executions, enabling dependable sources to exert greater influence over time while penalizing unreliable ones. Unlike conventional truth discovery approaches that reinitialize source reliability at each run, Skald leverages historical source behavior to improve adaptability in dynamic multi-source environments. Evaluations on real-world datasets show that Skald consistently outperforms established methods such as TruthFinder and CRH across multiple similarity metrics. These results indicate that integrating reputation-aware truth discovery into RAG-oriented evidence fusion provides a principled foundation for improving robustness, trustworthiness, and decision quality in intelligent systems operating under uncertainty.