Lightweight Hallucination Firewall for Enterprise LLM Applications: Evidence Consistency, Self-Checking, and Small-Model Detection on TruthfulQA

Abstract

Enterprise deployments of large language models require a guardrail that rejects unsupported or misleading answers before the answer reaches a user. This paper presents a lightweight hallucination firewall that treats a question and a candidate answer as a binary decision problem: accept a truthful candidate or block an untruthful candidate. The evaluation was conducted on the complete TruthfulQA CSV benchmark, which contains 817 questions across 38 categories. The official correct-answer and incorrect-answer fields were expanded into 5884 labeled answer candidates, and questions were split into group-disjoint train, development, and test partitions. Seven local detectors were trained: a majority prior, a self-check numeric logistic regression, word TF-IDF logistic regression, character TF-IDF logistic regression, word-character TF-IDF stochastic-gradient logistic regression, word-character TF-IDF stochastic-gradient SVM, and a hybrid firewall that combines TF-IDF evidence with deterministic self-check features. The best detector was Word TF-IDF LR, reaching AUROC 0.790, AUPRC 0.775, F1 0.688, precision 0.560, and recall 0.893 on the held-out test set. All detectors run offline with zero API cost, and the slowest measured detector processed 1,000 candidates in 172.0 ms. The results show that a small auditable classifier can serve as a practical first-line firewall for enterprise LLM applications, while category-level and error analyses identify the cases that still require retrieval-grounded verification or human review.