Financial RAG Answer Quality Regression with Table-Text Evidence Features on RAGBench FinQA and eManual

Abstract

Financial retrieval-augmented generation (RAG) systems must answer questions that combine narrative disclosures, tabular values, percentages, fiscal periods, and arithmetic relations. This study formulates answer evaluation as continuous quality regression and binary adherence classification on the RAGBench FinQA subset, with eManual used as a compact out-of-domain comparison. The experiments use all 16,562 FinQA rows and all 1,318 eManual rows in the official train, validation, and test splits. A composite quality target combines adherence, relevance, utilization, and completeness, while the predictors integrate lexical text, automatic RAG evaluator outputs, evidence-use counts, and table-text evidence features that measure numeric support, currency and percentage consistency, table cues, key coverage, and response-evidence mismatch. On the FinQA test split, the hybrid model reduces quality RMSE from 0.196 for the RAG-metric model to 0.174 and raises adherence AUROC from 0.742 to 0.819. On eManual, the same protocol achieves 0.160 RMSE and 0.842 AUROC. Ablation, calibration, error-slice, cross-domain, significance, and runtime analyses show that numeric consistency and evidence utilization are the most influential signals in finance, whereas evidence-key coverage remains useful across domains. The findings support a lightweight and auditable approach to monitoring RAG answer quality when financial evidence is distributed across tables and surrounding text.