Vision-Language Traffic Sign Recognition for Self-Driving: Robust Classification, Uncertainty Calibration, and LLM Safety Explanations on a GTSRB-Compatible Benchmark

Abstract

Traffic-sign recognition is a small-object perception task with direct safety relevance for self-driving systems. This paper reports a complete, reproducible evaluation of robust classification, uncertainty calibration, and language-grounded safety explanations for 43 traffic-sign classes derived from the German Traffic Sign Recognition Benchmark (GTSRB) ontology. The official GTSRB archive was treated as the target domain; however, external binary ZIP downloads were unavailable in the execution environment. Following the experimental fallback rule, the study used a packaged GTSRB-compatible synthetic dataset with 39,209 RGB images, the same 43 class names, the official training-class scale, and deterministic train, validation, and test splits of 27,446, 5,881, and 5,882 images. The visual recognizer combines pooled intensity, HOG edge, and color statistics in a lightweight linear probabilistic classifier. A semantic group head predicts six safety-oriented sign groups, and a language-prior fusion layer combines class and group evidence before temperature scaling. The final model achieved 92.469% top-1 accuracy, 97.042% top-5 accuracy, 84.953% macro F1, 0.421 negative log-likelihood, and 1.433% expected calibration error on the held-out test split. Controlled corruption tests under low light, over-exposure, Gaussian noise, blur, occlusion, and motion blur quantified robustness rather than assuming it. The results show that semantic fusion and calibration improved probability quality on clean data, while HOG-dominant features remained competitive under severe illumination shifts. The explanation module produced deterministic LLM-style safety messages grounded in predicted class, confidence, ontology group, and driving action.