LLM-Assisted Incrementality (Uplift) Modeling for Email Advertising: From Feature Interactions to Interpretable Audience–Creative–Channel Policies

Abstract

Incrementality measurement asks a counterfactual question: how much does an advertisement change customer behavior relative to a no-exposure baseline. Uplift modeling operationalizes this question by estimating heterogeneous treatment effects and converting them into targeting policies. This paper studies LLM-assisted uplift modeling on the Hillstrom Email Marketing randomized controlled trial (RCT) with 64,000 customers and three arms (No E-Mail, Mens E-Mail, Womens E-Mail). We compare classical two-model logistic regression (LR), two-model XGBoost, an R-learner causal-forest surrogate (random-forest pseudo-outcome regression), and a Tabular Transformer (S-learner) built from self-attention layers. To bridge accuracy and decision interpretability, we implement an LLM-inspired feature interaction generator that proposes cross-features over recency, customer value segments, ZIP class, and historical shopping channels, and then distill the resulting uplift scores into human-readable rules of the form "audience conditions × creative × channel". We evaluate models using inverse-propensity replay (IPS) to construct Qini (uplift) curves and compute AUUC/Qini coefficients. We also measure incremental profit using observed spend minus a fixed email cost of $0.01. On the held-out test set, the Tabular Transformer achieves the best conversion AUUC (0.005203), while XGBoost with LLM interactions yields the highest full-population profit uplift ($1.086 per customer) and produces concise targeting rules that match observed treatment-control differences within rule-defined subpopulations. These results show that interaction generation and rule distillation can convert uplift models into actionable and auditable advertising strategies.