A Semiparametric Approach to Discrete Choice Demand Estimation: Theory and Empirical Evidence
Abstract
Discrete choice models are fundamental to demand estimation and counterfactual policy evaluation. Existing approaches face a core tradeoff. Parametric specifications preserve the economic interpretability needed for policy analysis, but impose functional-form assumptions that may fail to capture the rich substitution patterns of real-world choice. Flexible nonparametric and machine-learning approaches relax these restrictions, but standard plug-in estimators generally do not deliver valid $\sqrt{n}$ inference on policy-relevant demand objects. In this paper, we propose SPDML, a semiparametric framework that combines structure and flexibility. The framework retains a low-dimensional parametric component for the policy lever of interest, such as price, commission, or promotion, preserving its economic interpretation, while a flexible, choice-set-aware component learns the remaining utility nonparametrically from own attributes, rival attributes, and observable choice context. Crucially, the researcher need not commit ex ante to a low-dimensional parametric specification of baseline utility, substitution patterns, observed preference heterogeneity, or behavioral mechanisms. Instead, the framework lets the data determine the shape of baseline utility while preserving valid inference for policy-relevant demand effects. We estimate the flexible component with a choice-probability-preserving deep neural network and apply Neyman-orthogonal debiasing to obtain $\sqrt{n}$-consistent, asymptotically normal estimators of own and cross demand responses, elasticities, and counterfactual policy effects. We then characterize the framework’s scope: it exactly encompasses a broad class of logit data-generating processes, including linear and nonlinear utility, complementarity, and observed or constructible behavioral mechanisms such as decoy effects, choice overload, and reference-dependent valuation. Through a mixed-logit bridge, we also provide bounded approximation results for all random-utility models. Monte Carlo experiments show that the estimator recovers policy effects accurately in settings where standard parametric models fail. In an application to salesperson commissions in Chinese retail pharmacies, the proposed estimator aligns closely with synthetic difference-in-differences benchmarks and outperforms standard parametric demand models.
