DRO-BAS: Decision Making under the Exponential Family DRO with Bayesian Ambiguity Sets

Conference: ICML 2025 Spotlight
arXiv: 2411.16829
Code: None
Area: Other
Keywords: distributionally robust optimization, exponential family, Bayesian ambiguity sets, decision making, KL divergence

TL;DR

Proposes the DRO-BAS framework, which leverages Bayesian posterior beliefs to construct two posterior-informed ambiguity sets (BASPP and BASPE). Under exponential family conjugate models, these can be reformulated as efficient single-stage stochastic programs, Pareto-dominating existing Bayesian DRO methods on the Newsvendor and Portfolio problems.

Background & Motivation

Background

Background: Distributionally robust optimization (DRO) makes robust decisions by optimizing the objective function under the worst-case distribution, representing a crucial approach to handling unknown data-generating processes. Standard techniques define ambiguity sets using KL divergence or Wasserstein distance, finding the worst-case distribution within the set to make decisions. Bayesian inference captures parameter uncertainty through the posterior distribution, but directly minimizing the Bayesian risk (BRO) lacks protection against model misspecification.

Limitations of Prior Work

Limitations of Prior Work: (1) The ambiguity set of standard KL-DRO contains KL neighborhoods over the entire distribution space, which may include physically unreasonable distributions and lead to over-conservatism; (2) Existing Bayesian DRO (BDRO, Shapiro et al. 2023) employs an "expected worst-case" approach, but its objective function does not correspond to a single worst-case distribution, lacking interpretability; (3) The dual problem of BDRO is a two-stage stochastic program, requiring nested expectation sampling over both the posterior and likelihood, leading to long solving times for SAA due to large sample requirements.

Key Challenge: Designing ambiguity sets requires balancing robustness and conservatism—too large is too conservative (classical KL-DRO), and too small lacks robustness (naive BRO). Meanwhile, computational efficiency is also critical (single-stage vs. two-stage stochastic programming).

Goal: Design posterior-informed ambiguity sets such that DRO (1) corresponds to true worst-case risk minimization, (2) can be efficiently solved under the exponential family framework, and (3) Pareto-dominates existing methods.

Key Insight: Leverage the Bayesian posterior and exponential family conjugacy to construct two classes of ambiguity sets—one based on the KL ball of the posterior predictive distribution, and the other based on the posterior expected KL divergence.

Core Idea: Posterior-informed Bayesian Ambiguity Sets (BAS) combine Bayesian parameter uncertainty with the worst-case protection of DRO, and their dual problems can be reformulated into efficient single-stage stochastic programs under exponential family models.

Method

Overall Architecture

Let the posterior distribution of the parameter \(\theta\) for the data-generating process be \(\Pi(\theta|\mathcal{D})\), and the model family be \(\{P_\theta\}\). DRO-BAS constructs posterior-informed ambiguity sets, searches for the worst-case distribution within them, and minimizes the worst-case risk. The framework offers two ambiguity set formulations (BASPP and BASPE), both of which can be reduced to single-stage optimization problems.

Key Designs

1. BASPP (Bayesian Ambiguity Set based on Posterior Predictive):

   • Function: Elevates Bayesian Risk Optimization (BRO) to DRO with worst-case protection.
   • Mechanism: The posterior predictive distribution \(p_n(\xi|\mathcal{D}) = \int p(\xi|\theta) d\Pi(\theta|\mathcal{D})\) serves as the center of the ambiguity set, constructing a KL ball \(\mathcal{B}_\epsilon(P_n) = \{Q : d_{KL}(Q, P_n) \le \epsilon\}\), and minimizing \(\min_x \sup_{Q \in \mathcal{B}_\epsilon} E_Q[f_x(\xi)]\). The dual formulation is a single-stage stochastic program: \(\inf_{\gamma \ge 0} \gamma\epsilon + \gamma \ln E_{p_n}[e^{f_x(\xi)/\gamma}]\).
   • Design Motivation: When \(n \to \infty\), the posterior collapses to the true parameter, and BASPP degenerates to KL-DRO centered at the true distribution. It is computationally much more efficient than the two-stage problem of BDRO.

2. BASPE (Bayesian Ambiguity Set based on Posterior Expectation):

   • Function: Resolves the issue where BASPP has an infinite moment-generating function under certain conjugate models.
   • Mechanism: The ambiguity set constraint is modified to expected KL: \(\mathcal{A}_\epsilon(\Pi) = \{Q : E_{\theta \sim \Pi}[d_{KL}(Q, P_\theta)] \le \epsilon\}\). For exponential family conjugate models, the expected KL can be computed analytically, simplifying to a KL ball centered at the posterior expected sufficient statistic in the natural parameter space—\(\mathcal{B}_\epsilon(P_{\bar{\eta}_n})\), where \(\bar{\eta}_n = E_\Pi[\eta(\theta)]\).
   • Design Motivation: Exploits the conjugate property of the exponential family (the posterior remains in the same family) to achieve analytical simplification, bypassing BASPP's requirement for a finite moment-generating function, and allowing exact solutions for linear objectives with Gaussian likelihoods.

3. Dual Problem and Efficient Solving:

   • Function: Transforms the minimax DRO problem into an efficiently solvable convex optimization problem.
   • Mechanism: Both BASPP and BASPE satisfy strong duality, and their dual problems are convex single-stage stochastic programs. BASPE has a closed-form solution under linear objective functions and Gaussian likelihoods. For non-linear convex objectives, SAA approximation is used, requiring sampling from only a single distribution (unlike BDRO, which requires double sampling from both the posterior and likelihood).
   • Design Motivation: Single-stage vs. two-stage BDRO—with the same SAA sample size, DRO-BAS achieves higher approximation quality and faster solving speed.

Theoretical Guarantees

Provides a finite-sample calibration method for the optimal tolerance \(\epsilon\): selecting \(\epsilon\) based on a \(\chi^2\) confidence set so that the true parameter lies within the ambiguity set with probability \(1-\alpha\). As \(\epsilon \to 0\), it degenerates to BRO (no robustness), and as \(\epsilon \to \infty\), it degenerates to minimax (extreme conservatism).

Key Experimental Results

Main Results: Newsvendor Problem

Method	Out-of-sample Mean	Out-of-sample Variance	Pareto Optimal
BRO (Bayesian Risk)	Lowest cost	High variance	✗ (No robustness)
BDRO (Shapiro 2023)	Moderate cost	Moderate variance	✗
DRO-BAS_PP (Ours)	Lower cost	Low variance	✓
DRO-BAS_PE (Ours)	Lower cost	Low variance	✓
KL-DRO (Non-Bayesian)	High cost	Lowest variance	✗ (Overly conservative)

DRO-BAS dominates BDRO on the mean-variance Pareto frontier, with a particularly pronounced advantage when the SAA sample size is small.

Portfolio Problem Efficiency Comparison

Method	Solving Time	Out-of-sample Robustness	SAA Sample Complexity
BDRO	Slower	Baseline	Double expectation (posterior × likelihood)
DRO-BAS_PP	Faster	Comparable	Single expectation (posterior predictive)
DRO-BAS_PE	Faster	Comparable	Single expectation (posterior expected parameter)

Ablation Study: Impact of Tolerance \(\epsilon\)

\(\epsilon\) Value	Effect	Equivalent Method
\(\epsilon = 0\)	No robustness	BRO (Naive Bayesian Risk)
Moderate \(\epsilon\)	Optimal mean-variance	DRO-BAS (Recommended)
\(\epsilon \to \infty\)	Most conservative	Minimax (Pure worst-case)

Key Findings

• DRO-BAS exhibits its greatest advantage over BDRO when the SAA sample size is small—the efficiency gap between single-stage and two-stage formulations is amplified under sample limitations.
• BASPE offers analytical advantages in exponential family conjugate models, while BASPP possesses broader applicability.
• The advantage is maximized when the prior knowledge is correct (i.e., data indeed conforms to the exponential family).

Highlights & Insights

• Posterior-informed ambiguity sets elegantly combine DRO and Bayesian inference: They lower conservatism by leveraging structured information from the prior/posterior while maintaining worst-case protection.
• Clever exploitation of the exponential family conjugate property: BASPE leverages the property that the posterior remains in the same family, simplifying complex function space optimization to convex optimization in a low-dimensional parameter space.
• Practical value of single-stage duality: Reducing from two-stage to single-stage not only accelerates computation but also mitigates SAA approximation errors.

Limitations & Future Work

• Limitations of the exponential family assumption: Real-world data may not belong to any exponential family—extensions to mixture distributions, heavy-tailed distributions, etc., are needed.
• Finite moment-generating function requirement for BASPP: Certain conjugate models (such as the Student-t posterior predictive corresponding to Normal-Inverse-Gamma) do not satisfy this condition.
• Analysis of parameter estimation error propagation: Further theoretical guarantees are desired regarding how the approximation error of the posterior \(\Pi\) itself impacts the quality of the DRO solution.
• Non-parametric Bayesian extensions: Combining with non-parametric priors such as the Dirichlet Process is a natural direction for future work.

Related Work & Insights

• vs. BDRO (Shapiro et al., 2023): BDRO employs an expected worst-case approach but does not correspond to a single worst-case distribution and requires a two-stage solution. DRO-BAS offers a genuine worst-case risk interpretation and single-stage efficiency.
• vs. KL-DRO (Hu & Hong, 2013): Classical KL-DRO sets a KL ball over the entire distribution space without utilizing parameter family prior information, making it overly conservative.
• vs. Wasserstein-DRO (Kuhn et al., 2019): As a metric space method, it is computationally expensive in high dimensions and conservative; DRO-BAS operates more efficiently in the parameter space.
• vs. Non-parametric Bayesian DRO (Wang et al., 2023): Dirichlet Process priors do not easily incorporate parametric family structural information.

Rating

• Novelty: ⭐⭐⭐⭐ Solid theoretical contributions with the posterior-informed ambiguity sets and exponential family dual simplification.
• Experimental Thoroughness: ⭐⭐⭐ Validated on two classic problems: Newsvendor and Portfolio.
• Writing Quality: ⭐⭐⭐⭐ Clear theoretical derivations and a comprehensive framework.
• Value: ⭐⭐⭐⭐ Makes significant contributions to the directions of structured DRO and Bayesian decision making.

Related Papers

• [ICLR 2026] Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets
• [ICML 2025] Fixing the Loose Brake: Exponential-Tailed Stopping Time in Best Arm Identification
• [ICML 2025] Improving Generalization with Flat Hilbert Bayesian Inference
• [ICML 2025] Latent Variable Estimation in Bayesian Black-Litterman Models
• [ICML 2025] IBDR: Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness