Decision-makers are often faced with a wealth of data, some of which are “gold standard” (the desired outcome) while the majority are “proxies” (closely related outcomes that still provide relevant information). For example, e-commerce platforms often use abundant customer click data (proxy) to make product recommendations rather than sparse customer purchase data (true outcome of interest); clinical trial regulators sometimes use quickly-observed surrogate outcomes like progression-free survival (proxy) to make drug approval decisions instead of overall survival (true outcome of interest). I will discuss two papers addressing the challenge of efficiently combining true and proxy data to improve decision-making.

The first paper considers personalized recommendations in high dimension, with a large amount of proxy data and a small amount of true data. We propose a novel two-step estimator that provably achieves the same accuracy as common heuristics used by data scientists with exponentially less true data. We demonstrate its effectiveness on e-commerce and healthcare datasets. The second paper considers clinical trials where the true outcome of interest takes a long time to measure relative to surrogate outcomes. We propose a new adaptive clinical trial design that leverages both surrogate and true outcomes to inform trial decisions. We illustrate our proposed design on a large-scale clinical trial database for metastatic breast cancer, estimating a 5% increase in trial benefits relative to existing clinical trial designs.

Based on joint work with Arielle Anderer (Wharton) and John Silberholz (Michigan Ross).