BNP 14 - Plenary Speaker: Li Ma

Описание: Li Ma

Generative models, two-sample problem, and additive tree ensembles

Generative models have become a popular tool for simulating observations from complex,
high-dimensional distributions, and they share a close connection with Bayesian
nonparametrics (BNP). Many popular BNP models correspond to specific generative
processes, although the traditional focus of BNP has been on inferring the underlying
distribution or function and quantifying the associated uncertainty. However, effectively
assessing the quality of generative models remains a challenge. Standard approaches, such
as reporting empirical distance estimates between real and generated samples, generally
do not reveal the nature of the discrepancy. Nor do they allow for reliable uncertainty
quantification. At its core, this is a two-sample problem involving high-dimensional,
nonparametric objects—a setting that has been studied in the BNP literature.
We show that a powerful approach to understanding such discrepancies is to learn the
density ratio between the generative model and the true data-generating model. We
demonstrate several strategies for characterizing the discrepancy once reliable density
ratio estimates and Bayesian uncertainty quantification become available. To this end, we
construct a reliable estimator for the density ratio. Our starting point is a loss function for
learning density ratios under a two-sample design, based on the variational form of the
Hellinger distance between the two sample densities. Drawing a connection between this
loss and the exponential loss commonly used in training additive tree ensembles for
supervised learning, we show that the new loss can be optimized by fitting a sequence of
weak tree learners that iteratively maximize the Hellinger distance between two sets of
pseudo-observations, constructed by reweighting the original observations at each
iteration.
This leads to a class of tree boosting algorithms that yields approximate M-estimators of
the underlying density ratio. We then exploit the resemblance between the Hellinger-based
loss and an exponential family kernel to design conjugate priors on the coefficients of each
weak tree learner, under a loss-based generalized Bayesian framework. This enables us to
construct a BART-like Bayesian backfitting sampler targeting the Gibbs posterior of the
resulting tree ensemble. In this way, we achieve generalized Bayesian uncertainty
quantification for the density ratio between the two samples. We illustrate applications of
this approach in the context of generative models for microbiome compositional data.

Keywords: Generalized Bayes, trees, ensemble models, additive models, generative
models

Co-authors: Naoki Awaya, Waseda University, [email protected] Yuliang Xu, Duke
University, [email protected]