Bayesian Background


In this chapter, we review the fundamental concepts of the Bayesian approach to statistical inference. Bayesian statistics was first introduced over than 250 years ago, but became only popular when it could address practical problems. For a long time Fisher’s theory based on the likelihood function as the fundamental engine of inference and the frequentist approach of Neyman and Pearson have ruled the statistical world. Until three decades ago, the Bayesian approach was looked upon as more of a curiosity rather than providing a tool for solving practical problems. This changed when Markov chain Monte Carlo techniques were introduced. The chapter starts with reviewing the concepts of the classical approach, also called the frequentist approach. Central to the Bayesian approach, is Bayes theorem. The origin of the theorem is a simple factorization of the joint probability into the product of a conditional and a marginal probability. The ingenious idea of Thomas Bayes is to apply this principle to the parameters of a statistical model and to assume that the uncertainty underlying their ‘true’ value can be described using a probability model. We illustrate how the posterior distribution arises and can be computed from prior and data information. The characteristics of the posterior distribution are illustrated for binary and Gaussian responses. In addition, the most common posterior summary measures are discussed. Independent and dependent sampling, including Markov chain Monte Carlo techniques, to approximate the posterior distribution and posterior summary measures are discussed and illustrated. A brief and incomplete review of Bayesian software is then given. Most Bayesian analyzes are based on parametric assumptions. Especially in the last decade, nonparametric Bayesian developments have seen the light but the theoretical level prevents us to go deep here. Bayesian tools for model selection and model checking are also reviewed. Additional topics are treated in the final section as well as suggestions for further reading.

In Lesaffre E, Baio G and Boulanger B (eds), Bayesian Methods for Pharmaceutical Research
Gianluca Baio
Gianluca Baio
Professor of Statistics and Health Economics