by Yubo 'Paul' Yang

# Gibbs Sampling

The basic Gibbs sampler samples a joint probability distribution one variable at a time. Each random variable is sampled from its full conditional probability distribution with all other variables fixed. Independent variables can be sampled simultaneously, making the Gibbs sampler ideal for the restricted Boltzmann machine.

## Presentation Summary

In this presentation, I present:

- Basic idea behind the Gibbs sampling algorithm.
- Basic application to sample the bivariate normal distribution.
- Application to change-point detection
- Application to restricted Boltzmann machine

## Examples

- example 1: sample bivariate normal distribution
- example 2: optimize change-point detection model
- example 3: train Bernoulli restricted Boltzmann machine on NMIST data

## References

- MCMC: The Gibbs Sampler, The Clever Machine
- Beyesian Inference: Metropolis-Hasting Sampling, Ilker Yildirim
- Beyesian Inference: Gibbs Sampling, Ilker Yildirim
- A Practical Guide to Training Restricted Boltzmann Machines, Geoffrey E. Hinton
- Introduction to Restricted Boltzmann Machines, Edwin Chen
- deeplearning.net

### All optimization.

Yubo "Paul" Yang ALGORITHM

machine learning