Skip to main content

Infer.NET user guide : Running inference : Working with different inference algorithms

Gibbs sampling

Gibbs sampling is a special case of the Metropolis-Hastings algorithm. From a given state of the variables it randomly moves to another state, preserving the desired equilibrium distribution. The simplest version is single-site Gibbs sampling, where each variable is visited in turn and sampled from its conditional distribution given the current state of its neighbors. Variables may be visited in any order and some variables may be sampled more than others, as long as each variable gets sampled sufficiently often.

A more efficient version of Gibbs sampling, especially in the case of deterministic constraints, is the block Gibbs sampler. Here the variables are grouped into acyclic blocks, each block is visited in turn, and the entire block is sampled from its conditional distribution given the state of its neighbors. This procedure is valid even if blocks overlap; variables in multiple blocks will simply be sampled more often. Typically you will not have to specify the blocking for Gibbs sampling as default blocking is automatically based on the deterministic factors and constraints in your graphical model. However, you do have the option of explicitly specifying blocks using the Group method on the inference engine.

The Gibbs sampling algorithm object has two configurable properties:

Gibbs sampling typically requires many more iterations than EP or VMP (each iteration is one sample).

// Use Gibbs sampling
GibbsSampling gs = new GibbsSampling();  
gs.BurnIn = 100;  
gs.Thin = 10;  
InferenceEngine ie = new InferenceEngine(gs);  
ie.NumberOfIterations = 2000;

When choosing an inference algorithm, you may wish to consider the following properties of Gibbs sampling: