Metropolis-Hastings Algorithms
CPSC 535 Metropolis-Hastings
AD
March 2007
AD ()
March 2007
1 / 45
Gibbs Sampler
Initialization: Select deterministically or randomly (0 ) (0 ) θ = θ 1 , ..., θ p .
AD ()
March 2007
2 / 45
Gibbs Sampler
Initialization: Select deterministically or randomly (0 ) (0 ) θ = θ 1 , ..., θ p . Iteration i; i 1:
AD ()
March 2007
2 / 45
Gibbs Sampler
Initialization: Select deterministically or randomly (0 ) (0 ) θ = θ 1 , ..., θ p . Iteration i; i 1:
For k = 1 : p
AD ()
March 2007
2 / 45
Gibbs Sampler
Initialization: Select deterministically or randomly (0 ) (0 ) θ = θ 1 , ..., θ p . Iteration i; i 1:
(i ) k where (i ) (i ) (i 1 ) (i 1 ) θ 1 , ..., θ k 1 , θ k +1 , ..., θ p (i )
For k = 1 : p
Sample θ k θ
(i ) k
π θk j θ
=
.
AD ()
March 2007
2 / 45
The Gibbs sampler requires sampling from the full conditional distributions π ( θk j θ k ) .
AD ()
March 2007
3 / 45
The Gibbs sampler requires sampling from the full conditional distributions π ( θk j θ k ) . For many complex models, it is impossible to sample from several of these “full” conditional distributions.
AD ()
March 2007
3 / 45
The Gibbs sampler requires sampling from the full conditional distributions π ( θk j θ k ) . For many complex models, it is impossible to sample from several of these “full” conditional distributions. Even if it is possible to implement the Gibbs sampler, the algorithm might be very ine¢ cient because the variables are very correlated or sampling from the full conditionals is extremely expensive/ine¢ cient.
AD ()
March 2007
3 / 45
Metropolis-Hastings Algorithm
The Metropolis-Hastings algorithm is an alternative algorithm to sample from probability distribution π (θ ) known up to a normalizing constant.
AD ()
March 2007
4 / 45
Metropolis-Hastings Algorithm
The Metropolis-Hastings algorithm is an alternative algorithm to sample from probability
AD
March 2007
AD ()
March 2007
1 / 45
Gibbs Sampler
Initialization: Select deterministically or randomly (0 ) (0 ) θ = θ 1 , ..., θ p .
AD ()
March 2007
2 / 45
Gibbs Sampler
Initialization: Select deterministically or randomly (0 ) (0 ) θ = θ 1 , ..., θ p . Iteration i; i 1:
AD ()
March 2007
2 / 45
Gibbs Sampler
Initialization: Select deterministically or randomly (0 ) (0 ) θ = θ 1 , ..., θ p . Iteration i; i 1:
For k = 1 : p
AD ()
March 2007
2 / 45
Gibbs Sampler
Initialization: Select deterministically or randomly (0 ) (0 ) θ = θ 1 , ..., θ p . Iteration i; i 1:
(i ) k where (i ) (i ) (i 1 ) (i 1 ) θ 1 , ..., θ k 1 , θ k +1 , ..., θ p (i )
For k = 1 : p
Sample θ k θ
(i ) k
π θk j θ
=
.
AD ()
March 2007
2 / 45
The Gibbs sampler requires sampling from the full conditional distributions π ( θk j θ k ) .
AD ()
March 2007
3 / 45
The Gibbs sampler requires sampling from the full conditional distributions π ( θk j θ k ) . For many complex models, it is impossible to sample from several of these “full” conditional distributions.
AD ()
March 2007
3 / 45
The Gibbs sampler requires sampling from the full conditional distributions π ( θk j θ k ) . For many complex models, it is impossible to sample from several of these “full” conditional distributions. Even if it is possible to implement the Gibbs sampler, the algorithm might be very ine¢ cient because the variables are very correlated or sampling from the full conditionals is extremely expensive/ine¢ cient.
AD ()
March 2007
3 / 45
Metropolis-Hastings Algorithm
The Metropolis-Hastings algorithm is an alternative algorithm to sample from probability distribution π (θ ) known up to a normalizing constant.
AD ()
March 2007
4 / 45
Metropolis-Hastings Algorithm
The Metropolis-Hastings algorithm is an alternative algorithm to sample from probability