Metropolis-Hastings (MH) algorithm
The Metropolis-Hastings (MH) algorithm is a Markov Chain Monte Carlo (MCMC) method used for sampling from a probability distribution, typically when direct sampling is difficult. It's particularly useful in Bayesian statistics, statistical physics, and many other fields. Here's an explanation of the algorithm: 1. **Problem Setting**: The MH algorithm is employed when you want to generate a sequence of random samples from a target probability distribution, which could be complex and multi-dimensional. Let's say you have a distribution defined by a function, usually denoted as a probability density function (PDF), and you want to sample from it. 2. **Initialization**: You start with an initial sample or state, often denoted as "x." 3. **Proposal Distribution**: You choose a proposal distribution (also known as a "jumping" distribution or proposal kernel), which defines how you suggest new candidate states from the current state. This distribution is typ...