Understanding the Metropolis Hasting algorithm - A tutorial

The Problem Say you want to evaluate the expectation of a function over a random variable \(E[f(x)] = \int p(x)f(x)dx\), or perhaps find the max of a probability distribution, typically the posterior, \(\arg \max p(x|y)\), where calculating the derivate is intractable since it depends on some feature that comes from some algorithm or some unknown […]

A short tutorial on Kernel Density Estimation (KDE)

The aim of Kernel Density Estimation(KDE) is: Given a set of \(N\) samples from a random variable, \(\mathbf{X}\), possibly multivariate and continuous, estimate the random variables probability density function(pdf) The univariate case To get a rough idea how one can think about the problem, we start out with a set of samples, \(X=[x_1,x_2,...,x_N]\), of a […]