This post will try to explain why in the optimization of the Rayleigh Quotient one constrains the norm of $$x$$ to be …
Category: Math
On the Trick for Computing the Squared Euclidean Distances Between Two Sets of Vectors
Many times one wants to compute the squared pairwise Euclidean distances between two sets of observations. As always it is enlightening to …
Understanding the Metropolis Hasting algorithm – A tutorial
The Problem Say you want to evaluate the expectation of a function over a random variable $$E[f(x)] = \int p(x)f(x)dx$$, or perhaps …
A short tutorial on Kernel Density Estimation (KDE)
The aim of Kernel Density Estimation(KDE) is: Given a set of $$N$$ samples from a random variable, $$\mathbf{X}$$, possibly multivariate and continuous, …
Partially fixing numerical underflow for mixture models
In a mixture model the probability of an event $$x$$ is written $$P(x=X) =\sum_{i}\pi_{i}P_{i}(x=X) $$, where $$\pi_{i}$$ is the probability of the …
The Trace Trick for Gaussian Log Likelihood
Maybe you have seen something like this when observing the log likelihood derivations for multivariate Gaussians $$ \ln p(X|\mu, \Sigma) = \frac{1}{2}\ln|\Sigma|- …
Calculating the indices for n choose k
So if you want to calculate n choose k, or n over k or all k different combinations of n unique items …