Simple post today. Just some definitions and explanations of stuff that is pretty familiar to most people.

Payoff / Loss Matrices
Payoff matrices describe the reward for choosing a particular action and loss matrix just describes the loss related to choosing a particular action given a certain state. The most famous payoff matrix I guess is the prisoners dilemma matrix. Two criminals are caught. If they keep quiet they both get say 5 years, but if one of them tells on the other he gets 3 years and the other 10. If both tell on each other then they both get 8 years. To visualize the possible outcomes one can write them in a matrix

(Criminal1, Criminal2)	Keeps quiet	Tells
Keep Quiet	(5,5) Years	(10,3) Years
Tell	(3,10) Years	(8,8) Years

It is easy to see from this matrix that the best strategy on average is to tell on the companion, this is called a decision rule. There are several other ways of choosing a decision rule depending on the wanted outcome. One problem with the payoff matrix is that it doesn’t take into account the possible repercussions that might happen if you snitch on your friend. One other interesting perspective comes when one considers the regret instead of the maximum payoff. All of a sudden the strategy becomes a minimization of the regret of all the outcomes. In the prisoner dilemma that would imply that we would incorporate the dilemma of being labeled a snitch etc. The minimization thus reflects a more conservative strategy which seems more reasonable in the terms of investing decision etc.

In a machine learning setting the payoff matrix is usually extended to a payoff or a loss function that reflects nonlinear effects and that factors other effects as well i.e. sums of non-linear functions. The functions are usually based upon statistics of previously seen data. In Bayesian generative modeling the payoff matrix or function would be the posterior probability and the decision rule would be the maximum or the average of the posterior.

Affinity Matrix / Similarity Matrix
An affinity/similarity matrix is matrix that describes the relation between all objects in a set. The matrix is symmetric and the relation between the objects in the set is described by the affinity function. The affinity function can be any function that takes as input any two objects from the set. Typical functions could be the exponential of the Euclidian distance or the covariance i.e. usually some metric. The affinity can be though of as a complete graph where each matrix entry describe the relation between the nodes in the graph.

Confusion Matrix
From Wikipedia “In the field of artificial intelligence, a confusion matrix is a specific table layout that allows visualization of the performance of an algorithm, typically a supervised learning one (in unsupervised learning it is usually called a matching matrix). Each column of the matrix represents the instances in a predicted class, while each row represents the instances in an actual class. The name stems from the fact that it makes it easy to see if the system is confusing two classes (i.e. commonly mislabeling one as another).”