Statistics Notes

Basic Notes

The Bayesian Notion. [O'Hagan 2000] Consider events, A and B. From the identity $ℙ (A) ℙ (B | A) = ℙ (A, B) = ℙ (B) ℙ (A | B)$ we have the simplest form of Bayes' theorem, $ℙ (B | A) = ℙ (B) ℙ (A | B) / ℙ (A)$ We interpret the Bayes' theorem in the following way. We are interested in the event B, and begin with an initial, prior probability $ℙ (B)$ for its occurrence. We then observe the occurrence of A. The proper description of how likely B is when A is known to have occurred is the posterior probability $ℙ (B | A)$ . Bayes' theorem can be understood as a formula for updating from prior to posterior probability, the updating consisting of multiplying by the ratio $ℙ (A | B) / ℙ (A)$ . It therefore describes how a probability changes as we learn new information.
Extend the binary bayes to General Discrete. [O'Hagan 2000] Let $B_{1}, B_{2}, \dots$ be a discrete partition of the sure event. Then $ℙ (B_{r} | A) = ℙ (B_{r}) \frac{ℙ (A | B_{r})}{\sum_{r} ℙ (A | B_{r}) ℙ (B_{r})}$ If the B_rs are a set of hypotheses of which one and only one is true, then observing event A changes the prior probabiilties $ℙ (B_{r})$ to posterior probabilities $ℙ (B_{r} | A)$ . The occurrence of A increases the proability of B_r if $ℙ (A | B_{r})$ is greater than the average of all the $ℙ (A | B_{r})$ s.
Likelihood. [O'Hagan 2000] The probability $ℙ (A | B_{r})$ is known as the likelihood of B_r given by A. The primitive notion, that hypotheses given greater likelihood by A should somehow have highter probability when A is observed to occur, possess compelling empirical logic.
Application of the Bayesian Notion
- [O'Hagan 2000, modified] Seeing a sequence of increasingly detailed appearance of a thing and to adaptively decide what it is from a pool of candidates, e.g., {tree, man, car, building, etc}.
- Recognizing a malleable object, e.g., a folded T-shirt, from a candidate pool.
- [O'Hagan 2000] Listening to a music piece and, as the music goes, trying to determine the composer from a candidate pool, e.g. {Beethoven, Bach, etc}.