Bayes' Formula

Let $A$ and $B_j$ be Sets. Conditional Probability requires that

$$P(A\cap B_j) = P(A)P(B_j\vert A),$$ (1)

where $\cap$ denotes Intersection (``and''), and also that

$$P(A\cap B_j) = P(B_j\cap A) = P(B_j)P(A\vert B_j)$$ (2)

and

$$P(B_j\cap A) = P(B_j)P(A\vert B_j).$$ (3)

Since (2) and (3) must be equal,

$$P(A\cap B_j) = P(B_j\cap A).$$ (4)

From (2) and (3),

$$P(A\cap B_j) = P(B_j)P(A\vert B_j).$$ (5)

Equating (5) with (2) gives

$$P(A)P(B_j\vert A)=P(B_j)P(A\vert B_j),$$ (6)

so

$$P(B_j\vert A) = {P(B_j)P(A\vert B_j) \over P(A)}.$$ (7)

Now, let

$$S \equiv \bigcup_{i=1}^N A_i,$$ (8)

so $A_i$ is an event in $S$ and $A_i\cap A_j=\emptyset$ for $i\not= j$, then

$$A = A\cap S = A\cap \left({\,\bigcup_{i=1}^N A_i}\right)= \bigcup_{i=1}^N (A\cap A_i)$$ (9)

$$P(A) = P\left({\,\bigcup_{i=1}^N (A\cap A_i)}\right)= \sum_{i=1}^N P(A\cap A_i).$$ (10)

From (5), this becomes

$$P(A) = \sum_{i=1}^N P(A_i)P(A\vert A{_i}),$$ (11)

so

$$P(A_i\vert A) = {P(A_i)P(A\vert A_i) \over \sum\limits_{j=1}^N P(A_j)P(A\vert A_j)}.$$ (12)

See also Conditional Probability, Independent Statistics

References

Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd ed. Cambridge, England: Cambridge University Press, p. 810, 1992.