# Log Probabilities

A log probability $\log \p(E)$ is simply the log function applied to a probability. For example if $\p(E) = 0.00001$ then $\log \p(E) = \log(0.00001) \approx -11.51$. Note that in this book, the default base is the natural base $e$. There are many reasons why log probabilities are an essential tool for digital probability: (a) computers can be rather limited when representing very small numbers and (b) logs have the wonderful ability to turn multiplication into addition, and computers are much faster at addition.

You may have noticed that the log in the above example produced a negative number. Recall that $\log b = c$, with the implied natural base $e$ is the same as the statement $e ^ c = b$. It says that $c$ is the exponent of $e$ that produces $b$. If $b$ is a number between 0 and 1, what power should you raise $e$ to in order to produce $b$? If you raise $e^0$ it produces 1. To produce a number less than 1, you must raise $e$ to a power less than 0. That is a long way of saying: if you take the log of a probability, the result will be a negative number. \begin{align} 0 &\leq \p(E) \leq 1 && \text{Axiom 1 of probability} \\ -\infty &\leq \log \p(E) \leq 0 && \text{Rule for log probabilities} \end{align}

The product of probabilities $\p(E)$ and $\p(F)$ becomes addition in logarithmic space: $$\log (\p(E) \cdot \p(F) ) = \log \p(E) + \log \p(F)$$
This is especially convenient because computers are much more efficient when adding than when multiplying. It can also make derivations easier to write. This is especially true when you need to multiply many probabilities together: $$\log \prod_i \p(E_i) = \sum_i \log \p(E_i)$$