$\DeclareMathOperator{\p}{Pr}$ $\DeclareMathOperator{\P}{Pr}$ $\DeclareMathOperator{\c}{^C}$ $\DeclareMathOperator{\or}{ or}$ $\DeclareMathOperator{\and}{ and}$ $\DeclareMathOperator{\var}{Var}$ $\DeclareMathOperator{\Var}{Var}$ $\DeclareMathOperator{\E}{E}$ $\DeclareMathOperator{\std}{Std}$ $\DeclareMathOperator{\Ber}{Bern}$ $\DeclareMathOperator{\Bin}{Bin}$ $\DeclareMathOperator{\Poi}{Poi}$ $\DeclareMathOperator{\Uni}{Uni}$ $\DeclareMathOperator{\Geo}{Geo}$ $\DeclareMathOperator{\NegBin}{NegBin}$ $\DeclareMathOperator{\Beta}{Beta}$ $\DeclareMathOperator{\Exp}{Exp}$ $\DeclareMathOperator{\N}{N}$ $\DeclareMathOperator{\R}{\mathbb{R}}$ $\DeclareMathOperator{\argmax}{argmax}$ $\newcommand{\d}{\, d}$

Conditional Distributions


Before we looked at conditional probabilities for events. Here we formally go over conditional probabilities for random variables. The equations for both the discrete and continuous case are intuitive extensions of our understanding of conditional probability:

Discrete

The conditional probability mass function (PMF) for the discrete case: \begin{align*} \P(X=x|Y=y)=\frac{P(X=x,Y=y)}{P(Y=y)} \end{align*} In the presence of multiple random variables, it becomes increasingly useful to use shorthand! The above definition is identical to this notation: \begin{align*} \P(x|y)=\frac{P(x,y)}{P(y)} \end{align*} Again where a lowercase symbol such as $x$ is short hand for the event $X=x$. The conditional definition works for any event and as such we can also write conditionals using cumulative density functions (CDFs) for the discrete case: \begin{align*} \P(X \leq a | Y=y) &= \frac{\P(X \leq a, Y=y)}{\p(Y=y)} \\ &= \frac{\sum_{x\leq a} \P(X=x,Y=y)}{\P(Y=y)} \end{align*} This last term can be rewritten, by a clever manipulation. We can make the sum extend over the whole fraction: \begin{align*} \P(X \leq a | Y=y) &= \frac{\sum_{x\leq a} \P(X=x,Y=y)}{\P(Y=y)} \\ &= \sum_{x\leq a} \frac{\P(X=x,Y=y)}{\P(Y=y)} \\ &= \sum_{x\leq a} \P(X=x|Y=y) \end{align*}

Continuous

The conditional probability density function (PDF) for the continuous case: \begin{align*} f(x|y) = \frac{f(x,y)}{f(y)} \end{align*} The conditional cumulative density function (CDF) for the continuous case: \begin{align*} P(X \leq a | Y=y) = \int_{-\infty}^{a} f(X=x|Y=y)\d x \end{align*}

Mixing Discrete and Continuous

These equations are straightforward once you have your head around the notation for probability density functions, $f(X=x)$, and probability mass functions, $\P(X=x)$. Let $X$ be continuous random variable and let $N$ be a discrete random variable. The conditional probabilities of $X$ given $N$ and $N$ given $X$ respectively are: \begin{align*} f(X=x|N=n) = \frac{\P(N=n|X=x)f(X=x)}{\p(N=n)} && \end{align*} \begin{align*} \p(N=n|X=x) = \frac{f(X=x|N=n)\p(N=n)}{f(X=x)} \end{align*}