The geometric distribution

Suppose we have a sequence of Bernoulli trials, each with a probability p of success and a probability q = 1 - p of failure. How many trials occur before we obtain a success? Let the random variable X be the number of trials needed to obtain a success. Then X has values in the range {1, 2,...}, and for k ≥ 1,

since we have k - 1 failures before the one success. A probability distribution satisfying equation (C.30) is said to be a geometric distribution. Figure C.1 illustrates such a distribution.

Assuming that q < 1, the expectation of a geometric distribution can be calculated using identity (A.8):

Thus, on average, it takes 1/ p trials before we obtain a success, an intuitive result. The variance, which can be calculated similarly, but using Exercise A.1-3, is

As an example, suppose we repeatedly roll two dice until we obtain either a seven or an eleven. Of the 36 possible outcomes, 6 yield a seven and 2 yield an eleven. Thus, the probability of success is p = 8/36 = 2/9, and we must roll 1/ p = 9/2 = 4.5 times on average to obtain a seven or eleven.

The binomial distribution

How many successes occur during n Bernoulli trials, where a success occurs with probability p and a failure with probability q = 1 - p? Define the random variable X to be the number of successes in n trials. Then X has values in the range {0,1,..., n}, and for k = 0,..., n,

since there are ways to pick which k of the n trials are successes, and the probability that each occurs is p^kq^n-k. A probability distribution satisfying equation (C.33) is said to be a binomial distribution. For convenience, we define the family of binomial distributions using the notation

Figure C.2 illustrates a binomial distribution. The name "binomial" comes from the fact that (C.33) is the kth term of the expansion of (p + q)ⁿ. Consequently, since p + q = 1,

as is required by axiom 2 of the probability axioms.

We can compute the expectation of a random variable having a binomial distribution from equations (C.8) and (C.35). Let X be a random variable that follows the binomial distribution b(k; n, p), and let q = 1 - p. By the definition of expectation, we have

By using the linearity of expectation, we can obtain the same result with substantially less algebra. Let X_i be the random variable describing the number of successes in the ith trial. Then E[X_i] = p · 1 + q · 0 = p, and by linearity of expectation (equation (C.20)), the expected number of successes for n trials is

The same approach can be used to calculate the variance of the distribution. Using equation (C.26), we have . Since X_i only takes on the values 0 and 1, we have , and hence

To compute the variance of X, we take advantage of the independence of the n trials; thus, by equation (C.28),

As can be seen from Figure C.2, the binomial distribution b(k; n, p) increases as k runs from 0 to n until it reaches the mean np, and then it decreases. We can prove that the distribution always behaves in this manner by looking at the ratio of successive terms:

This ratio is greater than 1 precisely when (n + 1) p - k is positive. Consequently, b(k; n, p) > b(k - 1; n, p) for k < (n + 1) p (the distribution increases), and b(k; n, p) < b(k - 1; n, p) for k > (n + 1) p (the distribution decreases). If k = (n + 1) p is an integer, then b(k; n, p) = b(k - 1; n, p), so the distribution has two maxima: at k = (n + 1) p and at k - 1 = (n + 1) p - 1 = np - q. Otherwise, it attains a maximum at the unique integer k that lies in the range np - q < k < (n + 1) p.

The following lemma provides an upper bound on the binomial distribution.