Bar 01 · Foundations

Probability distributions: an overview

A distribution is the complete probabilistic account of a random variable. This page separates the discrete families from the continuous ones, explains the mass function, the density and the cumulative function, and surveys the classical families that recur throughout the subject.

Discrete and continuous cases

When a random variable takes values on a countable set, its law is given by a mass function that assigns a probability to each single value. The probabilities are non-negative and sum to one. Examples include the uniform law on a finite set, the binomial law for a count of successes, and the Poisson law for a count of rare events in a fixed region.

When a random variable takes values across an interval, single points usually carry no probability at all. The law is then described by a density, a non-negative function whose integral over an interval gives the probability that the variable falls inside that interval. The uniform law on an interval, the exponential law, the gamma family and the normal law all belong here. A density may exceed one at some points, because it is a rate of measure rather than a probability itself.

The cumulative function as a common language

Whether a law is discrete or continuous, its cumulative distribution function is defined in the same way: the probability that the variable does not exceed a level. This function is non-decreasing and right-continuous, tends to zero at the far left and to one at the far right, and fixes the law completely. In the continuous case it is the integral of the density; in the discrete case it is a rising staircase whose steps are the masses.

Several useful properties follow at once. The probability that a variable lies strictly between two levels is the difference of the cumulative values at those levels. Quantiles are read from the cumulative function by inverting it. The median is the level at which the cumulative value reaches one half.

Common families

A small number of families carries most of the theory. The binomial law counts successes in a fixed number of independent trials of equal probability. The Poisson law describes counts in a region when events occur independently at a constant rate. The exponential law gives waiting times between such events, and the gamma family generalises it to a sum of several waits. The uniform law spreads measure evenly across a range, and the normal law gathers measure about a central value with tails that thin extremely quickly.

Transformations of a variable

A new variable formed by a smooth increasing map of an old one inherits a density that can be derived from the old density by the change-of-variables rule, in which the derivative of the map supplies a scale factor. For monotone maps the cumulative function transforms by composition, which is often the quicker route. These rules explain, for instance, why taking logarithms turns a strictly positive variable into one whose support is the whole line.

Mixing and conditioning

Distributions may be combined. A mixture places a weight on each member of a family and averages their densities; the compound Poisson family and the negative binomial family arise in this way. Conditioning slices a joint law at a fixed value of one coordinate and rescales, producing a conditional law whose mean may depend on the value held fixed. That dependence is precisely what the covariance and correlation pages go on to quantify.

With these foundations in hand, the reader is ready for the normal law, whose density is the natural next object of study and the gateway to the multivariate setting.