Probability distribution

A probability distribution is a mathematical approach to quantifying uncertainty.

There are two main classes of probability distributions: Discrete and continuous. Discrete distributions describe variables that take on discrete values only (typically the positive integers), while continuous distributions describe variables that can take on arbitrary values in a continuum (typically the real numbers).

In more advanced studies, one also comes across hybrid distributions.

A gentle introduction to the concept

Faced with a set of mutually exclusive propositions or possible outcomes, people intuitively put "degrees of belief" on the different alternatives.

A simple example

When you wake up in the morning one of three thing may happen that day:

You will get hit by a meteor falling in from space.
You will not get hit by a meteor falling in from space, but you'll be struck by lightning.
Neither will happen.

Most people will usually intuit a small to zero belief in the first alternative (although it is possible, and is known to actually have occurred), a slightly larger belief in the second, and a rather strong belief in the third.

In mathematics, such intuitive ideas are captured, formalized and made precise by the concept of a discrete probability distribution.

A more complicated example

Rather than a simple list of propositions or outcomes like the one above, one may have a to deal with a continuum.

For example, consider the next new person you'll get to know. Given a way to measure height exactly, with infinite precision, how tall will he or she be?

This can be formulated as an uncountably infinite set of propositions, or as a ditto set of possible outcomes of a random experiment.

Let's look at three of these propositions in detail:

...

The person is exactly 1.7222... m tall.

...

The person is exactly 2.333... m tall.

...

The person is exactly 25.010101... m tall.

...

Clearly, we don't believe the person will be over 25 meters tall. But neither do we believe any of the other propositions. Why should any particular proposition turn out to be the exact correct one among an infinity of others?

But we still somehow feel that the first proposition listed is more "likely" than the second, which again is more "likely" than the third.

Also, we feel that some "ranges" are more likely than others, f.i. a height between 1.6 and 1.8 meters feels "likely", a height between 2.2 and 2.4 m seems possible but unlikely, and a height larger than that usually seems safe to exclude.

In mathematics, such intuitive ideas are captured, formalized and made precise by the concept of a continuous probability distribution.

A formal introduction

Discrete probability distributions

Let $S=\{...,s_{0},s_{1},...\}$ . Let f be a function from S to $R$ such that

f(s) ∈ [0,1] for all s ∈ S
The sum $\sum _{i=-Inf}^{i=Inf}f(s_{i})$ exists and evaluates to exactly 1.

Then f is a probability distribution over the set S.

Continuous probability distributions

Let S be an ordered uncountably infinite set.

Let f be a function from S to $R$ such that

f(s) ∈ [0,1] for all s ∈ S
The Riemann integral $\int _{-inf}^{inf}f(s)\,ds$ exists and evaluates to exactly 1.

Then f is a probability distribution over the set S.

In advanced probability theory, one formulates distributions in terms of sigma algebras, measure theory and Lebesgue integrals. Using these tools, one can formulate definitions somewhat differently, allowing for certain rather contrived classes of functions to be dealt with and used as probability distributions.