Citizendium - a community developing a quality comprehensive compendium of knowledge, online and free. Click here to join and contribute—free
CZ thanks AUGUST 2014 donors; special to Darren Duncan. SEPTEMBER 2014 donations open; need minimum total $100. Let's exceed that. Donate here. Treasurer's Financial Report -- Thanks to August content contributors. --




Covariance

From Citizendium, the Citizens' Compendium

Jump to: navigation, search
This article has a Citable Version.
Main Article
Talk
Related Articles  [?]
Bibliography  [?]
External Links  [?]
Citable Version  [?]
 
This editable Main Article has an approved citable version (see its Citable Version subpage). While we have done conscientious work, we cannot guarantee that this Main Article, or its citable version, is wholly free of mistakes. By helping to improve this editable Main Article, you will help the process of generating a new, improved citable version.

Contents

The covariance — usually denoted as Cov — is a statistical parameter used to compare two real random variables on the same sample space (more precisely, the same probability space).
It is defined as the expectation (or mean value) of the product of the deviations (from their respective mean values) of the two variables.

The sign of the covariance indicates a linear trend between the two variables.

  • If one variable increases (in the mean) with the other, then the covariance is positive.
  • It is negative if one variable tends to decrease when the other increases.
  • If it is 0 then there is no linear correlation between the two variables.
    In particular, this is the case for stochastically independent variables. But the inverse is not true because there may still be other – nonlinear – dependencies.

The value of the covariance is scale-dependent and therefore does not show how strong the correlation is. For this purpose a normed version of the covariance is used — the correlation coefficient which is independent of scale.

Formal definition

The covariance of two real random variables X and Y with expectation (mean value)

 \mathrm E(X) = \mu_X \quad\text{and}\quad \mathrm E(Y) = \mu_Y

is defined by

 \operatorname{Cov} (X,Y) := \mathrm E( (X-\mu_X) (Y-\mu_Y) )
                                 = \mathrm E(XY) - \mathrm E(X)\mathrm E(Y)

Remark:
If the two random variables are the same then their covariance is equal to the variance of the single variable: Cov(X,X) = Var(X).

In a more general context of probability theory the covariance is a second-order central moment of the two-dimensional random variable (X,Y), often denoted as μ11.

Finite data

For a finite set of data

 (x_i,y_i) \in \R^2 \ \text{with}\ i=1,\dots,n

the covariance is given by

 {1\over n} \sum_{i=1}^n ( x_i - \overline{x} ) ( y_i - \overline{y} )
         \qquad \text{where}\ \overline{x} := {1\over n} \sum_{i=1}^n x_i
         \ \text{and}\ \overline{y} := {1\over n} \sum_{i=1}^n y_i

or, using a convenient notation

 [a_i] := \sum_{i=1}^n  a_i

introduced by Gauss, by

 {1\over n}( [ x_i y_i ] - [x_i][y_i] )

This is equivalent to taking the uniform distribution where each item (xi,yi) has probability 1/n.

Unbiased estimate

The expectation of the covariance of a random sample — taken from a probability distribution — depends on the size n of the sample and is slightly smaller than the covariance of the distribution.

An unbiased estimate of the covariance is

  \mathrm{Cov} (X,Y) = {n \over n-1} \mathrm{Cov}(x_i,y_i)
   = {1\over n-1} \sum_{i=1}^n ( x_i - \overline{x} ) ( y_i - \overline{y} )

Remark:
The distinction between the covariance of a sample and the estimated covariance of the distribution is not always clearly made. This explains why one finds both formulae for the covariance — that taking the mean with " 1 / n " and that with " 1 / (n-1) " instead.

Properties

The covariance is

  • (1) symmetric
  • (2) bilinear
  • (3) positive definite

because the following holds:

 \text{(1)}\ \qquad \operatorname{Cov} (X,Y) = \operatorname{Cov} (Y,X)
 \text{(2a)} \qquad \operatorname{Cov} (aX_1+bX_2,Y) =
       a \cdot \operatorname{Cov} (X_1,Y) + b \cdot \operatorname{Cov} (X_2,Y)
 \text{(2b)} \qquad \operatorname{Cov} (X,aY_1+bY_2) =
       a \cdot \operatorname{Cov} (X,Y_1) + b \cdot \operatorname{Cov} (X,Y_2)
 \text{(3)}\ \qquad 
         \operatorname{Cov} (X,X) \ge 0 \qquad \text{and} \qquad 
         \operatorname{Cov} (X,X) = 0 \Leftrightarrow X = \mu_X \ \text{almost surely}

Since the covariance cannot distinguish between random variables X1 and X2 that have the same deviation, (i.e., X1 − E(X1) = X2 − E(X2) holds almost surely) it does not define an inner product for random variables, but only for random variables with mean 0 or, equivalently, for the deviations.

Views
Personal tools