The theory of Statistics refers primarily to a branch of mathematics that specializes in enumeration, or counted, data and their relation to measured data. It may also refer to a fact of classification, which is the chief source of all statistics, and has a relationship to psychometric applications in the social sciences.
An individual statistic refers to a derived numerical value, such as a mean, a coefficient of correlation, or some other single concept of descriptive statistics . It may also refer to an idea associated with an average, such as a median, or standard deviation, or some value computed from a set of data. 
More precisely, in mathematical statistics, and in general usage, a statistic is defined as any measurable function of a data sample . A data sample is described by instances of a random variable of interest, such as a height, weight, polling results, test performance, etc., obtained by random sampling of a population.
Suppose one wishes to embark on a quantitative study of the height of adult males in some country C. How should one go about doing this and how can the data be summarized? In statistics, the approach taken is to assume/model the quantity of interest, i.e., "height of adult men from the country C" as a random variable X, say, taking on values in [0,5] (measured in metres) and distributed according to some unknown probability distribution F on [0,5] . One important theme studied in statistics is to develop theoretically sound methods (firmly grounded in probability theory) to learn something about the postulated random variable X and also its distribution F by collecting samples, for this particular example, of the height of a number of men randomly drawn from the adult male population of C.
Suppose that N men labeled have been randomly drawn by simple random sampling (this means that each man in the population is equally likely to be selected in the sampling process) whose heights are , respectively. An important yet subtle point to note here is that, due to random sampling, the data sample obtained is actually an instance or realization of a sequence of independent random variables with each random variable being distributed identically according to the distribution of (that is, each has the distribution F). Such a sequence is referred to in statistics as independent and identically distributed (i.i.d) random variables. To further clarify this point, suppose that there are two other investigators, Tim and Allen, who are also interested in the same quantitative study and they in turn also randomly sample N adult males from the population of C. Let Tim's height data sample be and Allen's be , then both samples are also realizations of the i.i.d sequence , just as the first sample was.
- . This statistic is known as the sample mean
- . This statistic is known as the sample variance. Often the alternative definition of sample variance is preferred because it is an unbiased estimator of the variance of X, while the former is a biased estimator.
Statisticians may transform data by taking the logarithm, square root, reciprocal, or other function if the data does not fit a normal distribution. Data needs to be transformed back to its original form in order to present confidence intervals.
Measurements of central tendency
Measurements of variation
- Standard deviation (SD) is a measure of variation or scatter. The standard deviation does not change with sample size.
- Variance is the square of the standard deviation:
- Standard error of the mean (SEM) measures the how accurately you know the mean of a population and is always smaller than the SD. The SEM becomes smaller as the sample size increases. The sample standard devision (S) and SEM are related by:
Inferential statistics and hypothesis testing
- Discriminant analysis
- Factor analysis
- Cluster analysis
- Propensity score
- Recursive partitioning
Problems in reporting of statistics
In medicine, common problems in the reporting and usage of statistics have been inventoried. These problems tend to exaggerated treatment differences.
- Trapp, Robert; Beth Dawson (2004). Basic & clinical biostatistics. New York: Lange Medical Books/McGraw-Hill. LCC QH323.5 .D38LCCN 2005-263. ISBN 0-07-141017-1.
- Mosteller, Frederick; Bailar, John Christian (1992). Medical uses of statistics. Boston, Mass: NEJM Books. ISBN 0-910133-36-0. Google Books
- Guilford, J.P., Fruchter, B. (1978). Fundamental statistics in psychology and education. New York: McGraw-Hill.
- Shao, J. (2003). Mathematical Statistics (2 ed.). ser. Springer Texts in Statistics, New York: Springer-Verlag, p. 100.
- This is the case in non-parametric statistics. On the other hand, in parametric statistics the underlying distribution is assumed to be of some particular type, say a normal or exponential distribution, but with unknown parameters that are to be estimated.
- Bland JM, Altman DG (March 1996). "Transforming data". BMJ 312 (7033): 770. PMID 8605469. PMC 2350481.
- Bland JM, Altman DG (May 1996). "The use of transformation when comparing two means". BMJ 312 (7039): 1153. PMID 8620137. PMC 2350653.
- Bland JM, Altman DG (April 1996). "Transformations, means, and confidence intervals". BMJ 312 (7038): 1079. PMID 8616417. PMC 2350916.
- What is the difference between "standard deviation" and "standard error of the mean"? Which should I show in tables and graphs?. Retrieved on 2008-09-18.
- Pocock SJ, Hughes MD, Lee RJ (August 1987). "Statistical problems in the reporting of clinical trials. A survey of three medical journals". N. Engl. J. Med. 317 (7): 426–32. PMID 3614286.