Summing Densities of Order Statistics The density function for the k th order statistic out of n i.i.d. random variables is given by f(k) (x) = n! fX (x)(F (x))k−1 (1 − F (x))n−k . (k − 1)!(n − k)! where F is the distribution function of X. This expression is achieved by considering the small n! arrangements interval [x−∆, x+∆] in which the k th order statistic falls. Then there are (k−1)!1!(n−k)! of the n values such that (k − 1) values less than x − ∆, and (n − k) values above x + ∆. Afterwards we multiply by the respective probabilities as expressed with the CDF. One might observe that the expression above closely resembles the distribution of a binomial random variable. Perhaps if we add all the density functions f(1) through f(n) (x), the expectation of a binomial random variable will rear its head? S(x) = = n X k=1 n X k=1 f(k) (x) n! fX (x)(F (x))k−1 (1 − F (x))n−k (k − 1)!(n − k)! = fX (x) F (x)S(x) F (x)S(x) n X k=1 n X n! (F (x))k−1 (1 − F (x))n−k (k − 1)!(n − k)! n! (F (x))k (1 − F (x))n−k (k − 1)!(n − k)! k=1 µ ¶ n X n = fX (x) k· (F (x))k (1 − F (x))n−k k = fX (x) k=1 F (x)S(x) = fX (x) · E(binomial(n, F (x))) = fX (x) · nF (x) S(x) = n · fX (x). Thus the sum of the order-statistic density functions is simply the density of the underlying distribution, scaled by n. In retrospect, this makes much intuitive sense. So for example, if the underlying distribution was uniform over [0, 1], S(x) = n for all x ∈ [0, 1]; if the distribution was an exponential with parameter 1, S(x) = 5 · e−x u(x). This is an interesting way of decomposing a probability distribution as a convex combination of order statistics distributions. On the following pages are plots that illustrate this decomposition for various distributions. 2−order statistics decomposition of exp(1) distribution 2 1.8 1.6 probability density 1.4 1.2 1 0.8 0.6 0.4 0.2 0 −1 0 1 2 x−values 3 4 5 4 5 5−order statistics decomposition of exp(1) distribution 5 4.5 4 probability density 3.5 3 2.5 2 1.5 1 0.5 0 −1 0 1 2 x−values 3 Figure 1: Order statistics decompositions of an exponential distribution with λ = 1. Top: Decomposition using only the max and min. Bottom: Decomposition with ranks 1 through 5. 10−order statistics decomposition of standard normal distribution 4 3.5 probability density 3 2.5 2 1.5 1 0.5 0 −5 −4 −3 −2 −1 0 x−values 1 2 3 4 5 Figure 2: Order statistics decomposition of standard normal distribution N (0, 1). 10−order statistics decomposition of binomial(10,3/4) 3 2.5 probability density 2 1.5 1 0.5 0 0 1 2 3 4 5 6 x−values (discrete) 7 8 9 10 Figure 3: Order statistics decomposition of a binomial distribution with n = 10, p = 3/4.