Lecture 12 - Variable Relationships

LECTURE 12 Variable relationships When we look at more than one variable at the same time, it's natural to explore if there is any association between them. * When two variables show some form of connection with each other, we define independence or dependence. * When two variables show no form of connection with each other, they show independence. Example: Titanic disaster [Two-way tables showing the relationship between survival and class] Distributions independence If the conditional distribution of Y given X are equal to each other, then they are also equal to the marginal distribution of Y. Y is independent in distribution (f) from X if Vi ∈ Y, P(Y = yi | X = xi) = P(Y = yi). Y is dependent in distribution (f) from X if Vi ∈ Y, P(Y = yi | X = xi) ≠ P(Y = yi). Example [A two-way table] We define the concept of independence/dependence in a two-way table: two variables are independent if conditional distributions are equal. We now want a tool to measure the dependence between the two variables given the two-way table that describes them jointly. The strategy is to define a table "of reference", that corresponds to the independence case, and measuring the "distance" of the observed table from the "reference" one. [A diagram illustrating the concept of independence and dependence in a two-way table] [Two-way tables showing the relationship between survival and class] The index X^2 The most commonly used index for distribution dependence is based on the comparison between expected and observed frequencies. This is Pearson's X^2. X^2 = Σ(Σ((nij - nij)^2 / nij)) Always greater than or equal to zero It grows while observed frequencies diverge from expected frequencies. The maximum is reached when the double distribution assumes a structure in particular, that is the one of a table of complete dependence: each modality of the variable X corresponds to the one and only modality of the variable Y. X^2max = N * min(s-1, t-1) X̃^2 = X^2 / (N * min(s-1, t-1)) 0 <= X̃^2 <= 1 Example X^2 = Σ(Σ((nij - nij)^2 / nij)) = (40-34)^2/34 + (10-16)^2/16 + (15-21)^2/21 + (15-9)^2/9 = 1.05 + 2.25 + 1 + 4 = 9.01 X̃^2 = X^2 / (N * min(s-1, t-1)) = 9.01 / (80 * min(3-1, 2-1)) = 0.11 It is closer to zero than 1, so we can say that the variables are independent. Mean independence and dependence If one of the variables is quantitative, let Y, we can look at the dependence on a second one, X, which can be qualitative, in terms of position indices. A variable, quantitative, Y is independent from another variable X, qualitative or quantitative, if the means of the distributions Y conditionally to the different modalities of X are all equal to the same value. In general, the application E(Y|X=x_i) is called regression function of Y on X. So, we can say we have mean independence if and only if the function of regression is constant. Relation between mean independence and distribution independence The concepts of mean independence from X and the one of distribution independence from X are not the same. If Y is independent in distribution from X then Y is independent in terms of mean from X, but it is not true the other way around. Mean independence → Distribution independence Distribution independence → Mean independence It is easy to build tables where it exists mean independence but not independence in distribution. [A two-way table] Correlation Another way in which the relation between two variables can be viewed. It can be calculated only for quantitative variables. It is related to whether and to what extent the two variables tend to "grow together", that is, if high values of one appear more frequently associated with high values of the other. The scatterplot is the representation of the pairs (couples) (x_1, y_1), (x_2, y_2), ..., (x_N, y_N), that is the double disaggregated distribution of the double variable (X, Y). Between X and Y there is positive correlation when they tend to grow together. Between X and Y there is negative correlation when as one grows the other tends to decrease. [A scatterplot]