Econometrics Fundamentals: A Cheat Sheet

Econometrics Cheat Sheet By Marcelo Moreno - Universidad Rey Juan Carlos The Econometrics Cheat Sheet Project Definitions Econometrics - is a social science discipline with the objective of quantify the relationships between economic agents, test economic theories and evaluate and implement government and business policies. Econometric model - is a simplified representation of the reality to explain economic phenomena. Ceteris paribus - if all the other relevant factors remain constant. Data types Cross section - data taken at a given moment in time, an static photo. Order doesn’t matter. Time series - observation of variables across time. Order does matter. Panel data - consist of a time series for each observation of a cross section. Pooled cross sections - combines cross section from different time periods. 1. Specification. 2. Estimation. Regression analysis 3. Validation. 4. Utilization. Ordinary Least Squares Objective - minimize Pn the2 Sum of Squared Residuals (SSR): min Under this assumptions, the OLS estimator will present i=1 ûi , where ûi = yi − ŷi good properties. Gauss-Markov assumptions: Simple regression model 1. Parameters linearity (and weak dependence in time y Equation: series). y must be a linear function of the β’s. yi = β0 + β1 xi + ui 2. Random sampling. The sample from the population Estimation: has been randomly taken. (Only when cross section) ŷi = β̂0 + β̂1 xi 3. No perfect collinearity. where: β1 ˆ There are no independent variables that are constant: β̂0 = y − β̂1 x Var(xj ) ̸= 0, ∀j = 1, . . . , k Cov(y,x) β̂ 1 = Var(x) ˆ There isn’t an exact linear relation between indepenβ0 x dent variables. 4. Conditional mean zero and correlation zero. Multiple regression model a. There aren’t systematic errors: E(u | x1 , . . . , xk ) = y Equation: E(u) = 0 → strong exogeneity (a implies b). yi = β0 + β1 x1i + · · · + βk xki + ui b. There are no relevant variables left out of the model: Estimation: Cov(xj , u) = 0, ∀j = 1, . . . , k → weak exogeneity. ŷi = β̂0 + β̂1 x1i + · · · + β̂k xki 5. Homoscedasticity. The variability of the residuals is where: the same for all levels of x: β̂0 = y − β̂1 x1 − · · · − β̂k xk Var(u | x1 , . . . , xk ) = σu2 Cov(y,resid x ) x 2 β̂j = Var(resid xj )j β0 6. No autocorrelation. Residuals don’t contain informax1 tion about any other residuals: Matrix: β̂ = (X T X)−1 (X T y) Corr(ut , us | x1 , . . . , xk ) = 0, ∀t ̸= s 7. Normality. Residuals are independent and identically Interpretation of coefficients Model Dependent Independent β1 interpretation distributed: u ∼ N (0, σu2 ) Level-level y x ∆y = β1 ∆x 8. Data size. The number of observations available must Level-log y log(x) ∆y ≈ (β1 /100)(%∆x) be greater than (k + 1) parameters to estimate. (It is Log-level log(y) x %∆y ≈ (100β1 )∆x already satisfied under asymptotic situations) Log-log log(y) log(x) %∆y ≈ β1 (%∆x) Econometric model assumptions Basic concepts Phases of an econometric model Assumptions and properties Study and predict the mean value of a variable (dependent variable, y) regarding the base of fixed values of other vari- Asymptotic properties of OLS ables (independent variables, x’s). In econometrics it is Under the econometric model assumptions and the Central common to use Ordinary Least Squares (OLS) for regres- Limit Theorem (CLT): ˆ Hold 1 to 4a: OLS is unbiased. E(β̂j ) = βj sion analysis. ˆ Hold 1 to 4: OLS is consistent. plim(β̂j ) = βj (to 4b Correlation analysis left out 4a, weak exogeneity, biased but consistent) Correlation analysis don’t distinguish between dependent ˆ Hold 1 to 5: asymptotic normality of OLS (then, 7 is and independent variables. necessarily satisfied): u ∼ N (0, σu2 ) a ˆ Simple correlation measures the grade of linear associaˆ Hold 1 to 6: unbiased estimate of σu2 . E(σ̂u2 ) = σu2 tion between two variables.Pn i −y)) ˆ Hold 1 to 6: OLS is BLUE (Best Linear Unbiased Esti√Pn i=1 ((xi −x)·(y r = Cov(x,y) Pn 2 2 σx ·σy = i=1 (xi −x) · i=1 (yi −y) mator) or efficient. ˆ Partial correlation measures the grade of linear associaˆ Hold 1 to 7: hypothesis testing and confidence intervals tion between two variables controlling a third. can be done reliably. Quadratic y x + x2 ∆y = (β1 + 2β2 x)∆x Error measurements Pn Pn Sum of Sq. Residuals: SSR = i=1 û2i = Pi=1 (yi − ŷi )2 n Explained Sum of Squares: SSE = Pi=1 (ŷi − y)2 n (yi − y)2 Total Sum of Sq.: SST = SSE + SSR = i=1q Standard Error of the Regression: Standard Error of the β̂’s: Root Mean Squared Error: Absolute Mean Error: Mean Percentage Error: SSR σ̂u = n−k−1 p se(β̂) = q σ̂u2 · (X T X)−1 Pn 2 i=1 (yi −ŷi ) RMSE = n AME = P MPE = Pn i=1 |yi −ŷi | n n i=1 |ûi /yi | n · 100 Individual tests R-squared Tests if a parameter is significantly different from a given Is a measure of the goodness of the fit, how the regression value, ϑ. fits the data: ˆ H0 : βj = ϑ SSE R2 = SST = 1 − SSR SST ˆ H1 : βj ̸= ϑ ˆ Measures the percentage of variation of y that is linβ̂ −ϑ Under H0 : t = se(j β̂ ) ∼ tn−k−1,α/2 early explained by the variations of x’s. j ˆ Takes values between 0 (no linear explanation) and 1 (total explanation). When the number of regressors increases, the value of the R-squared also increases, whatever the new variables are relevant or not. To solve this problem, there is an adjusted R-squared by degrees of freedom (or corrected): 2 n−1 n−1 2 R = 1 − n−k−1 · SSR SST = 1 − n−k−1 · (1 − R ) If |t| > |tn−k−1,α/2 |, there is evidence to reject H0 . Individual significance test - tests if a parameter is significantly different from zero. ˆ H0 : βj = 0 ˆ H1 : βj ̸= 0 Dummy variables Dummy (or binary) variables are used for qualitative information like sex, civil state, country, etc. ˆ Takes the value 1 in a given category and 0 in the rest. ˆ Are used to analyze and modeling structural changes in the model parameters. If a qualitative variable have m categories, we only have to include (m − 1) dummy variables. Structural change Structural change refers to changes in the values of the parameters of the econometric model produced by the effect Under H0 : t = ∼ tn−k−1,α/2 of different sub-populations. Structural change can be inIf |t| > |tn−k−1,α/2 |, there is evidence to reject H0 . cluded in the model through dummy variables. 2 For big sample sizes: R ≈ R2 The location of the dummy variables (D) matters: The F test ˆ On the intercept (additive effect) - represents the mean Simultaneously tests multiple (linear) hypothesis about the difference between the values produced by the structural Hypothesis testing parameters. It makes use of a non restricted model and a change. restricted model: Definitions y = β0 + δ 1 D + β 1 x 1 + u ˆ Non restricted model - is the model on which we want Is a rule designed to explain from a sample, if exist evˆ On the slope (multiplicative effect) - represents the efto test the hypothesis. idence or not to reject an hypothesis that is made fect (slope) difference between the values produced by ˆ Restricted model - is the model on which the hypothabout one or more population parameters. the structural change. esis that we want to test have been imposed. Elements of an hypothesis test: y = β0 + β1 x 1 + δ 1 D · x 1 + u Then, looking at the errors, there are: ˆ Null hypothesis (H0 ) - is the hypothesis to be tested. Chow’s structural test - analyze the existence of strucˆ SSRUR - is the SSR of the non restricted model. ˆ Alternative hypothesis (H1 ) - is the hypothesis that tural changes in all the model parameters, it’s a particular ˆ SSRR - is the SSR of the restricted model. cannot be rejected when H0 is rejected. expression of the F test, where H0 : No structural change SSRR −SSRUR n−k−1 Under H0 : F = · q ∼ Fq,n−k−1 SSRUR ˆ Test statistic - is a random variable whose probability (all δ = 0). where k is the number of parameters of the non restricted distribution is known under H0 . model and q is the number of linear hypothesis tested. ˆ Critical value (C) - is the value against which the test Changes of scale If F > Fq,n−k−1 , there is evidence to reject H0 . statistic is compared to determine if H0 is rejected or not. Global significance test - tests if all the parameters as- Changes in the measurement units of the variables: It sets the frontier between the regions of acceptance and sociated to x’s are simultaneously equal to zero. ˆ In the endogenous variable, y ∗ = y ·λ - affects all model rejection of H0 . ˆ H0 : β1 = β2 = · · · = βk = 0 parameters, βj∗ = βj · λ, ∀j = 1, . . . , k ˆ Significance level (α) - is the probability of rejecting ˆ H1 : β1 ̸= 0 and/or β2 ̸= 0 . . . and/or βk ̸= 0 ˆ In an exogenous variable, x∗j = xj · λ - only affect the the null hypothesis being true (Type I Error). Is chosen In this case, we can simplify the formula for the F statistic: parameter linked to said exogenous variable, βj∗ = βj · λ by who conduct the test. Commonly is 10%, 5% or 1%. n−k−1 R2 · ∼ F Under H : F = 2 k,n−k−1 0 ˆ Same scale change on endogenous and exogenous - only 1−R k ˆ p-value - is the highest level of significance by which H0 If F > Fk,n−k−1 , there is evidence to reject H0 . affects the intercept, β0∗ = β0 · λ cannot be rejected. Two-tailed test. H0 dist. One-tailed test. H0 dist. −C C 1−α α/ 2 α/ 2 Accept. region C 1−α α Accept. reg. β̂j se(β̂j ) Confidence intervals Changes of origin The confidence intervals at (1 − α) confidence level can be Changes in the measurement origin of the variables (encalculated: dogenous or exogenous), y ∗ = y + λ - only affects the β̂ ∓ t · se( β̂ ) j j n−k−1,α/2 The rule is: if p-value < α holds, there is evidence to model’s intercept, β0∗ = β0 + λ reject H0 , thus, there is evidence to accept H1 . Heteroscedasticity Autocorrelation ˆ Perfect multicollinearity - there are independent variables that are constant and/or there is an exact linear relation between independent variables. Is the breaking of the third (3) econometric model assumption. ˆ Approximate multicollinearity - there are independent variables that are approximately constant and/or there is an approximately linear relation between independent variables. It does not break any econometric model assumption, but has an effect on OLS. The residuals ui of the population regression function do not have the same variance σu2 : Var(u | x1 , . . . , xk ) = Var(u) ̸= σu2 Is the breaking of the fifth (5) econometric model assumption. The residual of any observation, ut , is correlated with the residual of any other observation. The observations are not independent. Corr(ut , us | x1 , . . . , xk ) = Corr(ut , us ) ̸= 0, ∀t ̸= s The “natural” context of this phenomena is time series. Is the breaking of the sixth (6) econometric model assumption. Consequences ˆ OLS estimators still are unbiased. ˆ OLS estimators still are consistent. ˆ OLS is not efficient anymore, but still a LUE (Linear Consequences Unbiased Estimator). ˆ Perfect multicollinearity - the equation system of ˆ Variance estimations of the estimators are biased: OLS cannot be solved due to infinite solutions. the construction of confidence intervals and the hypothˆ Approximate multicollinearity esis testing is not reliable. – Small sample variations can induce to big variations in Detection the OLS estimations. u y ˆ Graphs - look – The variance of the OLS estimators of the x’s that are for scatter patcollinear, increments, thus the inference of the paramterns on x vs. u eter is affected. The estimation of the parameter is x or x vs. y plots. very imprecise (big confidence interval). x Detection ˆ Correlation analysis - look for high correlations between independent variables, |r| > 0.7. ˆ Variance Inflation Factor (VIF) - indicates the increment of Var(β̂j ) because of the multicollinearity. 1 VIF(β̂j ) = 1−R 2 Consequences ˆ OLS estimators still are unbiased. ˆ OLS estimators still are consistent. ˆ OLS is not efficient anymore, but still a LUE (Linear Unbiased Estimator). ˆ Variance estimations of the estimators are biased: the construction of confidence intervals and the hypothesis testing is not reliable. Detection ˆ Graphs - look for scatter patterns on ut−1 vs. ut or make use of a correlogram. Ac. Ac. + Ac. − ut ut ut ˆ Formal tests - White, Bartlett, Breusch-Pagan, etc. Commonly, H0 : No heteroscedasticity. ut−1 Correction ˆ Use OLS with a variance-covariance matrix estimator robust to heteroscedasticity (HC), for example, the one proj 2 posed by White. where Rj denotes the R-squared from a regression beˆ If the variance structure is known, make use of Weighted tween xj and all the other x’s. Least Squares (WLS) or Generalized Least Squares – Values between 4 to 10 - there might be multicollinear(GLS): ity problems. – Supposing that Var(u) = σu2 ·xi , divide the model vari– Values > 10 - there are multicollinearity problems. ables by the square root of xi and apply OLS. One typical characteristic of multicollinearity is that the – Supposing that Var(u) = σu2 · x2i , divide the model regression coefficients of the model aren’t individually difvariables by xi (the square root of x2i ) and apply OLS. ferent from zero (due to high variances), but jointly they ˆ If the variance structure is not known, make use of Feaare different from zero. sible Weighted Least Squared (FWLS), that estimates a Correction possible variance, divides the model variables by it and ˆ Delete one of the collinear variables. then apply OLS. ˆ Perform factorial analysis (or any other dimension reduc- ˆ Make a new model specification, for example, logarithmic tion technique) on the collinear variables. transformation (lower variance). ˆ Interpret coefficients with multicollinearity jointly. ut−1 ut−1 ˆ Formal tests - Durbin-Watson, Breusch-Godfrey, etc. Commonly, H0 : No autocorrelation. Correction ˆ Use OLS with a variance-covariance matrix estimator robust to heterocedasticity and autocorrelation (HAC), for example, the one proposed by Newey-West. ˆ Use Generalized Least Squares. Supposing yt = β0 + β1 xt + ut , with ut = ρut−1 + εt , where |ρ| < 1 and εt is white noise. – If ρ is known, create a quasi-differentiated model where ut is white noise and estimate it by OLS. – If ρ is not known, estimate it by -for example- the Cochrane-Orcutt method, create a quasi-differentiated model where ut is white noise and estimate it by OLS.