University:
California State University, Northridge
Course:
PSY 524/L | Multivariate Analysis Computer and Lab
Academic year:

2024
Views:

96

Pages:

25
Author:

Dustin Green

Multiple Regression The Basics Multiple Regression (MR)  Predicting one DV from a set of predictors, the DV should be interval/ratio or at least assumed I/R if using Likert scale for instance Assumptions   Y must be normally distributed (no skewness or outliers) X’s    do not need to be normally distributed, but if they are it makes for a stronger interpretation linear relationship w/ Y no multivariate outliers among Xs predicting Y MV Outliers     Leverage – distance of score from distribution Discrepancy – misalignment of score and the rest of the data Influence – change in regression equation with and without value Mahalanobis distance and χ2 Homoskedasticity  Homoskedasticity – variance on Y is the same at all values of X Multicollinearity/Singularity  Tolerance = 1 – SMC (Squared Multiple Correlation)   Low tolerance values <.01 indicate problems Condition Index and Variance   Condition Index > 30 Plus variance on two separate variables > .50. Can you predict Y given the set of Xs?     Anova summary table – significance test R2 (multiple correlation squared) – variation in Y accounted for by the set of predictors Adjusted R2 – sample variation around R2 can only lead to inflation of the value. The adjustment takes into account the size of the sample and number of predictors to adjust the value to be a better estimate of the population value. R2 is similar to η2 value but will be a little smaller because R2 only looks at linear relationship while η2 will account for non-linear relationships. Is each X contributing to the prediction of Y?  Test if each regression coefficient is significantly different than zero given the variables standard error.  T-test for each regression coefficient, Which X is contributing the most to the prediction of Y?  Cannot interpret relative size of Bs because each are relative to the variables scale but Betas (standardized Bs) can be interpreted. y ' =+ a b1 x1 + b2 x2 + b3 x3 Zy ' =beta1 ( Zx1 ) + beta2 ( Zx2 ) + beta3 ( Zx3 )  a being the grand mean on y a is zero when y is standardized Can you predict future scores?   Can the regression be generalized to other data Can be done by randomly separating a data set into two halves,  Estimate regression equation with one half  Apply it to the other half and see if it predicts Sample size for MR    There is no exact number to consider Need enough subjects for a “stable” correlation matrix T and F recommend 50 + 8m, where m is the number of predictors    If there is a lot of “noise” in the data you may need more than that If little noise you can get by with less If interested generalizing prediction than you need at least double the recommended subjects. Regular old linear regression   We know y and we know the predictor values, the unknown is the weights so we need to solve for it This is where matrix algebra comes in Regression and Matrix Algebra   The above can also be conceptualized as: But what’s missing? Regression and Matrix Algebra  We can bring the intercept in by simply altering the design matrix with a column of 1s: Y 4 x1 (X * B)+ E 4x2 2 x1 4 x1 Column of 1s???    In regression, if you regress an outcome onto a constant of 1s you get back the mean of that variable. This only works in the matrix approach, if you try the “by hand” approach to regressing an outcome onto a constant you’ll get zero Computer programs like SPSS, Excel, etc. use the matrix approach Example Case A B C D E F G H I Y 46 40 32 42 61 46 32 51 55 X 90 78 57 65 80 75 45 91 67 CONST 1 1 1 1 1 1 1 1 1 Mean SD 45 9.785 72 15.091 1 0  Here is some example data taken from some of Kline’s slides Example - normal SUMMARY OUTPUT Regression Statistics Multiple R 0.634852293 R Square 0.403037433 Adjusted R Square 0.317757067 Standard Error 8.082373467 Observations 9 ANOVA df Regression 1 Residual 7 Total 8 Intercept X SS MS F Significance F 308.726674 308.726674 4.726028384 0.066230335 457.273326 65.32476086 766 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0% 15.3622393 13.89683242 1.105448985 0.305505467 -17.49854766 48.22302625 -17.49854766 48.22302625 0.411635565 0.189349746 2.17394305 0.066230335 -0.036105437 0.859376568 -0.036105437 0.859376568 Example – constant only SUMMARY OUTPUT Regression Statistics Multiple R 0.979624982 R Square 0.959665105 Adjusted R Square 0.834665105 Standard Error 9.785192895 Observations 9 ANOVA df Regression Residual Total Intercept CONST SS 1 8 9 18225 766 18991 MS F Significance F 18225 190.3394256 2.4809E-06 95.75 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0% 0 #N/A #N/A #N/A #N/A #N/A #N/A #N/A 45 3.261730965 13.79635552 7.35719E-07 37.47843491 52.52156509 37.47843491 52.52156509 Example - normal SUMMARY OUTPUT Regression Statistics Multiple R 0.987887431 R Square 0.975921577 Adjusted R Square 0.82962466 Standard Error 8.082373467 Observations 9 ANOVA df Regression Residual Total Intercept X CONST 2 7 9 SS MS F Significance F 18533.72667 9266.863337 141.8583584 8.88246E-06 457.273326 65.32476086 18991 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0% 0 #N/A #N/A #N/A #N/A #N/A #N/A #N/A 0.411635565 0.189349746 2.17394305 0.066230335 -0.036105437 0.859376568 -0.036105437 0.859376568 15.3622393 13.89683242 1.105448985 0.305505467 -17.49854766 48.22302625 -17.49854766 48.22302625 Solving for Bs Matrix Style    In regular algebra you can solve for b y = x * b -> y/x = b Another way to divide is to multiply by 1/x (inverse) Can’t just divide in matrix algebra you must multiply by inverse of the X matrix: B = Y * X-1  But in order to take the inverse the X matrix must be a square matrix (which rarely, if ever, happens) Pseudoinverse  The pseudoinverse - A pseudoinverse is a matrix that acts like an inverse matrix for non-square matrices:  Example: The Moore-Penrose pseudoinverse  Apseudo-1=(A’A)-1 *A’  The inverse of (A’A) is defined because it is square (so long as A is full rank, in other words no multicollinearity/singularity) Estimation   The goal of an estimator is to provide an estimate of a particular statistic based on the data There are several ways to characterize estimators B Estimators  bias   an unbiased estimator converges to the true value with large enough sample size Each parameter is neither consistently over or under estimated B Estimators  likelihood  the maximum likelihood (ML) estimator is the one that makes the observed data most likely ML estimators are not always unbiased for small N B Estimators  variance   an estimator with lower variance is more efficient, in the sense that it is likely to be closer to the true value over samples the “best” estimator is the one with minimum variance of all estimators

Multiple Regression (MR)

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock