Reading 2: Evaluating Regression Model Fit

Page No. 111Date//EvaluateRegressionModelFit&interpretingModelResultsANOVATableAnalysisofVarianceSource of VasiationDfSum of SquaresMean SquareRegression (explained)KEME9) RSSMSR = RSSKError ResidualSSEMSE = SSE(unexplained)E(4-4;)n-k-1Totaln-lC4:-9)SSTECY:-7)F-stat=MSRwith'k'&'n-k-1'DofMSEExampleAnanalystrunsaregressionofmonthlyvalue-stockK=returnonfiveindependentvariablesoverD=60-months Thetotalsum ofsquaresfortheregressionis460 &sumofsquarederrorsis170SSTSSEPrepareanANOVAtableRSS = 460 - 170=290Use informationfromtheANOVAtabletocalculateF- sest& R2Stddevofregressionresidued050.5StandarF-stat = MSRSSE-(MSE)SSEError of En-k-1.1]MSEEst (SEE) Page No. 112Cordation bet" predicted & actual value = JRDate //Source of VariationRegression (explained)ofSumofSquaresMean Squaie5=K29058Error (unexpected)54 n-k-170315Total59 n-l460Coefficient of Determination (R2)R2: variation % variation of dependent (4) variable explained (x) by%ofalltheindependentvariablesR2rangesbet"0&IAnR2 of 0.25 means "X" explains 25% of the variationR2 = Explained Variation = RSS2400.63=Total VariableSST960 = 63%= (Correlation (oeffient) 2For simple linear regression (i.e K= 1)For multiplelinearregression : R2 = (Ty.g)Ty4 iscalledthemultipler(R)All of theVariableAdjusted R2 OverfiltingUnady R2 increases when new variables are addedAdjR2appliesapenaltyfactortovariablesreflectaddedqualityofR2Radj=1-(n-1)h-k-I(1-R2)adjk Page No. 113Date / /Given regression with five independent variables , n=60SST 460 &= t SSE = 170 , calculate R2 & adj R2 ;R2 = 460-170 = 63 7.460Radj = 1-16-60-1 60-5-1X(1-0.63)=59.6%the betterInterpreting Adj R2Suppose we add four more independent variables R2 increasefrom 63% to Yues not 65% matter but adj R2 falls from 54.67. to58.7% which model should we choose ? If 4 variables were goodRais would increasAns:- Choose the original model because it has ahigheradjR2 It has better explanatory power & usesfive rather than rine variables.Example: 21:K= 6n=30R2 = 73%K=8n=30R2=75%Adj R2, = 1- ( 30-1(1-0.73)41.17.130-6-1Adj Rz = 1- ( 30-8- 30-1)(1-0.75%)39.58 %\Adding two more variables is not justified becauseadj R2 < adj R2 Page No. isDate/AIC & BIC Information CriteriaAkaike's info criterion (AIC) and schwarz's Bayesian infoqualitycriteria (BJC) can also be used to evaluate models.AIC 4Goodness of fitis BetterBIC are used to select the "best" model amonggroup with the same dependent variableLower values of both measures are betterBoth AIC & BIC evaluate the quality of modol fit amongCompeting modelsfor the same dependent variableAIC is used if the goal is to have a better forecast whileB)isusedifthegoalisabettergoodness of fitAIC =In (SSE t 2(ktl)Dont memorizeif K 1 AICBIC = nx In ( SSE ) + In(n) X (K+I)BIC imposes a higher penalty for overfittingK is a penalty parameter in both criteria : higher values ofk results in higher values of the criteriaBecause In(n) is greater than 2 for even small sumple sizes,the BIC metric imposes a higher penalty for Overfittingadding more V5 Page No. 115Date / /*Goodness of fit for the Rent ModelVariablesKSSRSSERR2-AdjAICBIC ModelAge\3318.932627.39.23%8.75%985.9992-4\Age Distance220946.115000.258.27%57.8%839.4849.22Age+Dist + Rest322395.613550 562.30%61.7822835.0 33) Considera model to explain variation in office rent usingage of the property dist from nearest metro stationand no. of restaurants in walking distanceQue : Which model is the most appropriate for use ingenerating forecasts !Ans: The model with all three independent variables hasthe lowest AIC & + hence is the most appropriatemodel for generating forecastQue : which model has a better goodness of fit !Ans : The model with all three independent variables hasthe lowest BIC & + hence the best goodness of fitAlso, the adj R2 for the three factor model is the highestof the three modelsJoint Hypothesis Test F-tesThe joint F-test is used to jointly test a Subset ofvariables in a multiple regresion StandardPage No. 116Confident Interval : intercept + t-stat ait Xerror ofDate/borcast/Nested models are models such that one model,Calledfull modelorunrestrictedmodelhasa higher no.ofindepen dentvariablewhileanothermodel.Calledtherestricedmoddhasonlyasubsetoftheindependent variableshe "restricked" model is based on a narrower set of indepvariablesnestedthebroader"unrestricked"modeinfull modelUnrestrictedmodel:Yi =bo+b1X1+b2X2tb3X3+EiRestrictedmodel:-Yi=b.+b1X1tE1Wewantto testHo :b,= b3 =0the following Ha hypothesisvsofbe of bs t 0atleast#0Thenullhypothesisisthattheslope coefficient of allindependentVariablesoutsidetherestrictednodeIarezeroDecisionrule: rejectHoifF(teststat)>Fe(criticalvalue)F-testevaluateswherewhethertherelativedecreasein SSE due to indusion of q additional variablesstatistically justified Page No. 11712th Jan' 2023Date / /We Calculate the F- stat to test this hypothesis as:F = CSSE - SSHEy)always +veqC SSEa)with 9 & (n-x-1) degree of freedomJMSE(n -----)with q and (n-k-1) degrees of freedomRrepresents the restricted model4respresents the unrestricted model9number of excluded variables in the restricted modelKindepent variable in the full modelExampleContinuing our previous rental model data, and using theinfo below 7 conduct a joint test of hypothese forthe slope coefficient 2 & 3 at a 5% ivl of significantModelKSSEnUnrestricted313,550.64191RestrictedI32,627.29191Numerator ofF= (SSER - SSEu) qDenominator ofI23(SSEu) / (n-k-1)1863.89 3.05 2.651873.89 3.0.4 2.65= (32627-29-13550.64) - 21883.89 30.4 265(13,550.64) / (191-3-1)Critical F-value with 2 & 187= 131.63degrees of freedom is 3.04 Page No. 118Date //at least One variabe is able explain variation=/ independent variableWe reject the null hypothesis & conclude that evduded at leastone of the two slope coefficient of O. theVariables is statistically diff fromF - testHo : b, b2 = b3 =0 Vs atleast Ha b toThe general linear F-test is an extension of the jointF- test.The null hypothesis is that the slope cofficients on allindependent variables in the unies tricted model = zeroHo : b, = bz = b3 =0 vers Ha : at least one bj F 0The F-stat then becomesF = CRSSuk(SSEu) / (n-k-1)Predicting the dependent VariablePredicting est the value of the dependent variable using inmultiple regression model is similar to thatwith a simple regression:For each independent variables mattly the est slopecofficient by the assumed value of that variableSum these valuesThen add the est inter cept coffient. PageNo. 119Date//Assume payout ratio (PR) = 50%VariablebiSbi& yield curve slope (4CS) = 47.bo-11.6%1657%PR0.250.032Predict the 10- yr real S&PYCS0.140.828earning growth.YEGIO = -11.6% + S(50%) t 0.14 (4%) = 1.46 %The model predicts a 1.46 7. S&P real earnings growthlate.Model Specification IssuesModel Specification- Selection of ex planatory variables- Transformation of variablesAffects reliability of inference [ hypothesis test-Point:- if model specified incorrectly regressionCoefficients will be biased andinconsistent