Study Notes - Quantitative Methods

Level II of the CFA® 2025 Exam Study Notes - Quantitative Methods Offered by AnalystPrep Last Updated: Oct 10, 2025 1 ©2025 AnalystPrep “This document is protected by International copyright laws. Reproduction and/or distribution of this document is prohibited. Infringers will be prosecuted in their local jurisdictions.” Table of Contents 1 - Basics of Multiple Regression and Underlying Assumptions 3 2 - 3 - Model Misspecification 39 4 - Extensions of Multiple Regression 58 5 - Time-Series Analysis 69 6 - Big Data Projects 155 Evaluating Regression Model Fit and Interpreting Model Results 19 2 © 2014-2025 AnalystPrep. Reading 1: Basics of Multiple Regression and Underlying Assumptions LOS 1(a) Describe the types of investment problems addressed by multiple linear regression and the regression process Multiple linear regression describes the variation of the dependent variable by using two or more independent variables. When used properly, it can improve predictions. However, if used incorrectly, it can create spurious relationships that can undermine predictions. Typically, a multiple regression model takes the following form: Yi = b0 + b 1 X1,i + b2 X2 ,i + … + b k Xk,i + ϵi Where: Y i = Dependent variable. b0 = Intercept term. b1 , b 2 ,… , bk = Slope coefficients. X1 ,i , X2 ,i , … , Xk ,i = Independent variables. ϵ i = Error term. n = Number of observations. A regression equation has k slope coefficients and k + 1 regression coefficients. The intercept term is defined as the value of the dependent variable when the independent variables are zero. On the other hand, the slope coefficient is defined as the estimated change in the dependent variable given a one-unit change in the independent variable, keeping the other independent variables constant. Researchers can use multiple regression to test existing theories, identify relationships between variables, or forecast. The researcher must specify the model to determine the good-fit criteria for the regression 3 © 2014-2025 AnalystPrep. model, including an independent variable. Once the regression model has been specified, it must be estimated and analyzed to ensure it satisfies all the key assumptions. It is equally noteworthy that a researcher can use multiple regression to test existing forecasting theories. Alternatively, multiple regression can further be used to identify relationships between variables after the model is tested and deemed acceptable for out-of-sample performance. A single factor cannot adequately explain or forecast the complex world of investments. Due to their complexity, statistical tests and fundamental justification are critical in the exhaustive explanation of financial and economic relations. There are several ways to use multiple regression, including: A company's profitability, growth, revenue, and market share are variables that an investor is interested in. These variables can predict if a company will run into financial difficulties. An analyst seeking to determine how a company's stock price and trading volume change daily can correlate them. An analyst can use linear regression to determine the relationship between variables. 4 © 2014-2025 AnalystPrep. Question Which of the following most accurately detects whether the underlying assumptions of multiple linear regression models are satisfied? A. Scatter plots. B. Residual plots. C. Diagnostic plots. Solution The correct answer is C. Diagnostic plots for multiple regression show the prediction errors against predicted values. Therefore, they are useful in the determination of the gaps that a researcher should address in their data to improve the accuracy of their predictions. Using diagnostic plots, a researcher can determine whether the assumptions of multiple linear regression are valid. A is incorrect. The scatterplot is useful for detecting nonlinear relationships between dependent and independent variables. B is incorrect. A residual plot is an effective tool for detecting violations of homoskedasticity and error independence. 5 © 2014-2025 AnalystPrep. LOS 1(b) Formulate a multiple linear regression model, describe the relation between the dependent variable and several independent variables, and interpret estimated regression coefficients Example: Multiple Regression in Investment World James Chase, an investment analyst, wants to determine the impact of inflation rates and real rates of interest on the price of the US Dollar index (USDX). Chase uses the multiple regression model below: P = b0 + b 1I N F + b 2 IR + ϵt Where: P = Price of USDX. I NF = Inflation rate. I R = Real rate of interest. ϵ t = Error term. The regression of the price of USDX on inflation and real interest rates generates the following results: Coefficients Intercept Inflation rates Real interest Rates 81 −276 902 Standard Error 7.9659 233.0748 279.6949 t Stat P-value 10.1296 −1.1833 3.2266 0.0000 0.2753 0.0145 Chase can express the multiple regression equation as follows: P = 81 − 276I NF + 902I R The regression coefficient estimate of the inflation rate is negative. This indicates that an increase in the inflation rates causes a decrease in the price of the US Dollar index (USDX). 6 © 2014-2025 AnalystPrep. Furthermore, the positive real rate of interest coefficient implies that an increase in the price of USDX accompanies the real interest rate. The t-statistic indicates that only the real interest rate variable is significant at the 5% significance level. The intercept term is defined as the value of the dependent variable when the independent variables are zero. On the other hand, each slope coefficient is the estimated change in the value of the dependent variable for a one-unit change in the value of the respective independent variable, keeping the other independent variables constant. Slope coefficients are also called partial slope coefficients. Continuing with this example: P = 81 − 276I NF + 902I R Where: P = Price of the US Dollar index. I NF = Annual inflation rate. I R = Annual real rate of interest. The regression equation is interpreted as follows: The intercept term of 81 implies that the price of USDX is $81 when both the inflation rate and real interest rate are 0. A 1% increase in the inflation rate leads to a $276 decrease in the price of USDX, keeping real interest rates constant. On the other hand, a 1% increase in the real interest rate leads to a $902 increase in the price of USDX, keeping the inflation rate constant. 7 © 2014-2025 AnalystPrep. Question Adil Suleman, CFA, wishes to establish the possible drivers of a company's percentage return on capital (ROC). Suleman identifies performance measures such as the profit margin (%), sales, and debt ratio as possible drivers of ROC. He obtains the following results from the regression of ROC on profit margin (%), sales, and debt ratio. Intercept Sales Debt ratio Profit Margin (%) SUMMARY OUTPUT Coefficients Standard Error 8.6531 0.9174 0.0009 0.0005 0.0229 0.0165 0.2996 0.0564 t Stat P-value 9.4323 1.7644 1.3880 5.3146 0.0000 0.0922 0.1797 0.0000 Which independent variable(s) is (are) most likely statistically and significantly different from zero at the 5% significance level, assuming the sample size is 25? A. Profit margin. B. Sales and profit margin. C. Sales and debt ratio. Solution The correct answer is A. An independent variable is statistically significant if its p-value is less than the significance level, in this case, 5% or 0.05. Therefore, only the profit margin is statistically and significantly different from zero at the 5% significance level. B and C are incorrect. At a 5% significance level, only the profit margin is statistically significantly different from zero. 8 © 2014-2025 AnalystPrep. LOS 1(c) Explain the assumptions underlying a multiple linear regression model and interpret residual plots indicating potential violations of these assumptions The following assumptions are used to build multiple regression models: The relationship between the dependent variable, and the independent variables, is linear. The independent variables are not random. There is no definite linear relationship between two or more independent variables. A high correlation between two or more independent variables is known as multicollinearity. The expected value of the error term, conditional on the independent variables, is equal to 0. The variance of the error term is equal for all observations. This is known as the homoskedasticity assumption. The error term is uncorrelated across all observations. The error term is normally distributed. The following assumptions are investigated, and their outcomes are analyzed (if violated) as follows: i. Linearity: The regression algorithm would mathematically fail to capture the trend when fitted to a nonlinear, non-additive data set. 9 © 2014-2025 AnalystPrep. A prediction based on unobserved data will also be incorrect. The model can capture the nonlinear effect by including polynomial terms. This plot may show non-linearity if any pattern (such as a parabolic shape) appears. In other words, the model fails to capture nonlinear effects. 10 © 2014-2025 AnalystPrep. ii. Autocorrelation: A model's accuracy is drastically reduced when correlation exists in error terms. A time series model is characterized by this interaction, in which the next instant depends on the previous one. Correlated error terms (autocorrelation) cause the estimated standard errors to be underestimated, meaning they are lower than the true standard errors. Note that confidence intervals and prediction intervals are narrowed if this occurs. The probability of the actual coefficient value being contained in a 95% confidence interval is lower than 0.95 if the confidence interval is narrower. To check for autocorrelation, calculate Durbin-Watson (DW) statistics. The value must be 11 © 2014-2025 AnalystPrep. between 0 and 4. If DW = 2 implies no autocorrelation, 0 < DW < 2 implies positive autocorrelation, while 2 < DW < 4 indicates negative autocorrelation. The residual values can also be plotted against time to identify seasonal or correlated patterns. iii. Multicollinearity: A moderately or highly correlated set of independent variables will exhibit this phenomenon. In a model with correlated variables, it is challenging to determine the true relationship between a predictor and response variable. The difficulty lies in determining which variable contributes to the response's prediction. Furthermore, standard errors tend to increase when correlated predictors are present. A large standard error also leads to wider confidence intervals, resulting in less accurate slope parameters. Correlated predictors have a different regression coefficient depending on which other predictors are included in a model. A variable that strongly / weakly affects a target variable will result in an incorrect conclusion. 12 © 2014-2025 AnalystPrep. Scatter plots can visualize correlations between variables to check for multicollinearity. VIF factor can also be used. VIF value <= 4 suggests no multicollinearity, whereas a value of >= 10 implies serious multicollinearity. iv. Heteroskedasticity: In heteroskedasticity, the error terms have non-constant variances. 13 © 2014-2025 AnalystPrep. An outlier or extreme leverage value will typically lead to non-constant variance. These values disproportionately affect a model's performance because they are given too much weight. Whenever heteroskedasticity occurs, confidence intervals for out-of-sample predictions become unrealistically large or small. If a residuals versus fitted values plot exhibits heteroskedasticity, the plot will show a funnel shape. Alternatively, you can conduct a Breusch-Pagan/Cook-Weisberg or White general test. 14 © 2014-2025 AnalystPrep. Aside from detecting homoskedasticity, the above plot is used to determine variance equality. As you can see, the residuals are spread out along the range of predictors. It uses standardized residual values instead of residuals versus fitted values. v. Normal Distribution of error terms: Confidence intervals may be too wide or narrow if the error terms are not normally distributed. Using least squares to minimize coefficients becomes difficult once confidence intervals become unstable. If non-normal distributions are present, there will probably be some unusual data points that will need to be closely examined to make a better model. The best way to determine the normal distribution of error terms is to plot a QQ plot. Kolmogorov-Smirnov and Shapiro-Wilk tests can also be used to test for normality. 15 © 2014-2025 AnalystPrep. 16 © 2014-2025 AnalystPrep. The q-q or quantile-quantile plot is used to verify the assumption that a data set follows a normal distribution. We can determine whether the data follows a normal distribution with this plot. There would be a fairly straight line on the plot if that were the case. Note that the straight line deviates when there is no normality in the errors. 17 © 2014-2025 AnalystPrep. Question Which of the following is least likely an assumption of the multiple linear regression model? A. The independent variables are not random. B. The error term is correlated across all observations. C. The expected value of the error term, conditional on the independent variables, is equal to zero. Solution The correct answer is B. The error term is uncorrelated across all observations. E (ϵ i ϵj ) = 0 ∀ i ≠ j Other assumptions of the classical normal multiple linear regression model include the following: i. The independent variables are not random. Additionally, there is no exact linear relationship between two or more independent variables. ii. The error term is normally distributed. iii. The expected value of the error term, conditional on the independent variables, is equal to 0. iv. The variance of the error term is the same for all observations. v. A linear relation exists between the dependent variable and the independent variables. A and C are incorrect. They both indicate the correct assumptions of the multiple linear regression model. 18 © 2014-2025 AnalystPrep. Reading 2: Evaluating Regression Model Fit and Interpreting Model Results LOS 2(a) Evaluate how well a multiple regression model explains the dependent variable by analyzing ANOVA table results and measures of goodness of fit R-squared (R 2 ) measures how well an estimated regression fits the data. It is also known as the coefficient of determination and can be formulated as: 2 ∑ni=1 ( Ŷi − Y¯) Sum of regression squares R = = 2 Sum of squares total ∑ ni=1 (Y i − Y¯ ) 2 Where: n = Number of observations. Y i = Dependent variable observations. Ŷ i = Dependent variables predicted value to the independent variable. Y¯ = Dependent variable mean. In the presence of independent variables, R 2 will either increase or remain constant. However, R 2 cannot be used to measure the goodness of fit of a model as it will not decrease with the addition of independent variables. Limitations of R2 It is impossible to determine the statistical significance of the coefficients from R 2. A bias in the predicted coefficients or estimates cannot be determined with R2 . When a model is good, it has a high R2 ; when it is bad, it has a low R 2 , usually due to overfitting and biases in the model. An overfitted regression model is one with too many independent variables to the number of 19 © 2014-2025 AnalystPrep. observations in a sample. Overfitting may produce coefficients that do not reflect the true relationship between the independent and dependent variables. 2 Multiple regression software packages usually produce an adjusted R2 (R̄ ) as an alternative measure of goodness of fit. Using adjusted R2 in regression is beneficial since it does not automatically increase when more independent variables are included, given that it adjusts for degrees of freedom. ⎡ R¯2 = 1 − ⎢ ⎣ Sum of squares error ⎤ ⎥ Sum of squares total ⎦ −1 n−k−1 n Therefore, the relationship between R̄ 2 and R2 can mathematically be derived as follows: R¯2 = 1 − [( n− 1 ) (1 − R2 )] n − k− 1 Note that: If k ≥ 1 then R 2 > adjusted R 2 the result is that adjusted R2 can be negative while R2 is zero at minimum. When including a new variable in the regression, the following should be taken into consideration: R̄ 2 increases when the coefficient t-statistic is > |1.0|. R̄ 2 decreases when the coefficient t-statistic is < |1.0|. At typical significance levels, 5% and 1%, a t-statistic with an absolute value of 1.0 does not indicate that the independent variable is different from zero. Therefore, the adjusted R 2 doesn't demonstrate that it will increase significantly. ANOVA Table One of the outputs of multiple regression is the ANOVA table. The following shows the general 20 © 2014-2025 AnalystPrep. structure of an Anova table. ANOVA Regression Df (degrees of freedom) k Residual n − (k + 1) Total n− 1 SS (Sum of squares) RSS (Explained variation) SSE (Unexplained variation) SST (Total variation) MSS (Mean sum of squares) MSR MSE We can use the information in an ANOVA table to determine R 2 , the F-statistic, and the standard error estimates (SEE) as expressed below: R2 = SSR SST MSR MSE F= SSE = √MSE Where: RSS k SSE MSE = n− k− 1 MSR = Example: Interpreting Regression Output Consider the following regression results generated from multiple regression analysis of the price of the US Dollar index on the inflation rate and real interest rate. 21 © 2014-2025 AnalystPrep. ANOVA Regression Residual Total df 2 7 9 SS 432.2520 200.6349 632.8869 Intercept Inflation rates Real interest Rates Coefficients 81 −276 902 Standard Error 7.9659 233.0748 279.6949 Significance F 0.0179 Given the above information, the regression equation can be expressed as: P = 81 − 276I NF + 902I R Where: P = Price of USDX. I NF = Inflation rate. I R = Real interest rate. R 2 and adjusted R 2 can also be calculated as follows: R2 = 432.2520 SSR = = 0.6830 = 68.30% SST 632.8869 n−1 10 − 1 ) (1 − R2 ) = 1 − (1 − 0.6830) n −k −1 10 − 2 − 1 = 0.5924 = 59.24% Adjusted R2 = 1 − ( It's important to note the following: Multiple regression does not provide a straightforward explanation of adjusted R 2 in terms of the variance explained by the dependent variable, as is the case in simple regression. Adjusted R2 does not indicate whether a regression coefficient's predictions are true or biased. Residual plots and other statistics are required to determine whether or not the predictions are accurate. 22 © 2014-2025 AnalystPrep. To assess the significance of the model's fit, we use the F-Statistic and other goodnessof-fit metrics from the ANOVA rather than R2 and adjusted R2 . 23 © 2014-2025 AnalystPrep. Question Which of the following is most appropriate for adjusted R2 ? A. It is always positive. B. It may or may not increase when one adds an independent variable. C. It is non-decreasing in the number of independent variables. Solution The correct answer is B. The value of the adjusted R 2 increases only when the added independent variables improve the fit of the regression model. Moreover, it decreases when the added variables do not improve the model fit sufficiently. A is incorrect: The adjusted R2 can be negative if R2 is low enough. However, multiple R2 is always positive. C is incorrect: The adjusted R2 can decrease when the added variables do not improve the model fit by a good enough amount. However, multiple R2 is nondecreasing in the number of independent variables. For this reason, it is less reliable as a measure of goodness of fit in regression with more than one independent variable than in a one-independent variable regression. 24 © 2014-2025 AnalystPrep. LOS 2(b) Formulate hypotheses on the significance of two or more coefficients in a multiple regression model and interpret the results of the joint hypothesis tests In multiple regression, the intercept in simple regression represents the expected value of the dependent variable when the independent variable is zero, while in multiple regression, it's the expected value when all independent variables are zero. The interpretation of slope coefficients remains the same as in simple regression. Tests for single coefficients in multiple regression are similar to those in simple regression, including one-sided tests. The default test is against zero, but you can test against other hypothesized values. In some cases, you might want to test a subset of variables jointly in multiple regression, comparing models with and without specific variables. This involves a joint hypothesis test where you restrict some coefficients to zero. To test the slope against a hypothesized value other than zero we will need to: You can conduct the test by either modifying the hypothesized parameter value, Bj, in the test statistic or by comparing Bj with the confidence interval boundaries derived from the regression coefficient output. At times, we may want to collectively test a subset of variables within a multiple regression. To illustrate this concept and set the stage, let's say we aim to compare the regression outcomes for a portfolio's excess returns using Fama and French's three-factor model (MKTRF, SMB, HML) with those using their five-factor model (MKTRF, SMB, HML, RMW, CMA). Given that both models share three factors (MKTRF, SMB, HML), the comparison revolves around assessing the necessity of the two additional variables: the return difference between the most profitable and least profitable firms (RMW) and the return difference between firms with the most conservative and most aggressive investment strategies (CMA). The primary goal in determining the superior model lies in achieving simplicity by identifying the most effective independent variables in 25 © 2014-2025 AnalystPrep. explaining variations in the dependent variable. Now, let's contemplate a more comprehensive model: Yi = b0 + b 1X 1 i + b2 X2 i + b3 X3 i + b 4 X4 i + b5 X5 i + εi The model above has five independent variables and it is referred to as the unrestricted model. Sometimes we may want to test whether and together have no significant contribution used to explain the dependent variable, i.eX4 = X5=0. We compare the full model (unrestricted model) to: Y i = b 0 + b1 X1 i + b2 X2 i + b 3 X3 i + εi This is referred to as the restricted model because it excludes X 4 and X5 , which will have the effect of restricting the slope coefficients on X4 and X5 to be equal to 0. These models are also termed nested models because the restricted model is contained within the unrestricted model. This model comparison entails a null hypothesis that encompasses a combined restriction on two coefficients, namely, H 0 : b4 = b 5 = 0 against H A : b 4 or b5 ≠ 0 We employ a statistic to compare nested models, pitting the unrestricted model against a restricted version with some slope coefficients set to zero. This statistic assesses how the joint restriction affects the restricted model's ability to explain the dependent variable compared to the unrestricted model. We test the influence of omitted variables jointly using an F-distributed test statistic. F = (Sum of squares error restricted model − Sum of squares error unrestricted)/q (Sum of squares error unrestricted model)/(n − k − l) Where q= Number of restrictions When we want to compare an unrestricted model to a restricted model \(q\)=2 because we are testing the null hypothesis of b4 = b5 = 0.The F-statistic has n − k − 1 and q degrees of freedom. In summary, the unrestricted model includes a larger set of explanatory variables, whereas the 26 © 2014-2025 AnalystPrep. restricted model has q fewer independent variables because the slope coefficients of the excluded variables are forced to be zero. Why not just conduct hypothesis tests for each individual variable and make conclusions based on that data? In many cases of multiple regression with financial variables, there's typically some level of correlation among the variables. As a result, there may be shared explanatory power that isn't fully accounted for when testing individual slopes. Example Table 1: Partial ANOVA Results for Models Using Three and Five Factors Source Factors Restricted 1, 2, 3 Unrestricted 1 , 2, 3, 4, 5 Residual sum of squares 66.9825 58.7232 Mean squares errors 1.4565 1.3012 Degrees of freedom 44 42 Test of Hypothesis for factors 4 and 5 at 1% Level of significance Step1: State the hypothesis H 0 : b4 = b5 = 0 vs. H a : at least bj ≠ 0 Step 2: Identify the appropriate test statistic. F = (Sum of squares error restricted model − Sum of squares error unrestricted)/q (Sum of squares error unrestricted model)/(n − k − l) Step 3: Specify the level of significance. α=1% (one-tail, right side) Step 4: State the decision rule. Critical F-value = 5.149. Reject the null if the calculated F-statistic exceeds 5.149. 27 © 2014-2025 AnalystPrep. Step 5: Calculate the test statistic. F= (66.9825 − 58.7232)/2 4.1297 = = 2.9536 58.7232/42 1.3982 Step 6: Make a decision Fail to reject the null hypothesis since the answer is less than the critical fvalue. Hypothesis testing involves testing an assumption regarding a population parameter. A null hypothesis is a condition believed to be false. We reject the null hypothesis in the presence of enough evidence against it and accept the alternative hypothesis. Hypothesis testing is performed on the estimated slope coefficients to establish if the independent variables explain the variation in the dependent variable. The t-statistic for testing the significance of the individual coefficients in a multiple regression model is calculated using the formula below: t= b̂j − bH 0 Sb̂ j Where: b̂j = Estimated regression coefficient. bH 0 = Hypothesized value. Sb̂j = The standard error of the estimated coefficient. It is important to note that the test statistic has n − k − 1 degrees of freedom, where k is the number of independent variables and 1 is the intercept term. A t-test tests the null hypothesis that the regression coefficient equals some hypothesized value against the alternative hypothesis that it does not. 28 © 2014-2025 AnalystPrep. H 0 : bj = v vs Ha : bj ≠ v Where: v = Hypothesized value. The F-test determines whether all the independent variables help explain the dependent variable. It is a test of regression's overall significance. The F-test involves testing the null hypothesis that all the slope coefficients in the regression are jointly equal to zero against the alternative hypothesis that at least one slope coefficient is not equal to 0. i.e.: H 0 : b1 = b2 =. . . = bk = 0 versus H a: at least one bj ≠ 0 We must understand that we cannot use the t-test to determine whether every slope coefficient is zero. This is because individual tests do not account for interactions among the dependent variables. The following inputs are required to determine the test statistic for the null hypothesis. i. The total number of observations, n. ii. The total number of slope coefficients to be estimated, k + 1, where k = number of slope coefficients. iii. The residual sum of squares (SSE), which is the unexplained variation. iv. The regression sum of squares (RSS), which is the explained variation. The F-statistic (which is a one-tailed test) is computed as: F = ( RSS ) k ( SS E ) n −(k +1) = Mean Regression sum of squares (MSR) Mean squared error(MSE) Where: RSS = Regression sum of squares. SSE = Sum of squared errors. n = Number of observations. 29 © 2014-2025 AnalystPrep. k = Number of independent variables. A large value of F implies that the regression model explains variation in the dependent variable. On the other hand, the value of F will be zero when the independent variables do not explain the dependent variable. The F-test is denoted as Fk, (n −(k +1) . The test should have k degrees of freedom in the numerator and n − (k + 1) degrees of freedom in the denominator. Decision Rule We reject the null hypothesis at a given significance level, α , if the calculated value of F is greater than the upper critical value of the one-tailed F distribution with the specified degrees of freedom. I.e., reject H 0 if F − statistic > Fc(critical value). Graphically, we see the following: Example: Calculating and Interpreting the F-statistic The following ANOVA table presents the output of the multiple regression analysis of the price of the US Dollar index on the inflation rate and real interest rate. 30 © 2014-2025 AnalystPrep. df Regression 2 Residual 7 Total 9 SS 432.2520 200.6349 632.8869 ANOVA MS 216.1260 28.6621 F 7.5405 Significance F 0.0179 Test the null hypothesis that all independent variables are equal to zero at the 5% significance level. Solution We test the null hypothesis: H 0 : β1 = β2 = 0 verses H a : at least one bj ≠ 0 with the following variables: Number of slope coefficients: (k) = 2. Degrees of freedom in the denominator: n − (k + 1) = 10 − (2 + 1) = 7. Residual sum of squares: RSS = 432.2520. Sum of squared errors: SSE = 200.6349. Thus, F = ( RSS ) k ( SS E ) n −(k+1) = 432.2520 2 200.6349 7 = 7.5405 For α = 0.05, the critical value of F with k = 2 and (n − k − 1) = 7 degrees of freedom F0.05 ,2 ,7, is approximately 4.737. 31 © 2014-2025 AnalystPrep. The calculated value of the F-test statistic, 7.5405, is greater than the critical value of F, 4.737. We, therefore, reject the null hypothesis that the coefficients of both independent variables equal 0. Additionally, you will notice that from the ANOVA table, the column “Significance F” reports a pvalue of 0.0179, which is less than 0.05. The p-value implies that the smallest level of significance at which the null hypothesis can be rejected is 0.0179. Analyzing Multiple Regression Models for Model Fit Statistic Adjusted R 2 Akaike's information criterion (AIC) Assessing criteria It is better if it is higher. It is better if it is lower. A lower number is better. Schwarz's Bayesian information criterion (BIC) An analysis of slope coefficients using The critical t-value(s) are located the t-statistic outside the given range for the selected significance level. Test of slope coefficients using the The F-value for the selected F-test significance level exceeds the critical value. Note that: Several models that explain the same dependent variable are evaluated using Akaike's information criterion (AIC). Often, it can be calculated from the information in the regression output, but most regression software includes it as part of the output. Sum of squares error 32 © 2014-2025 AnalystPrep. AI C = n ln [ Sum of squares error ] + 2(k + 1) n Where: n = Sample size. k = Number of independent variables. 2 (k + 1) = The model is penalized when independent variables are included. Models with the same dependent variables can be compared using Schwarz's Bayesian Information Criterion (BIC). BI C = n ln [ Sum of squares error ] + ln(n)(k + 1) n Models with more parameters incur a significant penalty, so BIC prefers models with fewer parameters. It is because ln(n) exceeds 2, even for very small sample sizes. 33 © 2014-2025 AnalystPrep. Question Which of the following statements is most accurate? A. The best-fitting model is the regression model with the highest adjusted R 2 and low BIC and AIC. B. The best-fitting model is the regression model with the lowest adjusted R2 and high BIC and AIC. C. The best-fitting model is the regression model with both high adjusted R2 and high BIC and AIC. Solution The correct answer is A. A regression model with a high adjusted R2 and a low AIC and BIC will generally be the best fit. B and C are incorrect. The best-fitting regression model generally has a high adjusted R 2 and a low AIC and BIC. 34 © 2014-2025 AnalystPrep. LOS 2(c) Calculate and interpret a predicted value for the dependent variable, given the estimated regression model and assumed values for the independent variable Multiple regression uses the same process employed in simple regression for predicting the dependent variable's value. It, however, does so with more items summed up, as shown below: k Ŷ f = ^ b0+^ b 1 X1f + ^ b 2 X2f + … + ^ b k Xkf = ^ b0 + ∑ ^ b j Xjf j=1 Where: Y^f = Predicted (forecasted) value of the dependent variable. ^ b j Xf = This value is the estimated slope of the coefficient multiplied by the assumed value of the variable. ^ b 0 = Estimated intercept coefficient. A note on predicting with multiple regression models. Any prediction of the dependent variable must also include all five independent variables, even if they are statistically insignificant. This, for example, applies if all five independent variables are used to estimate a regression model. To estimate slope coefficients, correlations between these variables were taken into account. The intercept term must also be included in any prediction of the dependent variable. Example: Calculating the Predicted Value of a Dependent Variable Consider the following regression equation of the price of USDX on inflation and real interest rates. P = b0 + b 1I N F + b 2 IR + ϵt The following table gives the regression results: 35 © 2014-2025 AnalystPrep. Multiple R R Square Adjusted R Square Standard Error Observations Coefficients Intercept Inflation rates Real Interest Rates 81 −276 902 0.8264 0.6830 0.5924 5.3537 10 Standard Error 7.9659 233.0748 279.6949 t Stat P-value 10.1296 −1.1833 3.2266 0.0000 0.2753 0.0145 Use the estimated regression equation above to calculate the predicted price of the US dollar index (USDX), assuming the inflation rate is 3.5% and the real interest rate is 4%. Solution Ŷ i = b̂0 + b̂1 X̂ 1i + b̂2 X̂2i + … + b̂k X̂ki = 81 + (−276 × 0.035) + (902 × 0.04) = $107.42 36 © 2014-2025 AnalystPrep. Question Consider the following multiple regression results of the return on capital (ROC) on performance measures (profit margin (%), sales, and debt ratio). Multiple R R Square Adjusted R Square Standard Error Observations Coefficients Intercept Sales Debt ratio Profit Margin(%) 8.6531 0.0009 0.0229 0.2996 0.7906 0.6251 0.5715 1.1963 25 Standard Error 0.9174 0.0005 0.0165 0.0564 t Stat P-value 9.4323 1.7644 1.3880 5.3146 0.0000 0.0922 0.1797 0.0000 Given that sales = 1000, debt ratio = 20, and profit margin = 20%, the predicted value of the return on capital (ROC) according to the regression model is closest to: A. 7.38%. B. 8.29%. C. 16.03%. Solution The correct answer is C. The regression equation is expressed as: ROC = 8.653 + 0.0009S + 0.0229DR + 0.2996PM ROC = 8.6531 + (0.0009 × 1000) + (0.0229 × 20) + (0.2996 × 20) = 16.03% A and B are incorrect. From the resulting calculation, the correct answer is 16.03%. 37 © 2014-2025 AnalystPrep. 38 © 2014-2025 AnalystPrep. Reading 3: Model Misspecification LOS 3(a) Describe how model misspecification affects the results of a regression analysis and how to avoid common forms of misspecification Model specification involves selecting independent variables to include in the regression and the functional form of the regression equation. Here, comprehensive guidelines are provided for accurately defining a regression, followed by an explanation of common model misspecifications. Exhibit 1 succinctly presents the principles for proper regression model specification: 1. Grounding the model in economic reasoning for variable selection. 2. Ensuring each included variable plays a vital role in the regression. 3. Assessing the model's performance beyond the training dataset to prevent overfitting. 4. Selecting an appropriate functional form, especially when anticipating nonlinear relationships among regressors. 5. Ensuring adherence to regression assumptions by addressing issues like heteroskedasticity, serial correlation, or multicollinearity. The subsequent discussion focuses on understanding model specification errors to enhance model development and foster a more informed approach to investment research. Incorrectly specified functional forms in regression estimation can manifest in various ways. These manifestations include: 1. Omitted variables. 2. Misrepresenting relationships between variables. 3. Inappropriate variable scaling. 4. Improper data pooling. Each of these errors may lead to issues like heteroskedasticity or serial correlation, impacting the reliability of regression results. Omitted Variables 39 © 2014-2025 AnalystPrep. Omitted variable bias occurs when an important independent variable is excluded from a regression. If the true model includes X2 but we estimate without it, like this: Yi = b0 + b1 X1 i + ε i instead of Yi = b 0 + b 1 X1 i + b2 X2 i + εi it causes misspecification. If the omitted variable ( X2) is uncorrelated with X1 , the misspecified regression's residual, b 2 X2 i + εi, deviates from an expected zero value and lacks an independent identical distribution based on X2. This bias affects the intercept estimate, but X 1's coefficient might still be accurate. However, if the omitted variable ( X2) correlates with the included variable ( X1 ), the model's error becomes correlated with X1. This correlation leads to biased and inconsistent estimations for the regression coefficients, affecting the accuracy of the coefficients, intercept, and residuals, and making standard errors unreliable for statistical tests. Inappropriate Form of VariablesTop of Form A common mistake in regression involves using an improper data form instead of a suitable transformed version. For instance, neglecting nonlinearity in the relationship between variables by assuming a linear connection can lead to misspecification. To address this, it's crucial to consider whether economic theory supports a nonlinear relationship. Plotting the data helps detect nonlinearity; if variables show linearity with proportional changes, transforming them, such as taking the natural logarithm, can rectify this misspecification. Inappropriate Scaling of Variables Using unscaled data in regressions instead of scaled data, when scaling would be more suitable, can result in model misspecification. Analysts frequently face the decision of whether to scale variables before comparing data among companies. For instance, analysts commonly employ common-size financial statements to compare companies. These statements streamline comparability across companies, enabling analysts to swiftly assess trends in profitability, 40 © 2014-2025 AnalystPrep. leverage, efficiency, and other factors within a group of companies. Inappropriate Pooling of Data Improper data pooling occurs when combining samples unsuitably, often due to structural breaks in data behavior. This could stem from changes in regulations or shifts from low to high volatility periods. Such data, in a scatterplot, appears as separate clusters with little correlation due to differing cluster means. Analysts facing discernible subsamples should estimate the model using the most representative data for the forecasting period. 41 © 2014-2025 AnalystPrep. Question Which of the following is NOT a potential consequence of misspecified functional form in regression analysis? A. Heteroskedasticity due to omitted variables. B. Multicollinearity caused by inappropriate variable scaling. C. Serial correlation arising from inappropriate data pooling. Solution The correct answer is B. Misspecified functional form in regression analysis can lead to several issues: Omitted variables may cause heteroskedasticity or serial correlation in the regression results. Inappropriate form of variables might lead to heteroskedasticity if a nonlinear relationship between variables is ignored. Inappropriate variable scaling can cause heteroskedasticity or multicollinearity. Inappropriate data pooling can result in heteroskedasticity or serial correlation in the model's output. Stating "Multicollinearity caused by inappropriate variable scaling," does not align with the outlined consequences of misspecified functional form in the regression analysis. Multicollinearity is generally related to high correlations between independent variables and is not explicitly associated with variable scaling. 42 © 2014-2025 AnalystPrep. LOS 3(b) Explain the Types of Heteroskedasticity and How It Affects Statistical Inference One of the assumptions underpinning multiple regression is that regression errors are homoscedastic. In other words, the variance of the error terms is equal for all observations: E (ϵ 2i ) = σϵ2 , i = 1, 2, … , n In reality, the variance of errors differs across observations. This is known as heteroskedasticity. Heteroskedasticity is a statistical concept that refers to the non-constant variance of a dependent variable. In other words, it occurs when the variability of a dependent variable is unequal across different values of an independent variable. This can often be seen in financial data, where the volatility of stock prices tends to be greater during periods of economic uncertainty. While heteroskedasticity can present challenges for statistical analysis, it can also provide valuable insights into the relationships between variables. For instance, by taking heteroskedasticity into account, economists can better understand how changes in one variable may impact the variability of another. One way to test for heteroskedasticity is to look at the residuals from the regression. Uneven distribution of residuals (errors) could signify heteroskedasticity. 43 © 2014-2025 AnalystPrep. Types of Heteroskedasticity There are two main types of heteroskedasticity: unconditional and conditional. Unconditional heteroskedasticity occurs when the heterogeneity variance is not correlated with the values of the independent variables. Even though this goes against the homogeneity assumption, it does not produce major problems concerning statistical inference. On the other hand, conditional heteroskedasticity happens when the error variance is dependent on the values of the independent variables. This creates considerable difficulties for statistical inference. Fortunately, there are many software programs that can identify and fix this issue. Effects of Heteroskedasticity Heteroskedasticity Estimators and the Consistency of Regression Parameter One common misconception about heteroskedasticity is that it affects the consistency of regression parameter estimators. This is not the case. The ordinary least squares (OLS) estimators are still consistent, meaning that they will converge in probability to the true values of the parameters as the sample size increases. However, estimates of the variance and covariance of the estimators will be biased if heteroskedasticity is present in the data. Heteroskedasticity and the F-test for Overall Significance Another implication of heteroskedasticity is that it can make the F-test for overall significance unreliable. The F-statistic depends on the homoscedasticity assumption. So, if this assumption is violated, the test may not be valid. Heteroskedasticity and t-tests for Individual Coefficients Another common issue with heteroskedasticity is that it can introduce bias into estimators of the 44 © 2014-2025 AnalystPrep. standard error of regression coefficients, making t-tests for individual coefficients unreliable. In particular, heteroskedastic errors tend to inflate t-statistics and underestimate standard errors. Testing Heteroskedasticity Breusch-Pagan Chi-square Test The Chi-square test is used to test heteroskedasticity in a linear regression model. Trevor Breusch and Adrian Pagan developed the model in 1980. It is a generalization of the White test for heteroskedasticity, which was developed in 1950. The test statistic is based on the regression of the squared residuals from the estimated regression equation on the independent variables. If the presence of conditional heteroskedasticity substantially explains the variation in the squared residuals, then the null hypothesis of no heteroskedasticity is rejected. The test statistic is given by: BP chi-square test statistic = n × R 2 Where: n = Number of observations. R 2 = Coefficient of determination from a regression of the squared residuals on the independent variables. In case the null hypothesis of no heteroskedasticity is true, this statistic will be distributed as a chi-square random variable. We'll have p degrees of freedom, where p is the number of independent variables in the model. The Breusch-pagan test is a one-tailed test since we should mainly be concerned with heteroskedasticity for large test statistic values. The Breusch-Pagan chi-square test can be used for heteroskedasticity in both pooled cross- 45 © 2014-2025 AnalystPrep. sectional and time series data. In pooled cross-sectional data, each observation corresponds to a different individual, such as a different firm in a panel data set. In time series data, each observation corresponds to a different time period, such as a different year. Example: Breusch-Pagan Chi-Square Test Consider the multiple regression of the price of the USDX on inflation and real interest rates. The investor regresses the squared residuals from the original regression on the independent variables. The new R2 is 0.1874. Test for the presence of heteroskedasticity at the 5% significance level. Solution The test statistic is: BP chi-square test statistic = n × R2 = 10 × 0.1874 = 1.874 The one-tailed critical value for a chi-square distribution with two degrees of freedom at the 5% significance level is 5.991. Therefore, we do not have sufficient evidence to reject the null hypothesis of no conditional heteroskedasticity. As a result, we conclude that the error term is not conditionally heteroskedastic. 46 © 2014-2025 AnalystPrep. Correcting Heteroskedasticity In the investment world, it is crucial to correct heteroskedasticity. This is because it may change inferences about a particular hypothesis test, thus impacting an investment decision. One can apply two methods in the correction of heteroskedasticity: calculating robust standard errors and generalized least squares. Computing Robust Standard Errors The first method of correcting conditional heteroskedasticity is to compute robust standard errors. This means that the standard errors of the linear regression model's estimated coefficients are adjusted to account for heteroskedasticity. This method is relatively easy to implement since many statistical software packages can automatically compute robust standard errors. However, it does have some limitations. For example, robust standard errors can lead to conservative estimates of the coefficients—meaning coefficients may be underestimated. Generalized Least Squares The second method of correcting conditional heteroskedasticity is called generalized least squares. This method involves modifying the original equation to eliminate heteroskedasticity. The new, modified regression equation is then estimated under the assumption that heteroskedasticity is no longer a problem. This method is more complex than computing robust standard errors, but it does not have the same limitations. For example, it does not necessarily lead to conservative estimates of the coefficients. 47 © 2014-2025 AnalystPrep. LOS 3(c) Explain Serial Correlation and How It Affects Statistical Inference Serial Correlation (Autocorrelation) Serial correlation, also known as autocorrelation, occurs when the regression residuals are correlated with each other. In other words, it occurs when the errors in the regression are not independent of each other. This can happen for various reasons, including incorrect model specification, not randomly distributed data, and misspecification of the error term. This is common with time-series data. One example of serial correlation is found in stock prices. Stock prices tend to go up and down together over time, which is said to be “serially correlated.” This means that if stock prices go up today, they will also go up tomorrow. Similarly, if stock prices go down today, they are likely to go down tomorrow. The degree of serial correlation can be measured using the autocorrelation coefficient. The autocorrelation coefficient measures how closely related a series of data points are to each other. Types of Serial Correlation Positive Serial Correlation Positive serial correlation occurs when a positive error for one observation increases the chance of a positive error for another observation. In other words, if there is a positive error in one period, there is a greater likelihood of a positive error in the next period as well. Positive serial correlation also means that a negative error for one observation increases the chance of a negative error for another observation. So, if there is a negative error in one period, there is a greater likelihood of a negative error in the next period. Negative Serial Correlation A negative serial correlation occurs when a positive error for one observation increases the 48 © 2014-2025 AnalystPrep. chance of a negative error for another observation. In other words, if there is a positive error in one period, there is a greater likelihood of a negative error in the next period. A negative serial correlation also means that a negative error for one observation increases the chance of a positive error for another observation. So, if there is a negative error in one period, there is a greater likelihood of a positive error in the next period. Effects of Serial Correlation The first thing to understand about serial correlation is that it does not cause bias in the regression coefficient estimates. However, the positive serial correlation will inflate the Fstatistic to test the overall significance of the regression because the mean squared error (MSE) will tend to underestimate the population error variance. Positive serial correlation will also 49 © 2014-2025 AnalystPrep. underestimate standard error thus t-statistic will increase this increases Type I errors (rejecting the null hypothesis when it is true). In other words, you are more likely to reject the null hypothesis when it is true if there is a positive serial correlation in your data. On the other hand, a negative serial correlation will deflate the F-statistic because the MSE will tend to overestimate the population error variance. This decreases Type I errors but increases Type II errors (the failure to reject the null hypothesis when it is false). So, you are more likely to fail to reject the null hypothesis when it is false if there is a negative serial correlation in your data. The positive serial correlation makes the ordinary least squares standard errors for the regression coefficients underestimate the true standard errors. Moreover, it leads to small standard errors of the regression coefficient, making the estimated t-statistics seem statistically significant relative to their actual significance. In addition to affecting significance tests, serial correlation affects confidence intervals and hypothesis tests for individual coefficients. The positive serial correlation makes the OLS standard errors for the regression coefficients underestimate the true standard errors. Moreover, it leads to small standard errors of the regression coefficient, making the estimated t-statistics seem statistically significant relative to their actual significance. Testing for Serial Correlation The first step of testing for serial correlation is plotting the residuals against time. The other most common formal test is the Durbin-Watson test. Durbin-Watson Test The Durbin-Watson test is a statistical test used to determine whether or not there is a serial correlation in a data set. It tests the null hypothesis of no serial correlation against the alternative positive or negative serial correlation hypothesis. The test is named after James Durbin and Geoffrey Watson, who developed it in 1950. 50 © 2014-2025 AnalystPrep. The Durbin-Watson Statistic (DW) is approximated by: DW = 2(1 − r) Where: r is the sample correlation between regression residuals from one period and the previous period. The test statistic can take on values ranging from 0 to 4. A value of 2 indicates no serial correlation, a value between 0 and 2 indicates a positive serial correlation, and a value between 2 and 4 indicates a negative serial correlation: If there is no autocorrelation, the regression errors will be uncorrelated, and thus DW = 2 DW = 2(1 − r) = 2(1 − 0) = 2 For positive serial autocorrelation, DW < 2. For example, if serial correlation of the regression residuals = 1, DW = 2(1 − 1) = 0. For negative autocorrelation, DW > 2. For example, if serial correlation of the regression residual = −1, DW = 2(1 − (−1)) = 4. To reject the null hypothesis of no serial correlation, we need to find a critical value lower than our calculated value of d*. Unfortunately, we cannot know the true critical value, but we can narrow down the range of possible values. Define d l as the lower value and d u as the upper value: If the DW statistic is less than d l, we reject the null hypothesis of no positive serial correlation. If the DW statistic is greater than (4– d l ), we reject the null hypothesis, indicating a significant negative serial correlation. If the DW statistic falls between dl and du , the test results are inconclusive. 51 © 2014-2025 AnalystPrep. If the DW statistic is greater than du , we fail to reject the null hypothesis of no positive serial correlation. Example: The Durbin-Watson Test for Serial Correlation Consider a regression output with two independent variables that generate a DW statistic of 0.654. Assume that the sample size is 15. Test for serial correlation of the error terms at the 5% significance level. Solution From the Durbin-Watson table with n = 15 and k = 2, we see that dl = 0.95 and d u = 1.54. Since d = 0.654 < 0.95 = dl , we reject the null hypothesis and conclude that there is significant positive autocorrelation. Correcting Autocorrelation One way is to adjust the coefficient standard errors for the regression estimates to account for serial correlation. This is done using the Hansen method or the Newey-West estimator. Danish statistician Thorvald Hansen developed the Hansen method in the early 20th century. Researchers have since refined the technique. The Hansen method proposes the estimation of the degree of autocorrelation in the data under review as the first step in autocorrelation 52 © 2014-2025 AnalystPrep. correction. The standard errors are then adjusted accordingly. This correction can be important in fields such as econometrics, where even small inaccuracies can lead to large errors in estimates. While the Hansen method is not perfect, it remains one of the most reliable tools for dealing with autocorrelation. Economists Whitney Newey and Ken West developed the Newey-West estimator. This method is widely used in econometrics and finance. It works by creating a weighting matrix that assigns lower weights to observations more likely to correlate with each other. This technique can be used with both cross-sectional data and time-series data. Another way to correct serial correlation is to modify the regression equation. This can be done by adding a lag term, which represents the value of the dependent variable at a previous period. By including this lag term, we can account for any correlations that may exist between the dependent variable and the error terms. As a result, our estimates will be more accurate, and we will be less likely to make a Type I error. Besides, there are several other methods that can be used to correct for serial correlation, including the use of instrumental variables and panel data methods. 53 © 2014-2025 AnalystPrep. Question Consider a regression model with 80 observations and two independent variables. Assume that the correlation between the error term and the first lagged value of the error term is 0.18. The most appropriate decision is: A. reject the null hypothesis of positive serial correlation. B. fail to reject the null hypothesis of positive serial correlation. C. declare that the test results are inconclusive. Solution The correct answer is C. The test statistic is: DW ≈ 2(1 − r) = 2(1 − 0.18) = 1.64 The critical values from the Durbin Watson table with n = 80 and k = 2 is dl = 1.59 and du = 1.69 . Because 1.69 > 1.64 > 1.59, we determine the test results are inconclusive. 54 © 2014-2025 AnalystPrep. LOS 3(d) Explain Multicollinearity and how it affects regression analysis Multicollinearity occurs when two or more independent variables are significantly correlated to each other. It results from the violation of the multiple regression assumptions that there is no apparent linear relationship between two or more independent variables. Multicollinearity is common with financial data. Effects of Multicollinearity Multicollinearity does not alter the consistency of the regression estimates. However, it renders them imprecise and unreliable. Multicollinearity makes it nearly impossible to determine how the independent variables influence the dependent variable individually. This inflates the standard errors for the regression coefficients, increasing the possibility of Type II errors. Detecting Multicollinearity A high value of R 2 and a significant F-statistic that contradicts the t-test signals multicollinearity. The insignificant t-statistic implies that the standard errors are overestimated. In addition, a high correlation between independent variables indicates multicollinearity. Notably, a low correlation between independent variables does not imply the absence of multicollinearity. Correcting Multicollinearity There are a few methods of correcting multicollinearity: i. Reducing the number of predictor variables in the model by excluding some of them or combining two or more correlated predictors into one. This is often done by conducting feature selection techniques, such as forward selection, backward selection and stepwise regression. 55 © 2014-2025 AnalystPrep. ii. Regularization methods such as ridge regression and lasso regression. These reduce the magnitude of coefficients for predictors that are highly correlated with each other, thus penalizing large coefficient values associated with these predictors and reducing their influence on the model. iii. Decorrelation methods which involve transforming the data so that predictors become uncorrelated. One popular way of doing this is Principal Component Analysis (PCA), where orthogonal components are derived from the original set of variables, with each component having uncorrelated predictors in it. iv. Collinearity diagnostics can also be used to identify pairs of highly correlated variables and then take appropriate action to reduce multicollinearity in those cases. This includes examining Spearman’s correlation coefficients and Variance Inflation Factors (VIF). 56 © 2014-2025 AnalystPrep. Question The regression problem that will most likely increase the chances of making Type II errors is: A. multicollinearity. B. conditional heteroskedasticity. C. positive serial correlation. Solution The correct answer is A. Multicollinearity makes the standard errors of the slope coefficients to be artificially inflated. This increases the likelihood of incorrectly concluding that a variable is not statistically significant (Type II error). B is incorrect. Conditional heteroskedasticity underestimates standard errors, while the coefficient estimates remain unaffected. This inflates the t-statistics, leading to the frequent rejection of the null hypothesis of no statistical significance (Type I error). C is incorrect. Positive serial correlation makes the ordinary least squares standard errors for the regression coefficients to underestimate the true standard errors. This inflates the estimated t-statistics, making them appear to be more significant than they really are. This increases Type I error. 57 © 2014-2025 AnalystPrep. Reading 4: Extensions of Multiple Regression LOS 4(a) Describe Influence analysis and methods of detecting influential data points In statistics, regression analysis is a method of modeling the relationships between a dependent variable (also called an outcome variable) and one or more independent variables (also called predictor variables). Regression analysis aims to find the best-fitting line or curve that describes how the dependent variable changes as the independent variable changes. A crucial part of regression analysis entails identifying the data points which are most influential in determining the shape of the best-fitting line or curve. These data points are known as influential data points. An influential data point has a significant effect on the fitted or predicted values of the model. Influential data points can greatly impact the regression analysis results, even if they are not necessarily representative of the entire dataset. Types of Influential Data Points There are a few different types of data points that can be influential in a multiple regression analysis. These include high-leverage points and outliers. A high-leverage point is a data point with an extreme value of the independent (explanatory) variable. High-leverage points have a relatively large influence on the fitted values of the regression line. This means that if you were to remove a high-leverage point from your dataset, the regression line would change quite a bit. An outlier is a point with a larger or smaller y value than the model suggests. Outliers don't fit well with the rest of the dataset. They can either be positive or negative outliers. Positive outliers are values that are too high, while negative outliers are values that are too low. 58 © 2014-2025 AnalystPrep. Methods of Detecting Influential Data Points There are a number of different methods for detecting influential data points in multiple regression. The two most common methods are leverage and Cook's distance. Leverage Leverage, h ii , is a measure of how far away an individual data point is from the mean of the rest of the data. For a particular independent variable, leverage measures the distance between the ith observation of that variable and the mean of the variable across n observations. Leverage lies between 0 and 1. The sum of individual leverages for all observations is k + 1, where k is the number of independent variables. So, what's the threshold for an observation to be considered an “influential” data point? As a general rule of thumb, an observation is potentially influential if its leverage exceeds 3[ (k+1) n ], 59 © 2014-2025 AnalystPrep. where k is the number of independent variables and n is the number of observations in the data set. For example, if you have a dataset with 50 observations and 10 independent variables, any observation with a leverage greater than 0.66 could be considered potentially influential. A data point with high leverage can significantly impact the overall trend, even if its value is not particularly extreme. Studentized Residual A studentized residual, t∗i , is a statistic that is used to identify outliers in a dataset. It is a residual that has been scaled by an estimate of its standard deviation. It is calculated by dividing the residual (the difference between the actual value and the predicted value) by the standard deviation of the residuals. t∗i = ei∗ n −k− 1 = ei √ ∗ s(e ) SS E (1 − hii ) − e2 i Where: ei∗ = Residual with the i th observation deleted. s∗e = Standard deviation of residuals. k = Number of independent variables. n = Total number of observations. SSE = Sum of square errors of the original regression model. hi i = Leverage value for the ith observation. When interpreting the studentized residual, it is important to remember that a zero value indicates that the data point is not an outlier. Positive values indicate that the data point is above the regression line, while negative values indicate that the data point is below the regression line. Generally speaking, a studentized residual should be considered significant if it is greater than 2 in absolute value. 60 © 2014-2025 AnalystPrep. A more reliable way to determine whether an observation is influential using the studentized residual is to compare it with the critical value of the t-distributed statistic with n − k − 2 degrees of freedom at the selected significance level. The observation is considered influential if the studentized residual is greater than the critical value. Cook's Distance Cook's distance measures the influence of each observation on the estimated regression coefficients. Statistician R. Dennis Cook developed Cook's distance in 1970. Cook's distance measures how much the estimate of the regression coefficients would change if a given observation were removed from a data set. In other words, it quantifies how “important” each observation is in relation to the fit of a model. The larger the value of Cook's distance, the greater the influence of the observation. Cook's distance formula is as follows: e2i h ii hii Di = ( )= ( ) 2 (k + 1) × MSE (1 − h ii ) (k + 1) × MSE (1 − hii )2 (yi − y^i )2 Where: ei = Residual for observation i. K = Number of independent variables. MSE = Mean square error of the estimated regression model. hi i = Leverage value for observation i. Data points with a Cook's distance greater than 1 or √ kn are considered to be influential. This means they can potentially skew the results of the analysis. Data points with a Cook distance between 0.5 and 1 are considered moderately influential. This means they may not significantly impact the results but could still be worth considering. Data points with a Cook distance of less than 0.5 are not considered influential, so they can safely be ignored when analyzing the data. 61 © 2014-2025 AnalystPrep. LOS 4(b) Formulate and interpret a multiple regression model that includes qualitative independent variables Dummy variables are binary variables used to quantify the effect of qualitative independent variables. A dummy variable is assigned a value of 1 if a particular condition is met and, otherwise, a value of 0. The number of dummy variables for n different classes must equal n − 1. The intercept term measures the average value of the dependent variable of the omitted class. On the other hand, the estimated coefficient on each dummy variable measures the average incremental effect of that dummy variable on the dependent variable. Example: Incorporating Dummy Variables in a Multiple Regression Model Adil Suleman, CFA, wishes to identify possible drivers of a company's percentage return on capital (ROC). He identifies performance measures, including margin (%), sales and debt ratios, and demographic measures, such as the region and the economic sector, as possible drivers of ROC. The dummy variable “region” is coded 1 when a company is located in the northern region and 0 if it's in the southern region. On the other hand, the dummy variable “economic sector” is coded 1 when a company belongs to the banking sector and 0 when it belongs to the technology sector. Suleman regresses ROC against sales, debt ratio, profit margin, region, and sector to obtain the following regression output. Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 0.8851 0.7833 0.7263 0.9561 25 62 © 2014-2025 AnalystPrep. Regression Residual Total Df 5 19 24 SS 62.7924 17.3676 80.1600 ANOVA MS 12.5585 0.9141 Coefficients Intercept Sales Debt ratio Profit Margin Region Sector 10.1241 0.0010 0.0166 0.1807 2.1755 −0.8703 F Significance F 13.7389 0.0000 Standard Error 0.8503 0.0004 0.0138 0.0552 0.6061 0.4202 t Stat P-value 11.9069 2.4003 1.2017 3.2713 3.5896 −2.0709 0.0000 0.0268 0.2443 0.0040 0.0020 0.0522 Interpreting the Results From the above results, the multiple regression equation can be expressed as follows: ROC = 10.1241 + 0.001SAL + 0.0166DR + 0.1807P M + 2.1755REG − 0.8703SEC Adjusted R2 72.63% of the variation in the return on capital is explained by three quantitative regressors (sales, debt ratio, and profit margin) and two qualitative regressors (region and sector). Testing the Significance of the Overall Model H 0 : b1 = b2 = b3 = b 4 = b5 = 0 versus at H a : At least one bj ≠ 0 From the ANOVA table, column “Significance F,” notice that the p-value is less than 5%. Thus, reject H 0 in favor of H a . Conclude that the model is statistically significant. Testing the Significance of Dummy Variables The coefficient of the region in this regression model is positive and statistically significant at the 0.05 level since the p-value is less than 0.05. We can, therefore, conclude that the northern 63 © 2014-2025 AnalystPrep. region is significantly different from the southern region at the 5% significance level. On the contrary, the banking sector is not significantly different from the technology sector since the p-value is greater than 0.05. Additionally, the sign of this coefficient is negative. 64 © 2014-2025 AnalystPrep. Question Ninel Khan, a technical trader, is analyzing the seasonality of the price of the GBP/USD. Khan believes the price is significantly different in the first quarter compared to the other three quarters. She uses the first quarter as the reference point in the regression. The number of dummy variables that Khan will need in her regression equation is: A. 2. B. 3. C. 4. Solution The correct answer is B. If we need to differentiate among n categories, the regression should include n − 1 dummy variables. In this case, we have four quarters. Thus, three dummy variables are needed. 65 © 2014-2025 AnalystPrep. LOS 4(c) Formulate and interpret a logistic regression model Qualitative (categorical) dependent variables are dummy variables used as dependent rather than independent variables. Remember that a dummy variable is a variable that takes on the value 0 or 1. The logistic transformation takes the probability that an event happens, p, divided by the probability that the event will not happen, 1 − p . A logit is the natural logarithm of the odds of an event happening. The logistic transformation tends to linearize the relationship between the independent and dependent variables. For example, suppose the probability of a company going bankrupt is 0.6, 0.6 (1−0.6) = 1.5 . In that case, the odds of a company becoming bankrupt are 1.5 times more than the probability of the company not going bankrupt. For this reason, we should use the logit model or discriminant analysis when estimating the probability of bankruptcy. The event probability can be calculated as follows: If p is binary, we use the maximum likelihood method to estimate logistic regression coefficients instead of using least squares. The maximum likelihood method maximizes the likelihood function for the data. The Bernoulli distribution is chosen as the probability distribution because p is binary. The maximum likelihood method is iterative. Each iteration will result in a higher loglikelihood until the difference in the log-likelihood of two successive iterations is the same. At this point, the iterating process will stop. Interpreting Logistic Regression Models In a logit model, the slope coefficient is the change in the logit that the event happens per unit change in the independent variable. The exponent of the slope coefficient is the ratio of odds that the event will happen with a unit increase in the independent variable. For a logit regression, the test of the hypothesis regression coefficient is significantly different from zero; it is the same as that of an ordinary linear regression. The overall performance of a logit regression can be evaluated by examining the likelihood chi-square test statistic. Since logistic regression cannot be fitted using at least-square approach, logistic regression has 66 © 2014-2025 AnalystPrep. no equivalent measure for the coefficient of determination. Researchers have proposed Pseudo − R 2 to compare different specifications of the same model. However, it is unsuitable when comparing models with different datasets. 67 © 2014-2025 AnalystPrep. Question Which of the following measures is least likely to be used in interpreting a logistic regression model? A. R-squared. B. P seudo − R 2 . C. P-value. Solution The correct answer is A. R-squared is not used because logistic regression cannot be fitted using a leastsquare approach. B is incorrect. The Pseudo − R2 is used in logistic regression to compare different specifications of the same model, and it is an alternative for the R-squared. C is incorrect. The p-value is used to evaluate the overall statistical significance of a model. 68 © 2014-2025 AnalystPrep. Reading 5: Time-Series Analysis LOS 5a: Calculate and evaluate the predicted trend value for a time series, modeled as either a linear trend or a log-linear trend, given the estimated trend coefficients A time series shows data on a variable's outcome in different periods—for example, a time series showing EURUSD exchange rates between a given time interval. Time series models explain the past and predict the future of a time series using a trend, which is a long-term pattern in a specific direction. Types of Trends 69 © 2014-2025 AnalystPrep. Linear Trend Models In a linear trend, the dependent variable changes at a constant amount with time. Consider a time series with a linear trend. The value of the time series at time t can be expressed as: y t = b0 + b1 t + ϵ t Where: y t = Value of the time series at time t. b0 = Intercept term. b1 = Trend coefficient. t = Time, the independent variable. t = 1, 2, . . . , T . ϵ t = Random error term. The following shows a linear trend of the EURUSD between 05 January 2021 to 06 February 2021. 70 © 2014-2025 AnalystPrep. Predicted Value of a Linear Trend The predicted value of a time series with a linear trend in period five is expressed as: ŷ5 = b̂0 + b̂1 (5) Out of Sample Prediction The predicted value of a time series for period T + 1 is given by: ŷT +1 = b̂0 + b̂ 1(T + 1) Example: Predicted Value of a Linear trend Given that b0 = 2 and b1 = 1.6, the predicted value of y after four periods is closest to: 71 © 2014-2025 AnalystPrep. y t = b0 + b1 t + ϵ t y4 = 2 + 1.6 × 4 = 8.4 Log-Linear Trend Models A log-linear trend analysis relies on an independent variable that changes exponentially over time or grows at a constant rate. It often works better with financial time series. A time series with exponential growth can be modeled as: yt = eb 0+b1 (t) Where: y t = The value of the dependent variable at time t. b0 = Intercept term. b1 = Constant growth rate. t = time = 1, 2,3, .. . , T . Taking the natural log on both sides of the above equation generates a log-linear model expressed as: lny t = ln (eb 0+b1t ) ln ln y t = b0 + b 1 t The following figure shows different forms of log-linear trends. 72 © 2014-2025 AnalystPrep. Example: Predicted Value of a Log-Linear Trend If b0 = 3 and b1 = 1.5, the predicted value of y after four periods is: ln yt = b 0 + b 1 t + ϵ t ln y 4 = 3 + 1.5 × 4 y4 = e9 = 8103.08 73 © 2014-2025 AnalystPrep. Question Consider a log-linear trend model with an intercept term of 0.5 and a slope coefficient of 0.10. The predicted value of the trend after six periods is closest to: A. 1.1. B. 1.6. C. 3.0. Solution The correct answer is C. yt = eb 0+b1 (t) y t = e0.5+(0.10×6) = 3.0 74 © 2014-2025 AnalystPrep. LOS 5b: Describe factors that determine whether a linear or a log-linear trend should be used with a particular time series and evaluate limitations of trend models A linear trend model should model a time series that increases over time by a constant amount. On the other hand, a time series that grows at a constant rate should be modeled by a log-linear model. To decide between linear and log-linear trend models, one should plot the data. If the data points are equally distributed above and below the regression line, use a linear trend model. On the other hand, if the data points are persistently above or below the trend line, the residuals are serially correlated. This is a violation of one of the key regression model assumptions that residuals are uncorrelated across all observations. For instance, stock prices and sales data are best modeled using log-linear models. Limitations of Trend Models 75 © 2014-2025 AnalystPrep. The error terms may be correlated. This violates the linear regression assumption that residuals are uncorrelated with each other, referred to as serial correlation. Serial correlation makes the linear regression estimates inconsistent and invalid. Serial correlation can be detected by performing a Durbin-Watson test. If the Durbin-Watson statistic is significantly different from 2, then the residual terms are serially correlated. We can correct serial correlation by transforming the data or using other regression models such as an autoregressive model, which will be discussed later in this reading. 76 © 2014-2025 AnalystPrep. Question Neeth Shinu, a portfolio manager, has determined that the price of a certain stock has increased over the last decade, but its growth rate has remained relatively constant. The model that is most appropriate for predicting the future prices of the stock is: A. P t + b 0 + b 1 (t) + ϵ t B. ln(pt ) = b0 + b1 (pt−1 ) + ϵt C. ln(pt ) = b0 + b1 (t) + ϵt Solution The correct answer is C. A log-linear model is most suitable for a time series that grows at a constant growth rate. A is incorrect. It is a linear trend model which is suitable for a time series that increases over time by a constant amount. B is incorrect. This is an autoregressive model, which is appropriate in the presence of serial correlation. 77 © 2014-2025 AnalystPrep. LOS 5c: Explain the requirement for a time series to be covariance stationary and describe the significance of a series that is not stationary A time series is said to be covariance stationary if its properties, such as the mean and variance, remain constant over time. A time series that is nonstationary leads to invalid linear regression estimates with no economic meaning. A time series is covariance stationary if it satisfies the following three conditions: The expected value of the time series is constant and finite over time; The volatility of the time series around its mean is constant and finite in all periods; and The covariance of the time series with leading or lagged values of itself is constant. Covariance stationarity can be detected by plotting the time series. We may assume covariance stationarity if the time series appears to have the same mean and variance, with no apparent seasonality. 78 © 2014-2025 AnalystPrep. Question Which of the following is least likely a condition for covariance stationary property? A. The expected value of the time series is constant and finite overtime. B. The volatility of the time series around its mean varies at a constant rate with time. C. The covariance of the time series with leading or lagged values of itself is constant. Solution The correct answer is B. A time series is covariance stationary if its volatility around its mean is constant and finite in all periods Options A and C are true statements. 79 © 2014-2025 AnalystPrep. LOS 5d: Describe the structure of an autoregressive (AR) model of order p and calculate one- and two period-ahead forecasts given the estimated coefficients The current-time values of a time series are related to the previous time values. This property is termed autoregressive. Autoregressive models are abbreviated (AR p) models. p is known as the order of the model. It indicates the number of lagged values of the dependent variable used. More specifically, a time series that has been regressed on past values is an autoregressive model. In the autoregressive model, we abandon the dependent (y) and independent (x) notion and usex t since there is no longer a difference. A p-order autoregressive model, AR p, is expressed as: xt = b 0 + b1 x t−1 + b2 x t−2 +. . . +bp x t−p + ϵ t Example: Autoregressive Model First-order autoregressive model: AR(1) : x t = b0 + b1 x t−1 + ϵ t Second order autoregressive model: AR(2) : x t = b0 + b1 x t−1 + b 2 xt−2 + ϵt Longer interval differences can be used to account for seasonality: xt = b 0 + b1 x t−1 + b2 x t−4 + ϵ t Chain Rule of Forecasting The chain rule of forecasting can be used to derive multiperiod forecasts using an AR p model. It involves calculating a one-step-ahead forecast before a two-step ahead forecast as the independent variable is a lagged value of the dependent variable. One-Step Ahead Forecast 80 © 2014-2025 AnalystPrep. The one-step-ahead forecast of an AR(1) model is given by: ^ xt =^ b 0 + b̂1 x t−1 Example: Calculating the One-step Ahead Forecast Given an AR(1) model where ^ b 0 = 2 and ^ b 1 = 1.8 , the one-step-ahead forecast of x1 when x0 = 2 is closest to: Solution ^ xt =^ b 0 + b̂1 x t−1 ^ x 1 = 2 + 1.8 × 2 = 5.6 Two-Step Ahead Forecast The two-step ahead forecast for an AR(1) model is determined as: ^ x t+1 = ^ b 0 +^ b 1^ xt Example: Calculating the Two-step Ahead Forecast Calculate the two-step ahead forecast of x 2 x 2 = 2 + 1.8^ ^ x1 x 2 = 2 + 1.8 × 5.6 = 12.08 ^ 81 © 2014-2025 AnalystPrep. Question Consider an AR(1) model with the following prediction equation: xt = 0.8 + 0.5x t−1 If the current value of x is 4.0, the two-step-ahead forecast is closest to: A. 2.20. B. 2.80. C. 1.60. Solution The correct answer is A. One-step ahead forecast: If xt = 4, then ^ x t+1 = 0.8 + 0.5 × 4 = 2.8 Two-step ahead forecast: If ^ x t+1 = 2.8, then ^ x t+2 = 0.8 + 0.5 × 2.8 = 2.2 82 © 2014-2025 AnalystPrep. LOS 5e: Explain how autocorrelations of the residuals can be used to test whether the autoregressive model fits the time series The autocorrelation of a time series refers to the correlation of that time series with its past values. The kth order autocorrelation is the autocorrelation between one time series observation and the value k periods before. We cannot use the Durbin-Watson statistic to test for serial correlation in autoregressive models. This is because the test becomes invalid when the independent variables include past values of the dependent variable. Nevertheless, we can use the t-test to check whether the autocorrelations of the error terms are significantly different from zero. (Recall that serial correlation refers to the positive or negative correlation of the error terms.) An accurately specified autoregressive model will have residual autocorrelations that do not vary significantly from zero. To test whether an AR model is correctly specified, the following steps are followed: Estimate the autoregressive model and calculate the residuals; Calculate the autocorrelations of the residuals; and Perform a t-test to check if the autocorrelations are statistically different from 0. tstatistic = Residual Autocorrelation 1 √T i.e. t= ρϵt , ϵt−k 1 √T Where: 1 √T = Standard Error 83 © 2014-2025 AnalystPrep. T= number of observations ρϵt , ϵt−k = Correlation of the Error Term t with the kth lagged error term. Note: The null hypothesis, H 0 is that there is no autocorrelation. Reject the null hypothesis if tstatistic is greater than the critical value of t at the specified significance level and T-2 degrees of freedom. Rejection of the null hypothesis implies that the model is not correctly specified and should be modified. On the other hand, failure to reject the null hypothesis implies that the model is statistically valid. Example: Testing an Autoregressive Model Consider the following AR(1) model: x t = b 0 + b 1 xt−1 + ϵt The following table shows the autocorrelations of the residuals from the estimation of the model using a sample of 23 observations. Lag Autocorrelation t-statistic 1 0.3834 1.8388 2 0.3483 1.6705 3 0.2897 1.3894 4 −0.1722 −0.8259 5 −0.0725 −0.3477 6 −0.3952 −1.8954 Check whether the model is correctly specified at the 5% level of significance. Standard error = 1 √T = 1 √23 = 0.2085 We can compute the t-statistic for lag one as: Residual Autocorrelation 84 © 2014-2025 AnalystPrep. tstatistic = Residual Autocorrelation 1 √T = 0.3834 = 1.8388 0.2085 The critical two-tail t-value using 21 degrees of freedom at the 5% level of significance is 2.08. Notice that all the absolute values of the t-statistics corresponding to the residual autocorrelations are less than 2.08. This implies that none of them is statistically different from zero. Therefore, we can conclude that the error terms from the AR(1) model are not serially correlated, and thus the model is correctly specified. 85 © 2014-2025 AnalystPrep. Question Consider an AR(1) model relating to 100 observations with residual autocorrelations as presented in the following table: Lag Autocorrelation 1 −0.0584 2 −0.0492 3 0.0625 4 0.1455 At the 5% significance level, the most appropriate statement is that: A. The AR(1) model is correctly specified. B. The AR model is not correctly specified because the autocorrelations of the residuals for lag 1 are statistically different from 0. C. The AR model is not correctly specified because the autocorrelations of the residuals for lag 4 are statistically different from 0. Solution The correct answer is A. Standard error = 1 √T = 1 √ 100 = 0.10 We can compute the t-statistic for lag one as: tstatistic = Residual autocorrelation 1 √T tStatistic = −0.0584 = −0.584 0.10 Repeating this for all the lags gives: 86 © 2014-2025 AnalystPrep. Lag Autocorrelation t-Statistic 1 −0.0584 −0.584 2 −0.0492 −0.492 3 0.0625 0.625 4 0.1455 1.455 The critical two-tail t-value using 98 degrees of freedom at the 5% level of significance is 1.98. Notice that all the absolute values of the t-statistics corresponding to the residual autocorrelations are less than 1.98, which implies that none of them is statistically different from zero. Therefore, we can conclude that the error terms from the AR(1) model are not serially correlated, implying that the model is correctly specified. 87 © 2014-2025 AnalystPrep. LOS 5f: Explain mean reversion and calculate a mean-reverting level Mean reversion refers to the behavior of a time series to fall when its values are above the mean and rise when they are below the mean. This is illustrated as follows: A mean-reverting time series tends to move towards its long-term mean. The model predicts that the value will stay the same if the time series is currently at its long-term mean. i.e., ^ x t = xt−1 Substituting the above in an AR(1) model, we have: x t = b0 + b1 x t−1 = b 0 + b1 x t The mean-reverting level can be expressed as: x t = b 0 + b 1 xt b 88 © 2014-2025 AnalystPrep. xt = b0 1 − b1 The value of an AR(1) model will: Remain the same when x t = Increase when x t < b0 1−b1 b0 1−b1 Decrease when xt > b0 1−b 1 Example: Mean Reversion Consider the following output of an AR(1) model. Coefficients Intercept 31.4 Lag 1 −0.0006827 Standard Error 4.81 0.0001088 The mean-reverting level is closest to: Solution The mean-reverting level can be expressed as: xt = xt = b0 1 − b1 31.4 = 31.39 1 − (−0.0006827) It is worth noting that all covariance stationary time series will have a finite mean-reverting level. This will be discussed in more detail in future learning objectives. 89 © 2014-2025 AnalystPrep. Question Consider the following autoregressive model: xt = 0.0407 − 0.5647x t−1 The mean-reverting level is closest to: A. 0.02601. B. 0.03911. C. 0.09350. Solution The correct answer is A. The mean-reverting level is given by: xt = b0 1 − b1 = 0.0407 = 0.02601 1 − (−0.5647) 90 © 2014-2025 AnalystPrep. LOS 5g: Contrast in-sample and out-of-sample forecasts and compare the forecasting accuracy of different time-series models based on the root mean squared error criterion In-sample Forecasts An in-sample forecast uses the fitted model to derive the predicted values within the period used to estimate model parameters. In-sample forecast errors are residuals generated from a fittedtime series model. For instance, if we use a linear trend to estimate the inflation rate using data from 2004 to 2018, the in-sample errors are the residuals from 2004 to 2018. In sample forecast errors are expressed as: y t − ^ y t where t is an observation within the sample period. Out-of-sample Forecast Errors An out-of-sample forecast uses the estimated model parameters to forecast values outside of the sample period. An out-of-sample forecast error is calculated by taking the difference between the predicted values from outside the sample period and the realized/actual values. For example, suppose we use the same model to predict the inflation rate outside this model's period (2004 to 2018). In this case, the difference between the forecasted and actual inflation rate is an out-of-sample forecast error. Models are selected based on out-of-sample forecasting error. This is because the future is always out-of-sample. A model's accuracy in forecasting out-of-sample values is assessed using the root mean squared error (RMSE). RMSE is the square root of the mean squared error. The model with the smallest RMSE is seen as the most accurate, as it is perceived to have better predictive power in the future. The following steps are used to assess model accuracy using RMSE: Calculate the residuals: This is done by taking the difference between the actual and 91 © 2014-2025 AnalystPrep. the forecasted value; Square the residuals; Sum the squared residuals; Determine the average of the squared residuals; and Take the square root of the average of the squared residuals. Example: Root Mean Squared Error (RMSE) An analyst uses two models to analyze the same time-series data. The mean squared error for the first model, AR(1), is 6.9591, and that of the second model AR(2), is 5.6217. We will use the RMSE criterion to determine the most accurate model in forecasting out-of-sample values. Solution We first calculate the RMSE of both models and choose the model with the smallest RMSE AR(1) : RMSE = √6.9591 = 3.1558 AR(2) : RMSE = √5.6217 = 2.3710 AR(2) has the smallest RMSE. Thus, it is the most accurate in forecasting out-of-sample values. 92 © 2014-2025 AnalystPrep. Question The following table gives the actual and the forecasted change in the log of the price of a certain commodity for four years. The observations after 2016 are out of sample Year 2017 2018 2019 2020 Actual values of Forecast values of changes in prices changes in prices 0.1275 0.1323 0.0336 0.1267 −0.3468 0.1239 −0.093 0.1228 The RMSE for the out-of-sample forecast errors is closest to: A. 6.92%. B. 7.84%. C. 26.31%. Solution The correct answer is C. The RMSE of the out-of-sample forecast errors is 26.31% percent. The out-of-sample error refers to the difference between the realized value and the forecasted value of change in the price for years beyond the estimation period. In this case, the out-ofsample period is 2017 to 2020. The calculations of RMSE are shown in the following table: Year 2017 2018 2019 2020 Actual values of Forecast values of Residuals Residuals2 changes in prices changes in prices 0.1275 0.1323 −0.0048 0.00002 0.0336 0.1267 −0.0931 0.00867 −0.3468 0.1239 −0.4707 0.22156 −0.093 0.1228 −0.2158 0.04657 Mean 0.06920 RMSE 26.31% 93 © 2014-2025 AnalystPrep. LOS 5h: Explain the instability of coefficients of time-series models Time series coefficient estimates can change over time. Regression coefficient estimates derived from an earlier sample period can differ from those approximated using a later period. Therefore, sample period selection is crucial in estimating valuable models. As a result, different models are best suited for different periods. Using only one model for the entire period can result in a poorly fitting model. Similarly, the estimated regression coefficients vary depending on the length of the sample periods. The choice of the model to be used in the analysis depends on the sample period. An AR(1) model may be best suited for a single period, but an AR(2) model may fit a previous or later period better. When the two periods are combined, we are likely to select either the AR(1) or AR(2) model for the combined period. This implies that at least a one-time span of the data will be fitted poorly. Moreover, a longer period increases the risk of mean and variance being unstable over time. Conversely, using a shorter period could produce insufficient data, resulting in lower confidence in estimated parameters. There are no definitive guidelines for determining the appropriate sample period. However, some basic guidelines are recommended, including: Use basic sampling theory: Avoid using two distinctly different populations. Consider basic time-series properties: Do not combine stationary and nonstationary series. Additionally, do not mix series with different mean and variance terms. The length of the sample period: A longer sample period implies that the sample probably comes from different populations. 94 © 2014-2025 AnalystPrep. Question Which of the following statements is most accurate? A. Coefficients of models estimated with longer time series are usually more stable than those with shorter time series. B. Using a shorter period to estimate coefficients results in higher confidence in estimated parameters. C. Coefficients of models estimated with shorter time series are usually more stable than those with longer time series Solution The correct answer is C. Models estimated with shorter time series are usually more stable than those with longer time series. A longer sample period increases the risk of the mean and variance being unstable over time. A is incorrect. Models estimated with longer time series are usually unstable relative to those with shorter time series because a longer sample period increases the likelihood that the underlying economic process has changed. B is incorrect. Using a shorter period could produce insufficient data, resulting in lower confidence in estimated parameters. 95 © 2014-2025 AnalystPrep. LOS 5i: Describe characteristics of random walk processes and contrast them to covariance stationary processes A time series is said to follow a random walk process if the predicted value of the series in one period is equivalent to the value of the series in the previous period plus a random error. A simple random walk process can be expressed as follows: x t = x t−1 + ϵ t Where: x t = Best prediction tomorrow. x t−1 = Best value today. ϵ t = Random error term. Characteristics of Random Walk Time Series An AR(1) time series with β0 = 0 and β1 = 1 is a random walk. This is because the best prediction for tomorrow is the best value today plus a random error term. The expected value of the error term ϵ t is equal to zero. The variance of the residuals is constant. Random walk with drift: A time series follows a random walk with drift if it has a non-zero constant intercept term. It is expressed as: x t = b 0 + b 1 xt−1 + ϵt Where: b0 = Constant drift b1 = 1 96 © 2014-2025 AnalystPrep. Note that a random walk is expressed as: x t = b 0 + b 1 xt−1 + ϵt b0 = 0 for a random walk without drift; b0 ≠ 0 for a random walk with drift; and b1 = 1 for a random walk with or without drift. A random walk has an undefined mean reversion level. If has a mean-reverting level, i.e., x t = b0 + b1 x t, then xt = b0 1−b1 . However, in a random walk, b0 = 0 and b1 = 1, so, x t = 0 1−1 = 0. A random walk is not covariance stationary. The covariance stationary property suggests that the mean and variance terms of a time series remain constant over time. However, the variance of a random walk process does not have an upper bound. As t increases, the variance grows with no upper bound. This implies that we cannot use standard regression analysis on a time series that appears to be a random walk. 97 © 2014-2025 AnalystPrep. Question The most accurate statement about a random walk is that it: A. Has a finite mean-reverting level. B. Has an undefined mean-reverting level. C. Is covariance stationary. Solution The correct answer is B. A random walk process has an undefined mean-reverting level, and thus it is not covariance stationary. 98 © 2014-2025 AnalystPrep. LOS 5j: Describe implications of unit roots for time-series analysis, explain when unit-roots are likely to occur and how to test for them, and demonstrate how a time series with a unit root can be transformed so it can be analyzed with an AR model An AR(1) series is said to be covariance stationary if the absolute value of the lag coefficient b 1 is less than 1. If the absolute value of b1 = 1, the time series is said to have a unit root. All random walks have a unit root since they have b 1 = 1 . This implies that they are not covariance stationary; hence we cannot apply the standard linear regression to test for b 1 = 1. A Dickey-Fuller test can be used to establish if the time series has a unit root. A time series with unit roots should be transformed by first-differencing it to a covariance stationary time series, which can be effectively analyzed using regression analysis. First-differencing is a technique that involves subtracting the dependent variable in the immediately preceding period from the current value of the time series to define a new dependent variable, y . Thus, we model the change in the value of the dependent variable. yt = x t − x t−1 = ϵ t The first-differenced time series can then be modeled as an autoregressive time series. A properly differenced random walk time series is covariance stationary with a mean reversion level of 0. 99 © 2014-2025 AnalystPrep. Question Which of the following is the most appropriate approach to transforming a time series with a unit root problem? A. Performing a Dickey-Fuller test. B. Modeling the first differences of the time series. C. Performing a log-linear transformation. Solution The correct answer is B. A time series with a unit root can be first differenced to transform it into one that is covariance stationary. This is followed by estimating an autoregressive model for the first-differenced series. First differencing involves subtracting the value of the time series in the immediately preceding period from the current value of the series to define a new dependent variable, y . A is incorrect. The Dickey fuller test is used to determine if the time series has a unit root. C is incorrect. A log-linear transformation is appropriate when data grows at a constant rate. 100 © 2014-2025 AnalystPrep. LOS 5k: Describe the steps of the unit root test for nonstationary and explain the relation of the test to autoregressive time-series models Unit root testing involves checking whether the time series is covariance stationary. We can either form an AR model and check for autocorrelations or perform a Dickey and Fuller test. A t-test is performed to examine the statistical significance of autocorrelations at various lags. The autocorrelations of a stationary process will be insignificantly different from zero at all lags or decrease rapidly to zero as the number of lags becomes large. Recall that an AR (1) model is said to be covariance stationary if the absolute value of is less than 1. Additionally, the time series will have a unit root if the lag coefficient equals 1. However, random walks with drift are NOT covariance stationary. Therefore, we cannot use standard linear regression to test for covariance stationary Dickey-Fuller unit root test is therefore used to test for unit roots. Dickey-Fuller Test The Dickey-Fuller test uses the transformed AR(1) model by substracting from both sides of the equation. i.e., x t = b 0 + b 1 xt−1 + ϵt x t − x t−1 = b 0 + b 1 xt−1 − xt−1 + ϵt xt − xt−1 = b0 + xt−1 (b1 − 1) + ϵ t Let the transformed coefficient, b1 − 1 = g. Then, x t − x t−1 = b 0 + g 1 xt−1 + ϵt We then perform a t-test with the null hypothesis H 0 : g1 = 0 (a test of b1 = 1, which implies the time-series has a unit root), against the alternative hypothesis Ha : g1 < 0 (a test of b1 = 1, which implies the time series has no unit root). 101 © 2014-2025 AnalystPrep. If g1 is not significantly different from 0, then b1 = 1, the series must have a unit root. The null hypothesis will be rejected if the time series does not have a unit root. On the other hand, failure to reject the null hypothesis means that the time series has a unit root. A time series with a unit root problem can be modeled by modeling the first differenced series with an autoregressive time series. 102 © 2014-2025 AnalystPrep. Question If the null hypothesis H 0 : g1 = 0 under the Dickey-Fuller test cannot be rejected; the most accurate conclusion is that: A. The time series is covariance stationary. B. The time series does not have a unit root. C. The time series has a unit root. Solution The correct answer is C. If the null hypothesis cannot be rejected, then b 1 − 1 = 0 and b 1 = 1 . This implies that the time series has a unit root. If the null hypothesis is rejected, the time series does not have a unit root. A is incorrect. Rejection of the null hypothesis implies that b 1 − 1 = 0 and so the time series has a unit root. However, a time series with a unit root is not covariance stationary. 103 © 2014-2025 AnalystPrep. LOS 5l: Explain how to test and correct for seasonality in a time-series model and calculate and interpret a forecasted value using an AR model with a seasonal lag Seasonality is a time series feature in which data shows regular and predictable patterns that recur every year. For example, retail sales tend to peak for the Christmas season and then decline after the holidays. A seasonal lag is the value of a time series one period before the current period, incorporated as an additional term in an autoregressive model. For example, if quarterly data are used, the seasonal lag is 4; if monthly data are used, the seasonal lag is 12; and so on. Seasonality is detected by autocorrelation of error terms when it differs significantly from 0. Using seasonal lags eliminates autocorrelation in the error terms. Consider quarterly sales data. The 4th autocorrelation will not be statistically different from zero if there is quarterly seasonality. Seasonality can be corrected by incorporating an additional lagged term. For quarterly data, we add a prior year quarterly seasonal lag as follows: xt = b 0 + b1 x t−1 + b2 x t−4 + ϵ t In the above expression, the seasonal lag is b2 x t−4 . We then run a regression analysis on the time series data to test whether the seasonal lag will eliminate statistically significant autocorrelation of error terms. Example: Testing and Correcting Seasonality Consider an AR(1) model used to forecast quarterly retail sales of a certain company based on 100 observations. x t = b0 + b1 + ϵ t The residual autocorrelations relating to a certain year are as presented in the following table: 104 © 2014-2025 AnalystPrep. Lag Autocorrelation 1 −0.0584 2 −0.0492 3 0.0625 4 0.6580 Test for seasonality in the time series and suggest how to correct it. Solution The first step is to calculate the t-statistic for each autocorrelation using the formula: tStatistic = Residual autocorrelation 1 √T The t-statistic for lag one is calculated as: tStatistic = −0.0584 = −0.584 √ 100 Lag Autocorrelation t-statistic 1 −0.0584 −0.584 2 −0.0492 −0.492 3 0.0625 0.625 4 0.658 6.580 There are 100 observations and two parameters, b0 and b 1 to be estimated. Thus 98 (100-2) degrees of freedom. The critical t-value at the 5% significance level with 98 degrees of freedom is 1.98. The table above shows that the 4th lag autocorrelation and the t-statistic are the highest. The t-statistics of the first three lagged autocorrelations are less than the critical value at the 5% significance level. We can therefore conclude that none of the first three lagged autocorrelations is significantly different from zero. However, notice that the t-statistic for the 4th lag autocorrelation is greater than the critical value. We thus reject the null hypothesis that the 4th lag autocorrelation is zero. The conclusion is that there is seasonality in the time series. This implies that the model is misspecified and not appropriate for use. 105 © 2014-2025 AnalystPrep. Correcting Seasonality Seasonality can be corrected by incorporating an additional lag of the dependent variable that corresponds to the same period in the previous year to the original AR(1) model as another independent variable to make the model more correctly specified. In the previous example, we have seen that there is seasonality in the quarterly time series. This implies that retail sales in each quarter are related to both the previous quarter and the corresponding quarter in the previous year. We can incorporate a seasonality lag to the original AR(1) model as follows: xt = b 0 + b1 x t−1 + b2 x t−4 + ϵ t We have added the seasonal lag b2 x t−4 to eliminate the regular quarterly pattern and seasonal non-stationarity if they exist. Forecasting with Seasonal Lags Assume that the regression coefficients after incorporating the seasonal lag are estimated as: b0 = 0.0080, b1 = −0.0650 and b2 = 0.8068. The estimated equation is: x t = 0.0080 − 0.0650xt−1 + 0.8068xt−4 + ϵt Where x t is the retail sales for the ith quarter. Given the following quarterly retail sales: Quarter 2020.1 2020.2 2020.3 2020.4 Retail sales (USD Millions) 100 300 150 200 We can forecast the retail sales for the first quarter of 2021 as follows: 106 © 2014-2025 AnalystPrep. y 2021.1 = 0.0080 − 0.0650(y 2020.4 ) + 0.8068(y 2020.1 ) = 0.0080 − 0.0650(200) + 0.8068(100) = 67.69 The forecasted value of the retail sales for the first quarter of 2021 is 67.69 million. 107 © 2014-2025 AnalystPrep. Question Consider the following monthly AR model with an additional lag incorporated to eliminate seasonality. xt = b0 + b1 x t−1 + β2 xt−12 + ϵ t Given the following information: Intercept lag 1 lag 12 Coefficients 0.0005 – 0.12 0.87 2019 Jan 2.8 2019 Feb 3.0 2019 Mar 3.5 2019 Apr 4.0 2019 May 4.6 2019 Jun 5.0 2019 Jul 5.4 2019 Aug 6.0 2019 Sep 7.0 2019 Oct 5.4 2019 Nov 6.0 2019 Dec 7.0 The forecasted value for January 2020 is closest to: A. 1.6. B. 3.3. C. 5.8. Solution The correct answer is A. x t = b 0 + b 1 xt−1 + b2 x t−12 + ϵt 108 © 2014-2025 AnalystPrep. x 2020.1 = 0.0005 − 0.12(x2019.12 ) + 0.87(x2019.1 ) = 0.0005 − 0.12 × 7 + 0.87 × 2.8 = 1.6 109 © 2014-2025 AnalystPrep. LOS 5m: Explain autoregressive conditional heteroskedasticity (ARCH) and describe how ARCH models can be applied to predict the variance of a time series Heteroskedasticity is the dependence of the variance of the error term on the independent variable. We have been assuming that time series follows the homoskedasticity assumption. Homoskedasticity is the independence of the variance of the error term on the independent variable. Violation of this condition leads to heteroskedasticity. Heteroskedasticity in the autoregressive model makes the standard errors of the regression coefficients of the model invalid, leading to misleading interpretations. Autoregressive Conditional Heteroskedasticity (ARCH) Models Autoregressive conditional heteroskedasticity is a problem associated with the correlation of variances of the error terms. An ARCH(1) model is an AR(1) model with conditional heteroskedasticity. The error terms in an ARCH(1) model are normally distributed with a mean of 0 and a variance of a0 + a1 ϵ2t−1 . 110 © 2014-2025 AnalystPrep. ϵt ∼ N(0 , a0 + a1 ϵ 2t−1 ) Steps for Testing for ARCH(1) Conditional Heteroskedasticity 1. Regress the squared residuals from each period on the prior period squared residuals. 2. Estimate ^ ϵ 2t = α 0 + α 1 ^ ϵ 2t−1 + μt . 3. If the estimated slope coefficient α ^1 is statistically different from 0, the series shows an ARCH(1) effect, and thus the need to correct for heteroskedasticity. 4. We can then predict the variance of the errors for time t + 1 . Predicting Variance Given an ARCH(1) model, we can use the parameter estimates from our test for conditional heteroskedasticity to predict next period variance as: σ ^2t+1 = α ^0 + α ^1 ^ ϵ 2t Example: Testing for ARCH (1) and Predicting the Variance of Errors The following results have been obtained from regressing the squared residuals of an estimated autoregressive model on a constant and one lag of the squared error terms. Regression Statistic R-squared 0.0430 Standard error 24.638 358 Observations Durbin-Watson 2.1021 Intercept ^ ϵ 2t−1 Coefficient Standard Error t-Statistic 9.1680 1.4104 6.55002 0.2050 0.0506 4.0514 a. Test whether the original time series has ARCH(1) errors at the 5% significance level. b. If the squared error in the current period is 0.5%, predicted variance in the next period is closest to: 111 © 2014-2025 AnalystPrep. Solution The test involves the first lag of the residuals of the approximated time series model, and thus the number of the observations is less by one. The number of degrees of freedom is 358 − 1 = 357, and the corresponding critical t-value at the 5% level of significance is 1.97. The t-statistic on ^ ^1 is statistically ϵ 2t−1 is greater than the critical value. This implies that α significantly different from 0. Therefore, the time series has ARCH(1) errors. The predicted variance for the next period is determined using the formula: ^2t = α ^0 + α ^1^ σ ϵ 2t−1 ^2t = 9.1680 + 0.2050^ σ ϵ 2t−1 So that the variance for the next period, in this case, is: ^2t = 9.1680 + 0.2050 × 0.005 = 9.169025 σ 112 © 2014-2025 AnalystPrep. Question Consider the following regression results for an ARCH(1) model: Constant Lag 1 Coefficients P-value 4.8347 < 0.001 0.2678 < 0.001 Given that the current period squared error is 0.678, the predicted variance of the error terms in the next period is closest to: A. 4.835. B. 5.103. C. 5.016. Solution The correct answer is C. The predicted variance for the next period is determined using the formula: ^2t = α ^0 + α ^1^ σ ϵ 2t−1 ^2t = 4.8347 + 0.2678 × 0.678 = 5.016 σ 113 © 2014-2025 AnalystPrep. LOS 5n: Explain how time-series variables should be analyzed for nonstationary and/or cointegration before use in linear regression Consider a time series of the inflation rate (y t) regressed on a time series of interest rates (x t) : y t = b 0 + b 1 xt + ϵt In this case, we have two different time series, y t and x t . Either one of the time series is subject to non-stationarity. Recall that we test for non-stationarity using the Dickey-fuller test for each time series to check for unit roots. Outcomes of the Dickey-Fuller test The results of a Dickey-Fuller test can either be: a. Both the time series are covariance stationary. This means that the time series do not have a unit root. In this case, we use linear regression to test the relationship between the two series. Besides, the regression coefficients are statistically reliable. b. Only the dependent variable time series is covariance stationary. This means that some of the linear regression assumptions are violated. Therefore, the estimated regression coefficients and the standard error would be inconsistent. c. Only the independent variable time series is covariance stationary. This has a similar implication to outcome two above. d. None of the time series is covariance stationary If none of the time series is covariance stationary, check whether or not they are cointegrated. 114 © 2014-2025 AnalystPrep. Two time-series are said to be cointegrated if an economic or financial relationship exists between them, preventing them from diverging without bound in the long run. We can test for cointegration using either the Engle-Granger or Dickey-Fuller test. The error term will be stationary, and the hypothesis tests will be valid if both the time series are covariance stationary and cointegrated. Testing for Cointegration We can test whether the two time series are cointegrated through the following steps: Regress one variable on the other variable using the model: y t = b0 + b1 x t + ϵ t Test the errors for a unit root using the Dickey-Fuller test. The critical t-values are determined using the Engle and Granger test. Rejection of the null hypothesis implies that the error terms are covariance stationary, and both series are cointegrated. Recall that the null hypothesis is that the error term has a unit root. Use regression to model the relationship between the two-time series if they are cointegrated. If the (Engler-Granger) Dickey-Fuller test fails to reject the null hypothesis, we conclude that the error term is not covariance stationary. This means that the two series are not cointegrated. Consequently, the linear regression is invalid. 115 © 2014-2025 AnalystPrep. Question An analyst tests the two time series errors for a unit root using the Dickey-Fuller test and determines the critical values using the Engle and Granger test. The test fails to reject the null hypothesis. The most accurate conclusion is that: A. The two series are cointegrated. B. The two series are not cointegrated. C. The error terms are covariance stationary. Solution The correct answer is B. If the (Engler-Granger) Dickey-Fuller test fails to reject the null hypothesis, we conclude that the error terms are not covariance stationary. This means that the two series are not cointegrated. Consequently, a linear regression cannot be used. 116 © 2014-2025 AnalystPrep. LOS 5o: Determine an appropriate time-series model to analyze a given investment problem and justify that choice The following guidelines are used to determine the most appropriate model depending on the need: 1. Understand the investment problem. This is followed by choosing the initial model. 2. Plot the time series to check for covariance stationarity. Observe if there is seasonality, a significant shift, a linear trend, or an exponential trend. Any trend in the model signals that the time series is covariance stationary. 3. In case of a linear or an exponential trend, take the following steps: Plot the time series to determine if the trend is linear or exponential: If the plot is a straight line with an upward or downward slope, use a linear trend model. On the other hand, if the plot is a curve, use a log-linear trend model. Estimate the trend. Calculate the residuals. Test for serial correlation in the residuals using the Durbin-Watson test. If there is no significant serial correlation of the residuals, then the model is suitable for forecasting. 4. On the other hand, serial correlation in the residuals calls for a more complicated model such as an autoregressive model. If the data has serial correlation, recheck it for covariance stationary before running the AR model. Transform the nonstationary data in the following ways: First difference the time series if the data has a linear trend. Take the natural log of the data with an exponential trend before first differencing. Estimate different time-series models before and after the shift in the instance the time series shifts significantly during the sample period. 117 © 2014-2025 AnalystPrep. Include seasonal lag for data with significant seasonality. 5. Model the transformed time series with an autoregressive model, AR (1) model. If there are serial correlations in the residual of the AR (1) model, then try an AR (2) model. Seasonal lags are added until there are no serial correlations. 6. Test if residuals have autoregressive conditional heteroskedasticity (ARCH). Regress the squared residuals from each period on the prior period squared residuals. Test whether the resulting coefficient is significantly different from 0. If the slope coefficient is statistically different from 0, the series shows an ARCH (1) effect and the need to correct it for heteroskedasticity. This can be done by the use of generalized least squares. The model is suitable for use if the slope coefficient is not significantly different from zero. 7. Test the forecasting performance of the out-of-sample forecasting performance and compare this to the in-sample performance of the RMSE. 118 © 2014-2025 AnalystPrep. Question Consider an AR(1) model used to forecast quarterly retail sales of a certain company based on 200 observations. x t = b0 + b1 + ϵ t The residual autocorrelations relating to a certain year are as presented in the following table: Lag Autocorrelation 1 −0.0832 2 −0.0701 3 0.0891 4 0.9379 The most appropriate statement regarding the AR(1) model at the 5% significance level is that: A. the AR(1) model is sufficient. B. the AR(1) model is misspecified and not appropriate for use. C. the AR (1) model's error terms are not serially correlated, and thus the model is correctly specified. Solution The correct answer is B. To check whether the model is correctly specified at the 5% level of significance, Standard error = 1 √T = 1 √200 = 0.0707 We can compute the t-statistic for lag one as: Residual Autocorrelation 119 © 2014-2025 AnalystPrep. tStatistic = Residual Autocorrelation 1 √T tStatistic = −0.0832 = −1.1768 0.0707 Calculating the t-statistic for each lag, we obtain: Lag Autocorrelation Autocorrelation T-statistic 1 −0.0832394 −0.0832 −1.1768 2 −0.0701264 −0.0701 −0.9915 3 0.08908331 0.0891 1.26025 4 0.93786911 0.9379 13.2659 There are 200 observations and two parameters, b0 and b 1 to be estimated. Thus 198 (200-2) degrees of freedom. The critical t-value at the 5% significance level with 198 degrees of freedom is 1.97. The t-statistics of the first three lagged autocorrelations are less than the critical value at the 5% significance level. We can therefore conclude that none of the first three lagged autocorrelations is significantly different from zero. However, notice that the t-statistic for the 4th lag autocorrelation is greater than the critical value. We thus reject the null hypothesis that the 4th lag autocorrelation is zero. The conclusion is that there is seasonality in the time series. This implies that the AR(1) model's error terms are serially correlated, and thus the model is misspecified and not appropriate for use. 120 © 2014-2025 AnalystPrep. Reading 6: Big Data Projects LOS 6a: Identify and explain steps in a data analysis project The term “big data” refers to structured or unstructured data that is significant, fast, or complex, thus strenuous or even impossible to process using traditional methods. The incorporation of big data has prompt implications for building a machine learning model and financial forecasting. Structured Data The steps for building a machine learning model using structured (traditional) big data are as follows: Step 1: Conceptualization of the modeling task. This step requires establishing the model output, how and who will use the model, and how the output will be planted in existing or new business. Step 2: Data collection. Structured data can be obtained from both internal and external sources. Step 3: Data preparation and wrangling. Cleansing and preprocessing of raw data takes place in this step. Cleansing involves resolving missing values or outliers, whereas preprocessing entails extracting, accumulating, filtering, and selecting relevant data columns. Step 4: Data exploration. The data exploration step involves exploratory data analysis, selecting, and engineering features. Step 5: Model training. The suitable machine learning approach is selected in this step. Moreover, the performance of the trained model is evaluated, and the model is tuned accordingly. 121 © 2014-2025 AnalystPrep. These steps are repetitive since model building is an iterative process. Unstructured Data Building text ML models requires different steps since they incorporate unstructured data sources of big data. The differences in steps between the text model and traditional model account for the characteristics of big data: volume, velocity, variety, and veracity. Here, we focus on the variety and veracity attributes of big data as they are apparent in the text. Text ML model building steps are as follows: Step 1: Text problem formulation. This step entails the formulate text classification problem and identifying the exact inputs and outputs for the model. Further, it is crucial to establish how the text ML model’s classification output will be put to use. Step 2: Text curation. Relevant text data is obtained from web pages through scraping or crawling programs. Step 3: Text preparation and wrangling. This step entails cleansing and preprocessing tasks required for converting unstructured data into a format that is usable by traditional modeling methods designed for structured inputs. Step 4: Text exploration. Here, text is visualized through methods such as word clouds. Additionally, it involves text feature selection and engineering Finally, the output can either be combined with other structured variables or used directly for forecasting and analysis. The following figure demonstrates the above steps of building a model for financial forecasting using structured (traditional) vs. unstructured (text) big data. 122 © 2014-2025 AnalystPrep. 123 © 2014-2025 AnalystPrep. Question Consider the following statements: Statement 1: “The second step in building text-based machine learning models is text preparation and wrangling. However, the second step in building machine learning models using structured data is data collection." Statement 2: “The fourth step in building both types of models encompasses data/text exploration.” Which of the statements relating to the steps in building structured data-based and text-based machine learning models is (are) accurate? A. Only Statement 1. B. Only Statement 2. C. Both Statement 1 and Statement 2. Solution The correct answer is B. The five steps in building structured data-based machine learning models are: 1. Conceptualization of the modeling task 2. Data collection 3. Data preparation and wrangling 4. Data exploration 5. Model training. The five steps in building text-based ML models are: 1. Text problem formulation 2. Data (text) curation 124 © 2014-2025 AnalystPrep. 3. Text preparation and wrangling 4. Text exploration 5. Model training. Statement 2 is correct since the fourth step in building both types of models involves data/text exploration. A and C are incorrect. Text preparation and wrangling is the third step in building text ML models and occurs after the second data (text) curation step. 125 © 2014-2025 AnalystPrep. LOS 6b: Describe objectives, steps, and examples of preparing and wrangling data Data preparation and wrangling is a crucial step that entails cleaning and organizing raw data in a consolidated format that allows for more convenient consumption of the data. Data collection precedes the data preparation and wrangling stage. Recall that before data collection begins, it is essential to state the problem, define the objectives, identify useful data points, and conceptualize the model. What follows is collecting the relevant data through exploring and downloading raw data from different sources. Preliminary to delving into data preparation and wrangling, it is vital to differentiate the two forms of data that can be collected, i.e., structured and unstructured data. The figure below gives the different features of the two data forms. 126 © 2014-2025 AnalystPrep. The data preparation and wrangling stage involves two crucial tasks: cleansing and preprocessing, respectively. Data Preparation (Cleansing): This refers to the process of inspecting, pinpointing, and reducing errors in raw data. Raw data can be invalid, inaccurate, incomplete, or have duplicates either due to mistakes during manual data entry or server failures or system bugs for data recorded by the system. Data Wrangling (Preprocessing): After data is cleaned, it needs to be processed by managing the outliers, extracting handy variables from the existing data points, and scaling the features of the data. This prepares the data for model consumption. 127 © 2014-2025 AnalystPrep. Structured Data Data Preparation (Cleansing) To establish possible errors associated with structured data, we look at the following sample of raw data obtained from a credit company. 1 2 3 4 5 6 7 8 ID 1 2 3 4 5 6 7 Name Ms. Phi Mr. Psi Ms. Tau Mr. Epsilon Mr. Rho Mr. Rho Ms. Chi Gender F M F F M M F Salary ($) 120, 000 100, 000 (40, 000) 90, 000 83, 000 83, 000 95, 000 Loan Amount ($) Loan Outcome 48, 000 No default Unknown No default 16, 000 No default 36, 000 Defaulted 33, 200 No default 33, 200 No default 38, 000 No default Loan Type Car Student loan Mortgage Car Mortgage Mortgage 1. Incompleteness error: This is missing data due to the absence of some data entries. The data shown for Mr. Rho shows an incompleteness error because the loan type is missing. Missing values should be omitted or replaced with NA. The NA can then be substituted with options such as mean, median, or mode, or 0. 2. Invalidity error: Arises when some data values are out of a meaningful range. The data shown for Mrs. Tau contains invalidity error since salary cannot be negative. Invalid entries should be verified with other data records. 3. Inaccuracy error: Occurs when the data is not a measure of actual value. The data for Mr. Psi indicates an unknown loan amount, yet the loan outcome shows that he did not default. The lender must know how much he lends this particular borrower. 4. Inconsistency error: This is as a result of some of the data conflicting with the corresponding data points or reality. The data for Mr. Epsilon is likely to be inconsistent as the Name column contains a male title, but the Gender column contains a female. Clarifying the data with another source solves this error. 5. Non-uniformity error: Emerges where the data are ambiguous or non-identical data formats. For example, the monetary unit for the salary and loan amount is $. This ambiguous because the dollar symbol can represent the US dollar, Canadian dollar, or others. Converting 128 © 2014-2025 AnalystPrep. the data points into a preferable standard format can resolve this error. 6. Duplication error: This is as a result of the repetition of identical data points. The data shown for Mr. Rho contains duplication error. This error can be resolved by removing the duplicates. Data Wrangling (Preprocessing) What follows after data cleaning is data preprocessing, which involves transforming and scaling data. These transformations may include: 1. Extraction: This entails extracting a new variable from the current variable to simplify the analysis and use it for the ML model training. 2. Aggregation: Two or more variables can be combined to form one variable to consolidate similar variables. 3. Filtration: Data rows not necessary for the analysis must be pinpointed and filtered. 4. Selection: The data columns that are intuitively not required for the analysis can be eliminated. For example, the Name column, in this case, is not required for training the ML model. 5. Conversion: The different variable types in the dataset must be converted into appropriate types to process further and analyze them correctly. For example, Name and Loan Type are nominal, Salary and Loan Amount are continuous, and Gender and Loan Outcome are categorical with 2 classes. The next step is to identify outliers present the data. For normally distributed data, a data value outside of 3 standard deviations from the mean may be considered an outlier. An interquartile range (IQR) can also be used to identify outliers. Data values outside 1.5 IQR are considered outliers, while those outside 3IQR are extreme values. Trimming/Truncation refers to removing extreme values and outliers from the data set. For example, a 10% trimmed dataset is one for which the 10% highest and the 10% lowest values have been eliminated. On the other hand, winsorization refers to replacing extreme values and 129 © 2014-2025 AnalystPrep. outliers with the maximum and the minimum values of data points that are not outliers. Feature scaling entails making sure that features are on a similar scale, shifting, and changing the scale of data. Scaling follows after eliminating the outliers. Scaling Techniques 1. Normalization is the process of adjusting one or more attributes to the range of 0 to 1. It is sensitive to outliers and can be used when the distribution of the data is unknown. To normalize a random variable X: Example 1: Xi(normalized) = Xi − XM in XM ax − XM in 2. Standardization typically means adjusting data to have a mean of 0 and a standard deviation of 1 (i.e., unit variance). It is reasonably less sensitive to outliers as it depends on the mean and standard deviation of the data. Standardization applies to data that has a normal distribution. Equation 2: Xi(Stand ardized) = Xi − μ σ Where: μ = The mean of the variable for each observation Xi ; and σ = The standard deviation of the feature X . Unstructured (Text) Data The objective of data preparation and wrangling of textual data is to transform the unstructured data into structured data. The output of these processes is a document term matrix that can be read by computers. The document term matrix is similar to a data table for structured data. The 130 © 2014-2025 AnalystPrep. cleansing and preprocessing of unstructured text data into a structured format is called text processing. Preparing unstructured data is more challenging relative to structured data. We will use text data related to the English language to demonstrate this section. Text Preparation/Cleansing This step involves removing unnecessary HTML tags, punctuation, and white spaces from the raw text. The cleansing process is as follows: Step1: Remove HTML tags. HTML tags that are not part of the actual text can be removed using a programming language or a regular expression (regex). Step 2: Remove punctuations. Some punctuations, such as percentage signs and question marks, may be useful for ML model training. Therefore, when such punctuation is removed, annotations such as /percentSign/ and /questionMark/ should be added to maintain their grammatical meaning in the text. Regex is commonly applied to remove or replace punctuations. Step 3: Remove numbers. Numbers present in the text should be removed or replaced by annotations such as /number/. This is critical because the computers treat each number as a separate word, complicating the analyses or adding noise. Step 4: Remove white spaces. Extra spaces such as tabs, line breaks, and new lines should be identified and removed to keep the text intact and clean. The stripWhitespace function in R can be utilized to can be used to eliminate unnecessary white spaces from the text. Text Wrangling (Preprocessing) We begin this section by defining token and tokenization to understand text processing further. A 131 © 2014-2025 AnalystPrep. token corresponds to a word, while tokenization is the process of breaking down a given text into separate tokens. For example, “This is good” has 3 tokens, i.e., “This,” “is,” and “good.” Text data also require normalization, just like structured data. The normalization process in text processing entails the following: Step 1: Lowercasing the alphabet aids the computer to process identical words appropriately. Step 2: Stop words such as “the,” “for,” and “are,” usually are removed to reduce the number of tokens involved in the training set for ML training purposes. Step 3: Stemming is the process of converting words from their base forms or stems using crude Heuristic rules. For example, the stem of the words “increased” and “increasing” is “increas.” Stemming solves the problem that emerges when some words appear very infrequently in a textual dataset posing the risk of training highly complex models. Step 4: Lemmatization is identical to stemming except that it removes endings only if the base form is present in a dictionary. Lemmatization is much more costly and advanced relative to stemming. What follows after text normalization is creating a bag-of-words (BOW). A BOW is a representation for analyzing text. It does not, however, represent the word sequences or positions. Suppose we have a series of sentences: “This is good.” “This is valuable.” “This is fine.” The following figure is a Bag-of-Words representation of the three sentences before and after the normalization process. 132 © 2014-2025 AnalystPrep. 133 © 2014-2025 AnalystPrep. The final Bow after normalizing is then used to build a document term matrix (DTM). It is a matrix where each row belongs to a text file, and each column represents a token. The number of rows is equivalent to the number of text files in a sample text dataset. The number of columns is equal to the number of tokens from the BOW built using all the text files in the same token is present in each document. The following figure shows a DTM constructed from the resultant BOW of the three sentences. DTM of Three Sentences and Using Normalized BOW Filled with Counts of Occurrence Good Sentence 1 1 Sentence 2 0 Sentence 3 0 Valu 0 1 0 Fine 0 0 1 As mentioned earlier, a BOW does not represent the word sequences or positions, which limits its use for some advanced ML training applications. For example, if a text has the word “no.” It can be treated as a single token and treated as a stop word during normalization. This would fail to imply negative meaning. N-gram is a technique used to overcome such a problem as it is a representation of word sequences. A two-word sequence is a bigram, a three-word sequence is a trigram, etc. 134 © 2014-2025 AnalystPrep. Question A data scientist of a large investment corporation is discussing with her senior manager about the steps involved in preprocessing raw text data. She tells her senior manager that the process can be accomplished in the following three steps: Step 1: Cleanse the raw text data. Step 2: Split the cleansed data into a collection of words for them to be normalized. Step 3: Normalize the collection of words and create a well-defined set of tokens from the normalized words. The data scientist’s step 2 is most likely to be: A. Lemmatization. B. Standardization. C. Tokenization. Solution The correct answer is C. Tokenization refers to the process of dividing a given text into separate tokens. This step takes place after cleansing the raw text data, i.e., removing HTML tags, numbers, and extra white spaces. The tokens are then normalized to create the bagof-words (BOW). A is incorrect. Lemmatization is a text normalization technique of reducing inflected words while ensuring that the root word belongs to the language. In Lemmatization, root word is called Lemma. A lemma is the dictionary form or citation form of a set of words. For example, the lemma of the words “analyzed” and “analyzing” is “analyze.” B is incorrect. Standardization is a scaling technique that entails adjusting data to 135 © 2014-2025 AnalystPrep. have a mean (μ) of 0 and a standard deviation (σ) of 1 (i.e., unit variance). 136 © 2014-2025 AnalystPrep. LOS 6c: Describe objectives, methods, and examples of data exploration The main objective of data exploration is to investigate and comprehend data distributions and relationships. Data exploration involves three critical tasks: exploratory data analysis, feature selection, and feature engineering. Exploratory Data Analysis Exploratory Data Analysis (EDA) is the first step in exploring data. The data is summarized and observed using exploratory graphs, charts, and other visualizations. The main objective of EDA is to act as a communication medium among project stakeholders and analysts. Additionally, EDA is intended to aid in: understanding data properties, finding patterns and relationships in data, inspecting basic questions and hypotheses, documenting data distributions, and planning modeling strategies for the next steps. Definitions Feature selection is a process of selecting only features that contribute most to the prediction variable or output from the dataset for ML model training. Selecting fewer features decreases ML model complexity and training time. Feature engineering is a process of generating new features by transforming existing features. Model performance depends heavily on feature selection and engineering. Structured Data 1. Exploratory Data Analysis For structured data, EDA can either be performed on one-dimension (a single feature) or multidimension (multiple features). Whereas histograms, bar charts, bar plots, and density plots are one dimension, visualizations, line graphs, and scatterplots visualize multi-dimension data. Additionally, descriptive statistics such as central tendency measures, minimum and maximum 137 © 2014-2025 AnalystPrep. values for continuous data are useful to summarize data. Counts and frequencies for categorical data can be used to gain insight into the distribution of possible values. 2. Feature Selection Features of structured data are represented by different columns of data in a table or matrix. The objective of the feature selection process is to assist in identifying significant features that, when used in a model, retain the essential patterns and complexities of the larger dataset. Further, these features should require less data overall. Feature selection methods are utilized to rank all features. If the target variable of interest is discrete, such techniques as chi-square test, correlation coefficient, and information gain would be applicable. These are univariate techniques that score feature variables individually. Both dimensionality reduction and feature selection seek to reduce the number of features in a data set. The dimensionality reduction method generates new combinations of features that do not correlate. However, feature selection includes and excludes features present in the data without altering them. 3. Feature Engineering Feature engineering is the process of optimizing and improving the features of the data further. Feature engineering techniques methodically modify, decompose, or combine existing features to produce more significant features. More essential features allow an ML model to train more rapidly and efficiently. This process depends on the context of the project, domain of the data, and nature of the problem. An existing feature can be engineered to a new feature or decomposed to multiple features. Categorical variables can be converted into a binary form (0 or 1) for machine-reading, a process called one-hot encoding. For example, if a single categorical feature represents gender identities with four possible values—male, female, transgender, gender-neutral—then these values can be decomposed into four new features, one for each possible value (e.g., is_male, is_female) filled with 0s (for false) and 1s (for true). 138 © 2014-2025 AnalystPrep. Unstructured Data 1. Exploratory Data Analysis Text data incorporates a collection of texts (also known as a corpus) that are sequences of tokens. It is useful to perform EDA of text data by calculating basic text statistics on the tokens. These may include term frequency (TF), which is the ratio of the number of times a given token occurs in all the texts in the dataset to the total number of tokens. Some examples of basic text statistics include word associations, average word and sentence length, and word and syllable counts. These statistics reveal patterns in the co-occurrence of words. Text modeling involves identifying the words which are most informative in a text by computing the term frequency of each word. The words with high term frequency values are removed since they are likely to be the stop words, making the resulting bag-of-words more compact. The chisquare measure of word association applies to sentiment analysis and text classification application to aid in understanding the significant word appearances in negative and positive sentences in the text. Similar to structured data, bar charts, and word clouds can be used to visualize text data. Word clouds can be made to visualize the most informative words and their term frequency values. Varying font sizes can show the most commonly occurring words. Further, color is used to add more dimensions, such as frequency and length of words. 2. Feature Selection Feature selection for text data entails selecting a subset of tokens in a data set to effectively reduce the bag-of-words size, making the ML model more efficient and less complicated. Feature selection eliminates noisy features from the dataset. The popular feature selection methods in text data are as follows: Frequency measures are used for vocabulary pruning to eliminate noise features. The tokens with very high and low term frequency values are filtered across all the texts. Noise features can be stopwords that usually occur repeatedly in all the texts across 139 © 2014-2025 AnalystPrep. the dataset. On the other end, noise features can be rare terms that are present in only a few text files. Document frequency (DF) is equivalent to the number of documents (texts) that contain the respective token divided by the total number of documents. It helps to discard the noise features that carry no specific information about the text class and are present across all texts. Chi-square test tests the independence of two events: occurrence of the token and occurrence of the class. Tokens with the highest chi-square test statistic values are selected as features for ML model training because of their higher discriminatory potential. Mutual information (MI) gauges the amount of information contributed by a token to a class of texts. The mutual information value is equivalent to 0 if the token’s distribution in all text classes is the same. Otherwise, the MI value approaches 1 as the token in any one class tends to occur more frequently in only that particular class of text. 3. Feature Engineering Similar to structured data, financial engineering is a fundamental step that dramatically improves ML model training. Some techniques for feature engineering include: Numbers: Different numbers are converted into different tokens. For example, 5-digit numbers can be replaced with “/number5/,” 10-digit numbers with “/number10/,” and so forth. N-grams: These refer to n consecutive words. Multi-word patterns that are discriminative can be identified, and their association kept intact. For example, when referring to “stock market,” which refers to an economic context, a bigram would be applicable as it treats the two adjacent words as a single token, i.e., stock_market. Name entity recognition (NER): This is an algorithm that takes individual tokens as inputs and pinpoints the relevant nouns such as a person, location, and organization. For example, the NER tag for the tokens, “CFA,” and “institute” is an 140 © 2014-2025 AnalystPrep. “ORGANIZATION.” Parts of speech (POS): Similar to NER, parts of speech uses language structure and dictionaries to tag every token in the text with a corresponding part of speech. For example, the POS tag for both of the tokens, “CFA,” and “institute” is “NNP,” which refers to a proper noun. POS tags can be useful for separating verbs and nouns for text analytics. Note that the fundamental objective of feature engineering maintaining the semantic essence of the text while simplifying and converting it into structured data for ML. 141 © 2014-2025 AnalystPrep. Question Amelia Parker is a junior analyst at ABC Investment Ltd. Parker is building an ML model that has an improved predictive power. She plans to improve the existing model that purely relies on structured financial data by incorporating finance-related text data derived from news articles and tweets relating to the company. After preparing and wrangling the raw text data, Parker performs exploratory data analysis. She creates and analyzes a visualization that shows the most informative words in the dataset based on their term frequency (TF) values to assist in feature selection. However, she is concerned that some tokens are noise features for ML model training; therefore, she wants to remove them. To address her concern in the exploratory data analysis, Parker is most likely to focus on those tokens that have: A. Low chi-square statistics. B. Very low and very high term frequency (TF) values. C. Low mutual information (ML) value. Solution The correct answer is B. Frequency measures are used for vocabulary pruning to eliminate noise features. The tokens with very high and low TF values are filtered across all the texts. Noise features are both the most recurrent and most rare tokens in the dataset. On one end, noise features can be stop words that are typically present frequently in all the texts across the dataset. On the other end, noise features can be sparse terms that are present in only a few text files. Recurring tokens strain the ML model to choose a decision boundary 142 © 2014-2025 AnalystPrep. among the texts during text classification as the terms are present across all the texts. The sparse tokens mislead the ML model into classifying texts containing the rare terms into a specific class, a case of overfitting. Thus, pinpointing and eliminating noise features are essential steps for feature selection procedures. A is incorrect. Chi-square test tests the independence of two events: occurrence of the token and occurrence of the class. Tokens with the highest chi-square test statistic values are selected as features for ML model training because of their due to higher discriminatory potential. C is incorrect. Mutual information (MI) gauges the amount of information contributed by a token to a class of texts. The mutual information value is equivalent to 0 if the token’s distribution in all text classes is the same. Otherwise, the MI value approaches 1 as the token in any one class tends to occur more frequently in only that particular class of text. 143 © 2014-2025 AnalystPrep. LOS 6d: Describe objectives, steps, and techniques in model training Machine learning (ML) model training entails three tasks: method selection, performance evaluation, and tuning. While there are no standard rules for training an ML model, having a fundamental understanding of domain-specific training data and ML algorithm principles is key to proper model training. Recall that in the previous stages, we have transformed unstructured text data into forms of data matrixes or tables for machine training. Therefore, the ML model training for structured and unstructured data is typically similar. In ML model training, all we are doing is fitting a system of rules on a training dataset to establish data patterns. Model training steps are as follows: 1. Method Selection The first step of the training process is selecting and applying an algorithm to an ML model. The following factors determine method selection: 1. Supervised or unsupervised learning: Supervised ML models use labeled training data to produce a known outcome. Typical methods of choice for supervised learning include: regression, ensemble trees, support vector machines (SVMs), and neural networks (NNs). On the contrary, unsupervised ML is reasonably challenging due to the absence of a target variable. Standard methods that apply here include dimensionality reduction, clustering, and anomaly detection. 2. Type of data: Classification and regression tree (CART) methods are appropriate for numerical data. For text data, GLMs and SVMs are commonly used, whereas NNs and deep learning methods are suitable for image data. Finally, when working with speech data, deep learning methods can generate great results. 3. Size of data: A standard dataset is characterized by the number of observations and the number of features. SVMs work well for broader data sets with 10,000 to 100,000 features but fewer observations. On the other hand, NNs often work better on “longer” datasets, where the observations are much larger than the number of features. 144 © 2014-2025 AnalystPrep. It is vital to note that more than one method can be used to exploit the advantages of each method. Additionally, before model training begins, the master data set is split into three subsets in case of supervised learning. The figure below demonstrates a possible way of splitting the data. 145 © 2014-2025 AnalystPrep. For unsupervised learning, no splitting is needed due to the absence of labeled training data. In some cases, the number of observations for a particular class may be significantly larger than for other classes. This is known as class imbalance, and it is linked to data used in supervised learning. Resampling the data set solves this problem. Undersampling and oversampling approaches apply to balancing the classes. Undersampling works by reducing the majority class (“0”). This method is applicable when the quantity of data is adequate. By keeping all samples in the minority class (“1”) and randomly selecting an equal number of samples in the majority class, a balanced new dataset can be retrieved for further modeling. Oversampling, on the contrary, is used when the quantity of data is not sufficient. Its goal is to balance the dataset by increasing the size of the minority samples. Instead of getting rid of majority samples, new minority samples are generated using, say repetition, and bootstrapping. The following figure shows the idea of balancing the majority and the minority classes. 146 © 2014-2025 AnalystPrep. 2. Performance Evaluation Performance evaluation of the model training is essential for validation of the model. In this section, we discuss the various techniques appropriate for measuring model performance. For simplicity, we will mostly deal with binary classification models. a. Error Analysis Error analysis entails four basic evaluation metrics: true positive (TP), false positive (FP), true negative (TN), and false-negative (FN) metrics. FP is also known as Type I error, whereas FN is also called a Type II error. The confusion matrix below gives a better visualization of these metrics: 147 © 2014-2025 AnalystPrep. The explanation of the terms in the confusion matrix is as follows: True Positives (TP): This is the case when the actual class and the predicted class of data point are both 1. True Negatives (TN): It occurs when the actual and predicted classes of the data point are both 0. False Positives (FP): It occurs when the actual class of data point is 0, whereas the predicted class of data point is 1. Also known as Type I error. False Negatives (FN): This occurs when the actual class of data point is 1, whereas the predicted class of data point is 0. Also known as Type II error. Precision 148 © 2014-2025 AnalystPrep. Precision (P) is the proportion of correctly predicted positive classes to total predicted positive classes. Precision is useful in situations where the cost of false positives (FP), or Type I error is high. Precision (P) = TP TP + FP Recall/Sensitivity Recall (sensitivity) is the proportion of correctly predicted positive classes to the total actual positive classes. It is useful where the cost of FN or Type II error is high. Recall (R) = TP TP + FN Accuracy Accuracy is the percentage of correctly predicted classes out of total predictions. It is a standard metric for evaluating model performance. However, it is not a clear indicator of the performance and can generate worse results, especially if the classes are imbalanced. Accuracy = TP + TN TP + FP + TN + FN F1 Score FI score is the harmonic mean of precision and recall. F1 score is more appropriate than accuracy when there is a class imbalance in the dataset, and it is necessary to measure the equilibrium of precision and recall. The higher the F1 score, the better the model performance. F1 score = (2 × Precision × Recall) Precision + Recall b. Receiver Operating Characteristic (ROC) ROC involves plotting a curve showing the trade-off between the false positive rate (x-axis) and true positive rate (y-axis) for various cutoff points where: FP 149 © 2014-2025 AnalystPrep. False positive rate (FPR) = FP TN + FP True positive rate (TPR) = TP TP + FN And, As TPR increases, FPR also increases. Furthermore, a more convex curve indicates better model performance. AUC is equivalent to the area under the ROC curve. The higher the numerical value of AUC, the better the prediction. The following figure demonstrates all you need to know about ROC curves and AUCs: c. Root Mean Squared Error (RMSE) RMSE is a single metric that captures all the prediction errors in the data (n). It is appropriate for continuous data prediction and is mostly used for regression methods. 150 © 2014-2025 AnalystPrep. n RMSE = ∑ ⎷ i=1 (Predictedi − Actuali ) n A small RMSE indicates potentially better model performance. 3. Tuning The main objective of tuning is to improve model performance. Model-fitting has two types of error: bias and variance. Bias error emerges when the model is underfitted, whereas variance error is associated with overfitting. Both errors can be reduced, so the total aggregate error (bias error + variance error) is at a minimum. The bias-variance trade-off is vital to finding an optimum balance where a model neither underfits nor overfits. Tuning is usually a trial-and-error process. It entails a continuous change of some hyperparameters. The algorithm is then run on the data again, and the output is compared to its performance using the validation set. This determines the set of hyperparameters, which results in the most accurate model. This method is known as a grid search. A fitting curve of the training sample error and cross-validation sample error on the y-axis versus model complexity on the x-axis is useful for managing the bias vs. variance error trade-off. Fitting curves visually aid in tuning hyperparameters. The following figure shows a bias-variance error trade-off by plotting a generic fitting curve for hyperparameter: 151 © 2014-2025 AnalystPrep. If there exists a high bias or variance even after the tuning of hyperparameters, a large number of training observations (instances) may be needed. Additionally, the number of features included in the model may need to be reduced in the case of high variance or increased in the case of high bias. The model then needs to be re-trained and re-tuned using the new training dataset. Ceiling analysis is employed in the case of a complex model, i.e., where a large model is comprised of sub-model(s). Ceiling analysis is a systematic process of assessing different components in the pipeline of model building. It assists in understanding the part of the pipeline that can potentially improve in performance by further tuning. 152 © 2014-2025 AnalystPrep. Question 153 © 2014-2025 AnalystPrep. Amelia Parker is a junior analyst at ABC Investment Ltd. Parker is building a model that has an improved predictive power. She plans to improve the existing model that purely relies on structured financial data by incorporating finance-related text data derived from news articles and tweets relating to the company. After preparing and wrangling the raw text data, Parker performs exploratory data analysis. She creates and analyzes a visualization that shows the most informative words in the dataset based on their term frequency (TF) values to assist in feature selection. Once satisfied with the final set of features, Parker selects and runs a model on the training set that classifies the text as having positive sentiment (Class “1”) or negative sentiment (Class “0”). She then evaluates its performance using error analysis. The resulting confusion matrix is as follows: Confusion Matrix Actual Training Results Class "1" Class "0" Predicted Results Class "1" TP=200 FP=70 TN=98 Class "0" FN=40 Based on the above confusion matrix, the models F1 score and accuracy metric are most likely to be: A. 74% and 83%, respectively. B. 74% and 78%, respectively. C. 78% and 73%, respectively. Solution The correct answer is C. The model’s F1 score, which is the harmonic mean of precision and recall, is computed as: (2 × Precision × Recall) 154 © 2014-2025 AnalystPrep. F1 score = (2 × Precision × Recall) Precision + Recall Precision (P) = TP TP + FP 200 200 + 70 = 74% Precision (P) = Recall (R) = TP TP + FN 200 200 + 40 = 83% Recall (R) = F1 score = (2 × Precision × Recall) Precision + Recall F1 score = (2 × 74 × 83) 74 + 83 78% = The model’s accuracy, which is the proportion of correctly predicted classes out of total predictions, is solved as follows: Accuracy = TP + TN TP + FP + TN + FN 200 + 98 200 + 70 + 98 + 40 = 73% Accuracy = 155 © 2014-2025 AnalystPrep. LOS 6e: Describe preparing, wrangling, and exploring text-based data for financial forecasting Sentiment analysis refers to the analysis of opinions or emotions from text data. In other words, it refers to how positive, negative, or neutral a particular phrase or statement is regarding a “target.” Such sentiment can provide critical predictive power for forecasting stock price movements for companies. Here, we delve into a financial forecasting project to examine how effectively text data sentiments, from financial and economic news sources, can be classified. This project uses sentences from a text file Sentences_50Agree labeled as positive, neutral, or negative sentiment class. The following table shows a sample data table corpus made up of 6 sentences from the Sentences_50Agree text file. A corpus is a collection of text data in any form, including lists, matrix, or data table forms. Sentence Sentiment Cramo slipped to a pretax loss of EUR 6.7 million from a pretax profit of EUR 58.9 million. Negative Profit before taxes decreased to EUR 31.6 mn from EUR 50.0 mn the year before. Negative Also, construction expenses have gone up in Russia. Negative Positive Finnish Bore that is owned by the Rettig family, has grown recently through the acquisition of smaller shipping companies. Positive The plan is estimated to generate some EUR 5 million (USD 6.5 m) in cost savings on an annual basis. Earnings per share EPS are seen at EUR 0.56 Positive Text Preparation/Cleansing Whenever the data is obtained from different sources, it is gathered in raw format, which is not feasible for analysis. The first step of converting the raw data into a proper format is data cleansing. Text preparation entails removing, or incorporating appropriate substitutions for, possible extraneous information present in the text. In this case, we will eliminate punctuations, numbers, and white spaces that are not necessary for model training as follows: 156 © 2014-2025 AnalystPrep. Step1: Remove HTML tags. There are no HTML tags present in this sample. Step 2: Remove punctuations. Percentage and dollar symbols are substituted with word annotations to retain their importance in the textual data. Further, periods, semi-colons, and commas are removed. Regex is commonly applied to remove or replace punctuations. Step3: Remove numbers. Numbers present in the text should be removed as they do not have significant use for sentiment analysis. It is crucial to note that sentiment analysis seeks to understand the context in which the numbers are used. Additionally, before removing numbers, abbreviations representing orders of magnitude, such as million (commonly represented by “m,” “mln,” or “mn”), billion, or trillion, should be replaced with the complete word. Step 4: Remove white spaces Extra spaces such as tabs, line breaks, and new lines should be identified and removed to keep the text intact and clean. The stripWhitespace function in R can be utilized to can be used to eliminate unnecessary white spaces from the text The cleansed data is free of punctuations and numbers, with useful substitutions, as shown in the table below: Sentence Cramo slipped to a pretax loss of a EUR million from a pretax profit of EUR million Profit before taxes decreased to EUR million from EUR million the year before Also, construction expenses have gone up in Russia Finnish Bore that is owned by the Rettig family has grown recently through the acquisition of smaller shipping companies The plan is estimated to generate some EUR million USD million in cost savings on an annual basis Earnings per share EPS are seen at EUR Sentiment Negative Negative Negative Positive Positive Positive Text Wrangling 157 © 2014-2025 AnalystPrep. After textual data is cleansed, it should be normalized. The normalization process in text processing entails the following: Step 1: Lowercasing the alphabet aids the computer to process identical words appropriately. For example, “AND,” “And,” and “and”). Step 2: Stop words such as “the,” “for,” and “are,” usually are removed to reduce the number of tokens involved in the training set for ML training purposes. However, some stop words such as not, more, very, and few are not eliminated as they carry significant meaning in the financial texts that are useful for sentiment prediction. Step 3: Stemming a process of linguistic normalization, which reduces words to their word root word. For example, connection, connected, connecting word reduce to a common word "connect." Step 4: Lemmatization is identical to stemming except that it removes endings only if the base form is present in a dictionary. Lemmatization is much more costly and advanced relative to stemming. Step 5: Tokenization is the process of breaking down a text paragraph into smaller chunks, such as words. A token is a single entity that is a building block for a sentence or a paragraph. Data Exploration Exploratory Data Analysis Exploratory data analysis (EDA) performed on text data provides insights on word distribution in the text. These word counts can be used to examine words that are most commonly and least commonly present in the texts, i.e., outliers. The graph below is a frequency distribution of the sample tokens before stop words are eliminated. 158 © 2014-2025 AnalystPrep. From the figure, we observe that “currencysign,” and “million” are the most repeated words due to the financial nature of the data. A word cloud is a data visualization approach used for representing text data. Word clouds can be made to visualize the most informative words and their term frequency (TF) values. Varying font sizes can show the most commonly occurring words. Further, color is used to add more dimensions, such as frequency and length of words. The feature selection process will eliminate common words and highlight useful words for better 159 © 2014-2025 AnalystPrep. model training. 160 © 2014-2025 AnalystPrep. Question Which of the following visualizations is most likely to be appropriate in the exploratory data analysis step if our objective is to create a visualization that shows the most informative words in the dataset based on their term frequency (TF) values? A. Scatter plot. B. Word cloud. C. Document term matrix. Solution The correct answer is B. A word cloud is a common visualization when working with text data as it can be made to visualize the most informative words and their TF values. Varying font sizes can show the most commonly occurring words in the dataset, and the color is used to add more dimensions, such as frequency and length of words. A is incorrect. A scatter plot is a two-dimensional chart that can be employed to summarize and approximately measure relationships between two or more features. C is incorrect. A document term matrix (DTM) is a matrix where each row belongs to a text file, and each column represents a token. The number of rows is equivalent to the number of text files in a sample text dataset. The number of columns is equal to the number of tokens from the BOW built using all the text files in the same token is present in each document. A DTM is not a visualization tool. 161 © 2014-2025 AnalystPrep. LOS 6f: Describe methods for extracting, selecting and engineering features from textual data Feature Extraction Feature extraction entails mapping the textual data to real-valued vectors. After the text has been normalized, the next step is to create a bag-of-words (BOW). It is a representation of analyzing text. It does not, however, represent the word sequences or positions. The following figure represents a BOW of the textual data extracted from the Sentences_50Agree file for the first 233 words: Feature Selection Feature selection for text data entails keeping the useful tokens in the BOW that are informative and eliminating different classes of texts, i.e., those with positive sentiment and those with negative sentiment. The process of quantifying how important tokens are in a sentence and the corpus as a whole is known as frequency analysis. It assists in filtering unnecessary tokens (or features). Term frequency (TF) is calculated and examined to pinpoint noisy terms, i.e., outlier terms. TF at the corpus level—also known as collection frequency (CF)—is the number of times a given word appears in the whole corpus divided by the total number of words in the corpus. Terms with low TF are mostly rare terms, for example, proper nouns or sparse terms, which rarely appear in the data and do not contribute to the differentiating sentiment. On the other hand, terms with high TF are mostly stop words, present in most sentences, so do not contribute to the differentiating sentiment. These terms are eliminated before forming the final document term matrix (DTM). The next step involves constructing the DTM for ML training. Different TF measures are 162 © 2014-2025 AnalystPrep. calculated to fill in the cells of the DTM as follows 1. SentenceNo: This is a unique identification number given to each sentence in the order they appear in the original dataset. For example, sentence number 11 is a sentence in row 11 from the data table. 2. TotalWordsInSentence: It is the count of the total number of words present in the sentence. 3. Word: This is a word token that is present in the corresponding sentence. 4. TotalWordCount: This is the total number of occurrences of the word in the entire corpus or collection. TF (Collection level) = TotalWordCount Total number of words in collection 5. WordCountInSentence: This refers to the number of times the token is present in the corresponding sentence. For example, in the sentence, “The profit before taxes decreased to EUR 31.6 million from EUR 50.0 million the year before.” The token “the” is present two times. 6. SentenceCountWithWord: This is the number of sentences in which the word is present. 7. Term Frequency (TF) at Sentence Level: This is the proportion of the number of times a word is present in a sentence to the total number of words in that sentence. TF at Sentence Level = WordCountInSentence TotalWordsInSentence 8. Document frequency (DF): This is calculated as the number of sentences that contain a given word divided by the total number of sentences. DF is essential since words frequently occurring across sentences provide no differentiating information in each sentence. DF = SentenceCountWithWord Total number of sentences 9. Inverse Document Frequency (IDF): This is a relative measure of how unique a term is across the entire corpus. A low IDF implies a high word frequency in the text 1 163 © 2014-2025 AnalystPrep. IDF = log( 1 ) DF 10. TF–IDF: TF at the sentence level is multiplied by the IDF of a word across the entire dataset to get a complete representation of the value of each word. High TF–IDF values indicate words that appear more frequently within a smaller number of documents. This signifies relatively more unique terms that are crucial. The converse is true. TF–IDF values can serve as word feature values for training an ML model TF − IDF = TF × IDF Feature Engineering Up to now, we have been dealing with single words or tokens, which, combined with the bag of words model, implied the assumption of independent words. One way of minimizing the effect of this simplification by adding some context to the words is to use n-grams. N-grams helps us to understand the sentiment of a sentence as a whole. An n-gram refers to a set of n consecutive words that can be used as the building blocks of an ML model. Notice that we have been using the n-gram model for the particular case of n equals one, which is also called a unigram (for n=2, the n-gram model is a bigram, for n=3 trigram). When dealing with n-grams, unique tokens to denote the beginning and end of a sentence are sometimes used. Bigram tokens help keep negations intact in the text, which is crucial for sentiment prediction. The corresponding word frequency measures for the DTM are then computed based on the new BOW formed from n-grams. The following figure shows a sample of 3-gram tokens for the dataset sentences_50Agree for the first 233 words. 164 © 2014-2025 AnalystPrep. Example: Calculating and Interpreting Term Frequency Measures Peter Smith is a data scientist and wants to use the textual data from sentences_50Agree file to develop sentiment indicators for forecasting future stock price movements. Smith has assembled a BOW from the corpus of text being examined and has pulled the following abbreviated term frequency measures tables. The text file has a total of 112,622 non-unique tokens that are in 4,820 sentences. Term Frequency Measures Table 1 Sentence No < int > 7 4 22 22 4792 TotalWordsIn Word TotalWord WordCountIn SentenceCount Sentence Count Sentence WithWord < int > < chr > < int > < int > < int > 43 a 1745 3 1373 34 the 6068 5 3678 43 3213 3 2120 of 43 the 6068 4 3678 28 a 1745 3 1373 Term Frequency Measures Table 2 Word SentenceNo TotalWordsIn TotalWord WordCountIn SentenceCount WithWord Sentence Count Sentence < int > < int > < chr > < int > < int > < int > 4 34 increase 103 3 100 647 27 decrease 11 1 11 792 12 great 6 1 6 4508 47 drop 7 1 7 33 36 rise 17 1 17 1. Determine and interpret term frequency (TF) at the collection level and at the sentence level for the token “the” in sentence 4 in term frequency measures Table 1. 2. Determine and interpret term frequency (TF) at the collection level, and the sentence 165 © 2014-2025 AnalystPrep. level for the token “decrease” in sentence 647 in term frequency measures Table 2. 3. Determine and interpret TF–IDF (term frequency-inverse document frequency) for the token “the” in sentence 4 in term frequency measures Table 1. Solution to 1: Total Word Count Total number of words in collection 6 , 068 = 112, 622 5.39% = TF (Collection Level) = TF at the collection level is an indicator of the frequency that a token is used throughout the whole collection of texts (here, 112,622). It is vital for identifying outlier words: Tokens with the highest TF values are mostly stop words that do not contribute to differentiating the sentiment embedded in the text (such as “the”). Word Count In Sentence Total Words In Sentence 5 = 34 = 0.1471 or 14.71% TF at Sentence Level = TF at the sentence level is an indicator of the frequency that a token is used in a particular sentence. Therefore, it is useful for understanding the importance of the specific token in a given sentence. Solution to 2: For the token “decrease,” the term frequency (TF) at the collection level is given by: = 11 = 0.0098% 112, 622 This token has a meager TF value. Recall that tokens with the lowest TF values are mostly proper nouns or sparse terms that are also not important to the meaning of the text. 1 166 © 2014-2025 AnalystPrep. 1 27 = 0.037 or 3.70% TF at Sentence Level for the token "decrease" = TF at the sentence level is an indicator of the frequency that a word is used in a particular sentence. Therefore, it is useful for understanding the importance of the specific token in a given sentence. Solution to 3: To calculate TF–IDF, document frequency (DF) and inverse document frequency (IDF) also need to be calculated. DF = Sentence Count with Word Total number of sentences For token “the” in sentence 4, 3, 678 4, 820 0.7631or76.31% = DF = Document frequency is important since tokens frequently occurring across sentences (such as “the”) provide no differentiating information in each sentence. IDF is a relative measure of how important a term is across the entire corpus: IDF = log( 1 ) DF For token “the” in sentence 4: 1 ) 0.7631 = 0.1174 IDF = log( Using TF and IDF, TF–IDF can now be calculated as: TF − IDF = TF × IDF 167 © 2014-2025 AnalystPrep. For token "the" in sentence 4, T F − I DF = 0.1471 × 0.1174 = 0.0173 or 1.73 As TF–IDF combines TF at the sentence level with IDF across the entire corpus, it provides a complete representation of the value of each word. A low TF–IDF value indicates tokens that appear in most of the sentences and are not discriminative (such as “the”). TF–IDF values are useful in extracting the key terms in a document for use as features for training an ML model. 168 © 2014-2025 AnalystPrep. Question The TF–IDF (term frequency-inverse document frequency) for the token “decrease” in sentence 647 in term frequency measures Table 2 is most likely: A. 19.64%. B. 20.58%. C. 9.80%. Solution The correct answer is C. To calculate TF–IDF, document frequency (DF), and inverse document frequency (IDF) also need to be calculated. DF = Sentence Count With Word Total number of sentences For token "decrease" in sentence 647, 11 4, 820 = 0.00228or0.228% DF = Document frequency is important since tokens frequently occurring across sentences (such as “the”) provide no differentiating information in each sentence. IDF is a relative measure of how important a term is across the entire corpus. IDF = log( 1 ) DF For token "decrease" in sentence 647: 1 169 © 2014-2025 AnalystPrep. 1 ) 0.00228 = 2.642 IDF = log( Using TF and IDF, TF-IDF can now be calculated as: TF − IDF = TF × IDF For token "the" in sentence 647, TF − IDF = 0.037 × 2.642 = 0.098, or9.8% As TF–IDF combines TF at the sentence level with IDF across the entire corpus, it provides a complete representation of the value of each word. A high TF–IDF value indicates the word appears many times within a small number of documents, signifying an important yet unique term within a sentence (such as “decrease”). 170 © 2014-2025 AnalystPrep. LOS 6g: Evaluate the fit of a machine learning algorithm Model Training Suppose that the target variable (y) for the ML training model has the sentiment class labels (positive and negative). To ease the calculation of the performance metrics, we relabel them as 1 (for positive) and 0 (for negative). The performance metrics that can be employed here are receiver operating characteristic (ROC) curve and area under the curve (AUC) from the trained model results. The cleansed and preprocessed data set is split into three separate sets: training set, crossvalidation (CV) set, and test set in the ratio 60:20:20, respectively. The final document term matrix (DTM) is built using the sentences (rows), which are the instances, and resulting tokens (columns), which are the feature variables, from the BOW of the training dataset. The columns of DTMs for the splits are the same, equal to the number of unique tokens (i.e., features) from the final training corpus BOW. 171 © 2014-2025 AnalystPrep. A summary of the three data splits is shown in the following table: Corpus Split( %) Master 100% Training 60% 20% Cross-validation Test 20% Use Data exploration ML Model training Turning and validating the trained model Testing the trained, tuned, and validated model Method Selection Sentiment analysis involves the use of textual data, which is more extensive with many possible variables and a known output, i.e., positive, neutral, or negative sentiment classes. A supervised ML algorithm, therefore, applies to sentiment analysis. ML algorithms such as SVM, decision trees, and logistic regression can also be examined in this case. For simplicity, we will assume that logistic regression is used to train the model. In this case, we assume that texts are the sentences, and the classifications are either positive or negative sentiment classes (labeled 1 and 0, respectively). The tokens are feature variables, and the sentiment class is the target variable. Logistic regression utilizes the method of maximum likelihood estimation. As a result, the output of the logistic model is a probability value ranging from 0 to 1. A mathematical function uses the logistic regression coefficient (β) to calculate the probability (p) of sentences having positive sentiment (y = 1). For example, if the p-value for a sentence is 0.90, there is a 90% probability that the sentence has a positive sentiment. We aim to find an ideal threshold value of p, which is a cutoff point for p values, and it depends on the dataset and model training. When the p values are above the ideal threshold p-value, the sentences have a high probability of having a positive sentiment (y = 1). The ideal threshold p-value is estimated heuristically using performance metrics and ROC curves. Performance Evaluation and Tuning 172 © 2014-2025 AnalystPrep. This step involves predicting the sentiments of the sentences in the training and cross-validation (CV) DTMs using the trained ML model. ROC curves are plotted for model evaluation. Recall that for ROC curves, the x-axis is a false positive rate, FP , TN +F P and the y-axis is a true positive rate, TP . T P+F N The following figure shows sample ROC Curves of model results for Training and cross-validation data: The ROC curves are significantly different between the training and the CV datasets, i.e., the AUC is 96.5% on training data and 86.2% on CV data. This finding suggests that the model performs relatively poorly on the CV data when compared to training data, implying that the model is overfitted. Regularization approaches such as least absolute shrinkage and selection operator (LASSO) are applied to the logistic regression model to mitigate the risk of overfitting. LASSO penalizes the coefficients of the model to prevent overfitting of the model. The penalized model only selects 173 © 2014-2025 AnalystPrep. the tokens with statistically non-zero coefficients, and that contributes to the model fit. The ROC curves are similar for model performance on both datasets, with an AUC of 95.7% on the training dataset and 94.8% on the CV dataset. This implies that the model performs almost identically on both training and CV data and thus indicates an excellent fitting model. Error analysis can be used to evaluate the model further. It is performed by computing a confusion matrix using the ML model results from the CV dataset. The threshold p-value of 0.5 is the theoretically suggested threshold (cutoff). When p > 0.5, the prediction is assumed to be y = 1 (positive sentiment). Confusion Matrix for CV Data with threshold p-value of 0.5 Actual Training Labels Class “1.” Predicted Results Class “1” TP = 256 Class “0” FN = 6 Class “0.” FP = 27 TN = 103 174 © 2014-2025 AnalystPrep. Performance Metrics TP TP + FP 256 = = 0.9046 256 + 27 Precision (P) = TP TP + FN 256 = = 0.9771 256 + 6 Recall (R) = F1 score = (2 × Precision × Recall) Precision + Recall 2 × 0.9046 × 0.9771 = 0.9046 + 0.9771 = 0.9394 T P + TN TP + FP + TN + FN 256 + 103 = 256 + 27 + 6 + 103 = 0.9158 Accuracy = In this case, the model accuracy is 91.58%, with a theoretically suggested threshold p-value of 0.5. It is a standard practice to simulate many model results with different threshold values and to look for maximized accuracy and F1 statistics that minimize trade-offs between FP (Type I error) and FN (Type II error). What follows is to identify the threshold p-value that generates the highest accuracy and F1 score. For example, the following figure shows the F1 score and accuracy performance measures for various threshold p values. The threshold p-value that results in the highest accuracy and F1 score can be identified as 0.6. 175 © 2014-2025 AnalystPrep. Confusion Matrix for CV Data with threshold p-value of 0.6 Actual Training Labels Class “1.” Class “0.” Predicted Results Class “1” TP = 256 FP = 19 FN 6 TN = 100 Class “0” = Performance Metrics 256 256 + 19 = 0.9309 Precision (P) = 256 256 + 6 = 0.9771 Recall (R) = 2 × 0.9309 × 0.9771 0.9309 + 0.9771 = 0.9534 F1 score = 256 + 100 176 © 2014-2025 AnalystPrep. 256 + 100 256 + 19 + 6 + 100 = 0.9344 Accuracy = The model performance metrics improved in the final model relative to the earlier case when the threshold p-value was 0.5. With a threshold p-value of 0.6, the precision, accuracy, and F1 score have increased. The precision has increased by approximately 3% to 93%, while the accuracy and F1 score both increasing by approximately 1% to 93% and 95%, respectively. Results and Interpretation The final ML model, with the appropriate threshold p-value, has been validated and is now ready for application. It can be used to predict the sentiment of new sentences from the test data corpus as well as new sentences from almost identical financial text data sources. The model is now applied to the test data. Note that the test data has neither been used to train the model nor validate it. The confusion matrix for the test data using a threshold p-value of 0.6 is as shown: Confusion Matrix for Test Data with Threshold p-value of 0.6 Actual Training Labels Class “1.” Class “0.” Predicted Results Class “1” TP = 256 FP = 24 TN = 107 Class “0” FN = 6 Performance Metrics 256 256 + 24 = 0.9413 Precision (P) = Recall (R) = 256 == 0.9771 256 + 6 2 × 0.9413 × 0.9771 0.9413 + 0.9771 = 0.9313 F1 score = 256 + 107 177 © 2014-2025 AnalystPrep. 256 + 107 256 + 24 + 6 + 107 = 0.9237 Accuracy = From the confusion matrix for the test data, we get accuracy and F1 score of 92% and 93%, respectively, with precision and recall of 94% and 98%, respectively. Therefore, it is clear that the model performs similarly on the training, CV, and test datasets. Additionally, this implies that the model is robust and is not overfitting. It also suggests that the model generalizes well out-ofsample and can thus be used to predict the sentiment classes for new sentences from similar financial text data sources. The following figure recaps the entire section of classifying and predicting sentiments using textual data from financial data sources. 178 © 2014-2025 AnalystPrep. The derived sentiment classification can be used to visualize insights regarding a large text without reading the entire document. 179 © 2014-2025 AnalystPrep. Question In the previous analysis using the cross-validation dataset, the threshold value of 0.60 was determined to be the p-value that maximizes model accuracy and F1 score; the confusion matrix for this model is shown in Table A below: Table A: Confusion Matrix for CV Data with Threshold p-value of 0.6 Actual Training Labels Class “1.” Class “0.” Predicted Results Class “1” TP = 256 FP = 19 TN = 100 Class “0” FN = 6 You are given the following confusion matrix of the same model with threshold p values of 0.4, as shown in Table B below: Table B: Confusion Matrix for CV Data with Threshold p-value of 0.4 Actual Training Labels Class “1.” Class “0.” Predicted Results Class “1” TP = 256 FP = 30 TN = 106 Class “0” FN = 6 When the performance metrics of Table B (using a threshold p-value of 0.4) are compared with those of Table A (using a threshold p-value of 0.60), Table B is more likely to have a: A. Lower accuracy and a lower F1 score relative to Table A. B. Higher accuracy and a higher F1 score relative to Table A. C. Lower accuracy and the same F1 score relative to Table A. Solution The correct answer is A. 180 © 2014-2025 AnalystPrep. Performance metrics for confusion matrix A with threshold p-value of 0.6: 256 256 + 19 = 0.9309 Precision (P) = 256 256 + 6 = 0.9771 Recall (R) = 2 × 0.9309 × 0.9771 0.9309 + 0.9771 = 0.9534 F1 Score = 256 + 100 256 + 19 + 6 + 100 = 0.9344 Accuracy = Performance metrics for confusion matrix B with a threshold p-value of 0.4: 256 256 + 30 = 0.8951 Precision (P) = 256 256 + 6 = 0.9771 Recall (R) = 2 × 0.8951 × 0.9771 0.8951 + 0.9771 = 0.9343 F1 Score = 256 + 106 256 + 30 + 6 + 106 = 0.9095 Accuracy = It is clear that Table B has the same number of TPs (256) and TNs (106) as Table A. It follows that Table B also has lower accuracy (0.9095) and a lower F1 score (0.9343) relative to Table A. It is apparent that the ML model using the threshold p-value of 0.6 is the better model in this sentiment classification context with a threshold pvalue of 0.4. 181 © 2014-2025 AnalystPrep.