Answers - Reading 38 Backtesting and Simulation

Question #1 of 16 Question ID: 1473998 A risk-averse investor is most likely to desire which of the following attributes of a multivariate return distribution? A) Excess kurtosis. B) Negative skewness. C) Positive skewness. Explanation Positive skewness indicates an above-average probability of returns above the mean, a desirable attribute. Negative skewness indicates an above-average probability of returns below the mean, which would not be a desirable attribute for a risk-averse investor. Excess kurtosis or fat tails indicates higher (than normal) probability of extreme events— again, an undesirable attribute for a risk-averse investor. (Module 38.4, LOS 38.f) Question #2 of 16 Question ID: 1473989 Which of the following most accurately describes the steps in backtesting an investment strategy? Obtain estimates of the regression parameters, determine the assumed values A) of the independent variables, and compute the predicted value of the dependent variable. B) Strategy design, historical investment simulation, and analysis of output. C) Conceptualization of the modeling task, data collection, data preparation and wrangling, data exploration, and model training. Explanation The three steps in backtesting an investment strategy are: (1) strategy design, (2) historical investment simulation, and (3) analysis of output. (Module 38.2, LOS 38.b) Question #3 of 16 Question ID: 1474000 Question #3 of 16 Question ID: 1474000 In conducting a sensitivity analysis, an analyst is most likely to take fat tails and negative skewness into account by repeating a Monte Carlo simulation using a multivariate: A) Bernoulli distribution. B) skewed Student’s t-distribution. C) normal distribution. Explanation To conduct a sensitivity analysis, we fit return data to a distribution that accounts for skewness and excess kurtosis, such as a multivariate skewed Student's t-distribution and then repeat the Monte Carlo simulation. . (Module 38.4, LOS 38.h) Question #4 of 16 Question ID: 1473993 Which of the following is the least likely to result from using information that would have been unavailable at the time of the investment decision? A) Data snooping. B) Look-ahead bias. C) Survivorship bias. Explanation Data snooping refers to a situation where a model is chosen based on backtesting performance. Look-ahead bias refers to using information that would have been unavailable at the time of the investment decision. Survivorship bias is a form of lookahead bias in which results are based on data that only includes entities that have persisted until today. (Module 38.3, LOS 38.d) Question #5 of 16 Question ID: 1473994 Bill Cassidy, CFA, is the portfolio manager for Applied Logistics pension fund. Cassidy is meeting with Alex Swary, the senior quantitative analyst, to discuss the results of backtesting of a model developed by Swary. The model uses several factors in selecting stocks, including EPS growth over the past year, the industry competitiveness index, and price-to-book ratio. The model makes picks on the first trading day of each calendar year with annual rebalancing. While evaluating the results of backtesting, Cassidy should be most likely concerned with: A) survivorship bias. B) data snooping bias. C) look-ahead bias. Explanation Factors such as the price-to-book ratio rely on accounting data (from the balance sheet), which is usually available with a lag—therefore, it may not be available at the time of stock selection. This fact is often overlooked while using historical data and is called the lookahead bias. Survivorship bias results from inclusion of only survivors in the investment universe, while data snooping involves selection of a winning model (from many) based on statistical strength of the test results. The question does not provide any evidence to support either the survivorship bias or the data snooping bias. (Module 38.3, LOS 38.d) Question #6 of 16 Question ID: 1473987 Which of the following statements about backtesting an investment strategy is least accurate? Backtesting: A) is incompatible with quantitative and systematic investment styles. B) is based on the implied assumption that the future will somewhat resemble history. C) can take the randomness of the future into account. Explanation Backtesting is a natural fit for quantitative and systematic investment styles. Backtesting is based on the implied assumption that the future will somewhat resemble history. Methods such as Monte Carlo analysis can allow backtesting to take into account the randomness of the future. (Module 38.1, LOS 38.a) Question #7 of 16 Question ID: 1473995 Which of the following most accurately describes a scenario analysis? A) B) C) Backtesting a model during periods of high volatility and periods of low volatility. Backtesting a model using U.S. market data as well as using the European market data. Backtesting a model for large-cap securities as well as for medium-cap securities. Explanation Historical scenario analysis involves backtesting over discrete periods of structural breaks —over different regimes. The specification of different investment universes is not scenario testing. (Module 38.4, LOS 38.e) Question #8 of 16 Question ID: 1473986 Which of the following statements about backtesting an investment strategy is least accurate? Backtesting is: A) a new methodology that is slowly gaining acceptance in the investment community. B) widely used by managers that use a fundamental investment style. C) useful as a rejection or acceptance criterion for an investment strategy. Explanation Backtesting is not a new methodology; rather, it has been widely used in the investment community for many years. Backtesting can be employed as a rejection or acceptance criterion for an investment strategy. Backtesting is a natural fit for quantitative and systematic investment styles, but it is also widely used by fundamental managers. (Module 38.1, LOS 38.a) Question #9 of 16 Question ID: 1473988 Which of the following most accurately describes a step in backtesting an investment strategy? A) In the “strategy design” step, we form investment portfolios for each period. B) C) In the “historical investment simulation” step, we calculate portfolio performance statistics. In the “historical investment simulation” step, we rebalance the portfolio periodically. Explanation The three steps in backtesting an investment strategy are: (1) strategy design, (2) historical investment simulation, and (3) analysis of output. In the "strategy design" step, we specify investment hypothesis and goal(s), determine investment rules and process, and decide key parameters. In the "historical investment simulation" step, we form investment portfolios for each period according to the rules specified in the previous step, and rebalance the portfolio periodically based on predetermined rules. In the "analysis of backtesting output" step, we calculate portfolio performance statistics and compute other key metrics (such as turnover, etc.) (Module 38.2, LOS 38.b) Question #10 of 16 Question ID: 1473991 Which of the following metrics are most likely to be reported in a backtest of an investment strategy? A) Altman Z-score, Sloan ratio, and Beneish M-score. B) Maximum drawdown, Sharpe ratio, and Sortino ratio. C) Enterprise value, volume, and market capitalization. Explanation The backtest of an investment strategy will produce return metrics, such as average return, as well as risk measures, such as volatility and downside risk. Other measures commonly calculated include the Sharpe ratio, the Sortino ratio, and maximum drawdown (the maximum loss from a peak to a trough). (Module 38.3, LOS 38.c) Question #11 of 16 Question ID: 1473992 Which of the following identifies problems that are most likely to arise in a backtest of an investment strategy? A) Survivorship bias, look-ahead bias, and data snooping. B) Including lagged dependent variables as independent variables. C) Heteroskedasticity, serial correlation, and multicollinearity. Explanation Problems likely to emerge in a backtest of an investment strategy include: Survivorship bias (when using data that only includes entities that have persisted until today), Lookahead bias (from using information that would have been unavailable at the time of the investment decision), and Data snooping (when a model is chosen based on backtesting performance). (Module 38.3, LOS 38.d) Question #12 of 16 Question ID: 1473985 Which of the following statements about backtesting an investment strategy is least accurate? Backtesting: A) approximates the real-life investment process. B) ensures that a strategy will perform well in the future. C) lends rigor to the investment process. Explanation Backtesting approximates the real-life investment process by using historical data to evaluate whether a strategy would have produced desirable results in the past. However, not all strategies that perform well in a backtest will produce excess returns in the future. Backtesting provides investors with additional insight, and lends rigor to the investment process. (Module 38.1, LOS 38.a) Question #13 of 16 Question ID: 1473999 In the historical simulation approach, bootstrapping is most likely to be used when: A) zero-coupon rates are available but par yields are unknown. B) the number of trials is larger than the dataset. C) a merger transaction impacts earnings. Explanation Historical simulation sometimes makes use of bootstrapping, whereby random samples are drawn with replacement. Bootstrapping is useful when the number of simulations needed is large relative to the size of the historical dataset. . (Module 38.4, LOS 38.g) Question #14 of 16 Question ID: 1473997 In the presence of return distribution asymmetry and excess kurtosis, the most appropriate approach would be to make use of a Monte Carlo simulation using a: A) normal distribution. B) skewed Student’s t-distribution. C) F-distribution. Explanation The use of a skewed Student's t-distribution is most likely to help account for properties such as skewness (asymmetry) and excess kurtosis in the underlying return data. . (Module 38.4, LOS 38.f) Question #15 of 16 Question ID: 1473990 A rolling-window backtesting is most accurately described when: A) a data set is divided into two distinct samples. B) the out-of-sample data becomes the in-sample data for the subsequent period. C) repeated sampling from the same data set leads to the use of redundant sources. Explanation Rolling window relies on an overlap between in-sample and out-of-sample data, allowing repeated in-sample training data to adjust portfolio positions based on information available at that time. Data is not divided into just two samples (one for training and the other for testing). (Module 38.2, LOS 38.b) Question #16 of 16 Question ID: 1473996 Which of the following is least likely an example of historical stress testing? A) B) C) Backtesting the performance of the strategy during the high market return period of 2017–2018. Backtesting the performance of the strategy, assuming that the CBOE VIX Index is greater than 55. Backtesting the performance of the strategy during the great recession, a period following the global financial crisis of 2008. Explanation Historical scenario analysis (or historical stress testing) involves backtesting a strategy during actual historical periods. Assumed VIX level is not a historical period. The recessionary period following the global financial crisis of 2008 and the high market return period of 2017–2018 are both historical periods. (Module 38.4, LOS 38.e)