Learning Module 4: Backtesting and Simulation

Learning Module 4 Backtesting and Simulation Sarah Koh heads the quantitative research team at Newton Research Pte. SWF Fund, one of Newton's biggest clients, has asked Koh to help develop new investment strategies by rigorously and independently evaluating their risk and return profiles. SWF Fund would like Koh to evaluate the merits of a "value" equity strategy does owning "cheap" stocks and avoiding (or short-selling) "expensive stocks add alpha?-as well as two multifactor fundamental strategies that incorporate several other factors besides value. SWF Fund's Investment Committee will use Koh's findings to decide whether to begin using these strategies. Koh's work and findings for SWF Fund will be illustrated throughout the reading. EXAMPLE 1 Strategy Design After antal venation with the restment mater at SWF Fund, Sarak Ke notes the following Goal: Superior stisk adjusted absoane return Hypothesis: The studs those with lower selative valuation will autperorss "openuse" tacks in other words, exposure to the "value" factor will lead to outperformance Wah must now farther specify the hypothesis to allow backtesting, as well detine key parameters for the bucktest The value factuer can be described using simost any combination of mucket price and fandanantal perfimuna, un a historical called trailing) or forward-looking hasts. Kath selects a simple valuation metra-trailing varmings yield, the imene of the PE-to quantify the "thexpress of a stock. Although PE is more commonly understood than earnings yield, a serious flaw is that cannot be computed or logically interpreted if EPS is sero or orgative Earsings vield, on the other hand, can be computed for any stik se long as EPS and prior data are available Trang sumings yold Specification of Key Parameters: Inverse Russell 3000 for the US market and S&T Farmge EMI be the European machet. Total returns will be firilged back ano US dollan Start and end date: Because data required bir this strategy are widdy available. Noh will use a long time period: luuary 1986-Mar 2013 Nehalancing frequency, mathly, including transaction costa, but 12-misth nousing average basis will be p Gantt the hacktesting strategy legs utfined here, whath of the killowing rupcom aluut witich the investment committee of SWF Fund should be A the stumes that the US dollat will appreciate against the The historical period of the data includes reci, mency regime champes and perinde af varying innemt lates C. There are serious with computing earnings yield for many Solution: Ris conect. The prothidio matagen is sing a long date hamry that inches engime changes in uulation, cuncies, and treat rates, so the dati statimary Comments, hacktesting perfienunce tous should be supplemented with examinations of perfiomance doing the dacreir A is incomment became the analys makes no asumption about exchange sates. Cia incorrect because earnings yield can be computed as long as EPS and price data are available 2. Which of the illewing describes the endatiomhip between huling be que and пана A. Changing the rehalancing frequency from monthly to weekly would lly incre transaction costs. Changing the rehalasing, frequency from monthly in quarterly would lily transaction conta CBebalancing frequency has no effect on transaction com A let. Behalancing frompartey refers in how ofhun a purtilisis upda of to relect current du chants this caset changes in earnings iek verse Topically, the more frequently vebalancing is done, the more nuding repared, which incurs ne tractions Bis incorrect because decidere in balancing freришу which would decrease tration Cainment became thalancing equency is the primary driver of trading volume, which accurs transaction 1. Which of the blowing is haut a potem of inga shut time per od be a backdest 4. The backtent will cover a limited number of usine cycle, infation The backtest may not be useful because the findings may apply under the conditions presem is the time frame The backtone is likely to come multiple huumees cycle, anflation, and inderest rate regases Cis correct, Comering machiple marromicinesta ssociated with using a short time period for a hacktest, because marne economic mishes send to be multi year in length. A and I am incorrect because thary are cod with wing a short tane period: The hacktest may capture only a kamated experience, and thus the finding may leslevant fur le that experience EXAMPLE 2 Historical Investment Simulation and Output Analysis 1. Describe how the backtest performance of value investing, based on the earnings yield factor, in Europe compares with that in the United States over the 1986-2019 period, as shown in Exhibit 3. Solution to 1: In the United States, the average annual return from the value investing strategy is about 9.2%, with a Sharpe ratio of 0.75, over the backtesting period (January 1986-May 2019), as seen in Exhibit 3, Panel A. In Europe, the same investment strategy generated a significantly lower (by 250 bps) av- erage annual return, about 6.7%, but with significantly lower volatility (Panel B). Hence, the Sharpe ratio for the European strategy, 0.67, is close to that of the US strategy. In both markets, the maximum drawdown is just over three times the volatility of the strategy. Therefore, as a long-term value strategy, the earnings yield factor offers slightly better performance in the United States than in Europe. 2. Describe the cumulative performance of value investing across the different markets shown in Exhibit 4 and the distributions of returns in the United States from this strategy in Exhibit 5. Solution to 2: The value strategy has delivered strong performance over the long run across the several markets, especially in Asia ex-Japan (Exhibit 4). Perfor mance has flattened since 2016, however, in the United States, Europe, and Japan after first leveling off in all geographies except Asia ex-Japan after 2002. Significant drawdowns and potential structural breaks can also be observed in late 1990s (ie., during the tech bubble) and in March-May 2009 (i.e., the risk rally during the global financial crisis) in most regions. More problematically, the strategy in the United States seems to suffer from excess kurtosis (i.e., fat tails) and negative skewness (Exhibit 5). The excess kurtosis implies that this strategy is more likely to generate surprises-that is, extreme returns whereas the negative skewness suggests that those surprises are more likely to be negative (than positive). EXAMPLE4 Backtesting the Performance of Factor Allocation Strategies During the presentation of her backtesting results to SWF's investment com mittee, Kok is asked the following questions 1. Regarding rolling window backtesting, which one of the following state ments is wate The data are divided into just two samples. Out-of-sample data become part of the nest period's in-sample data C. Repeated in-sample training and eut-of-sample testing allow managers to adjust security positions on the basis of the arrival over time of new information Solution to 1: A is correct, became the statement is inaccurate. Band Care incorrect because they accurately describe the rolling-veladow backtesting technique. 2. Which of the following is a drawback of the long-short hedged portfolio approach for implementing factor-based portfolios? A. The bedged portfolio is fernsed by going long the top quantile (with the best factor scores) and shorting the botton cpaantile (with the worst factor scores) B. Securities toast be sunked by the factor being scrutinized and then grouped into quantiles based on their factor senms. C Not every manager can short stocks. Solution to 2: C is correct, because it best describes a drawbuck of the long-short hedged portfolio approach. A and B are incorrect because they describe the ap proach itselt 3. Which one of the following is rest a metric ur visual used in assessing back testing of a tacice-based investnest strategy A. Distribution plots of factot ceturos R. A word cloud of text describing the characteristics of the factor C Maximum drawdown Solution to 3: Iliu correct, hecause a wond cloud is not a visual azad in assessing back testing of a facter-based investment strategy. A and Care incorrect, be cause they are vinaals and metrics, respectively, sand to assess backtests of factor-hased strategies 4. llegarding the aue of rolling window backtesting in assessing factor alloca thon to a risk parity-based strategy, which statement is correct? A. The procedure is toed once for estimating factor returno over the nalling window B. The procedure is saed once for dividing the data into jast two samples The procedure is used twice-once for estimating factor returns over the rolling window, and a second time for estimating the covanance matrix of factor returns (for deriving risk parity weights) over the miling window. Solution to 4: C is correct, because the procedure must be used a second time for enstanat ing the covariance matrix of factor returns foc deriving risk parity weights) over the rolling window. A is incorrect because the procedure must be done twice once for estimating factor returns over the rolling window and a sec und time for estimating the covariance matrix of factor rotaras (for descong risk parity weights) Bis inomrest because the rolling-window procedure divides the sample into many samples