Reading 38: Backtesting and Simulation

Page No. 45Date//Reading 38 Backtesting AndSimulationof Backtesting is a process by which historical data is usedto emulate the invest process- This helps use determine whether a particular invst"strategy or tech would have delivered the anticipatedresults as well as any hidden risksThe primary goal of backtesting is to assess the risk &return of an invest" strategy by simulating the investprocessBacktesting uses past data to evaluate wheter a par ticularinvest" strategy would have produced excess return historicallyThis assessment allows an investor to optimize their invest"process and strategyThe backtesting process is intuitive because it minics howinvest" happens in real life- Formulate f invest strategies (Pie determint)- - Test those ideas using historical infoAssess how Well those strategies would have performedbakitesting has been extensively used in the financialindustry for decades. Page No 46Date //Backtesting is a nutural fit for systematic and quantitativeinvest styles-Fundamental managers also make extensive use ofbacktesting .Increases in data availability t dramatic T in computingpower, have facilatated quantitative assessmentsSoftware allow investor allowBacktest and simulate infinite Combinations of possible investstrategiesAnalze the risk and return properties of these strategiesCreate multifactor modelsBuild invest portfalioall before making invest" decisionSteps in Backtesting an Invest StrategyIStrategy designSpecify invest hypothesis and goalsSpecify invest universe and return defination in whichRebalandag freq and transactioncostcursenyStart / end datekey parameter PageNo. 47Date //0 Factor portfolioBM - Benchmark : equal weights (25% in 4 folios)BM - Risk parity: equal contribution to portfolio riskDefensive valueGrowthAnalyst sentimentCydical valuePrice momentumPropitabilityLeverageEarnings quality2Historical Simulation2 layerRelling - window backtestingAllows generation of large no. of observations from the historytime period sample.From invest" portfolios for each period according to the rulesspecified in previous stepRebalancing the folio periodically based on pre-specified rules3Analysis of Backtesting OutputCompute performence statistics, , such as risk and return for folioCalculate other instext relavent metrics, such as turnoverLong / shorthedged postfolioCompare results of diff methodologies Stale data - old data is usedPage No. SoDate/Metrics Reported in a BacktestThe backtest of a strategy will produce various metics:- Avy returnRisk measures such as volatility & downside riskThere are other measures that can be calculated:- Sharpe ratio better- Sortino ratio- Max drawdown (greatest loss from poak to tough)lower the betterVisuals Reported in a backtestVisuals used in backtest of an invest strategy ofteninclude return distribution plotsProbability distribution of return plot for the RP portfoliousing historical simulated returnsProblem S in backtesting a StrategySurvievership Bias case of look ahea d biosDeriving entities conclusions from the data that reflect only thosethat have Survived to that dateLook ahead BiasUsing historical info that was unknown of unavailable during theUse period over which the backest is conductedpoint -in-time data sol nHigher than normal -FvaluePage No. 51SoftCross validationDate / /501mData Snooping / Data miningMaking inference after looking at statisical results ratherthan testing the data prior inference (P - hacking)Cross - Validation - Backtesting with out of Sample dataGross - validation is when a model is first fitted usingtraining data, and then its performance is assessed(often over several rounds) using seprate testing dataAn invest" strategy can also be cross validation usingdata from diff geographic regiun- performance from other global markets is used to assesswhater a strategy is robust Machine LearningInterpreting Historical Scenarios AnalysisType of backtesting that ex plores the performance and riskof an invest" Strategy in diff Structural regimesAsset return distributions often exhibit skewness and excessKur tosis (i.e fat tails)holling window backtesting may not fully account for these:- The dynamic nature of financial markets- Possible extereme downside riskwelpp)Out off Sample data becomo inspaple data for the subsequent period (dataScenario analysis & Simulation Can provide a more tholoughpast Ragul of invest" straggy performance.Pertrayal Page No. 5zDate /Monte Carlo Vs Historical SimulationMonte Carlo SimulationStatastical distribution is specified and calibrated usinghistorical return data.When the assets or factors are correlated a multivariatedistribution should be used rather than modeling eachasset or factor on a stand alone basisEach key variable is assigned a statistical distributionPobservations are drawn at random from the assigneddistributionsIt is highly flexible and an array of diff distributioncan be used across variety of key variables.This Kind of simulation is especially useful in meaningthe downsiderisk of invest strategiesIt is complex & computationatly intensiveHistorical SimulationImplement a Strategy at some past date & collectingresults as the Strategy run over time.Relative simpleShare many of the advantages & disadvantages ofrolling -window backtesting.Like rolling - - window backtesting , historical simulation dependson assumption that randomness in the future can bepredicted using return distribution from the past, Page No. 53Date / /This approach assumes "Past asset returns provides someguidance about future retuins"Particularly used by banks for market risk analysisInputs & Decisions in SimulationIn historical simulation, observations are Landerly chosenfrom a historical dataset So that each observation hasan equal prob of being selected.Bootstrapping is sometimes used in historical simulation :- Random sample are drawn with replacement- It is useful when the no. of simulations needed islarge relative to the size of the historical datasetsSensitivity Analysis What ifA method for evaluating how a target variable leypostfolio return) varies due to changes in the inputvaria ble (eg asset of factor retuins)Skewed + distributionConducting a Sensitivity AnalysisFit factor return data to a distribution that accountsfor Skeness and excess Kurtosis.Then, repeat Monte Carlo SimulationSensitivity analysis can overcome shortionings of atraditional Monte Carlo Simulation-Not limited to multivariate normal distributions (which drnot take into account fet tails or -ve Skeneus) Page No. 54Date //Skewed multivariante t.- distributionHelps to take fat tails and distribution skewnessinto allButalsoTthe possibility ofesterror- Because it req est of more parameters