# Parameter Optimisation & Backtesting – Part 2

This is a follow on from: http://gekkoquant.com/2012/08/29/parameter-optimisation-backtesting-part1/  The code presented here will aim to optimise a strategy based upon the simple moving average indicator. The strategy will go Long when moving average A > moving average B. The optimisation is to determine what period to make each of the moving averages A & B.

Please note that this isn’t intended to be a good strategy, it is merely here to give an example of how to optimise a parameter.

Onto the code:

Functions

• TradingStrategy this function implements the trading logic and calculates the returns
• RunIterativeStrategy this function iterates through possible parameter combinations and calls TradingStrategy for each new parameter set
• CalculatePerformanceMetric takes in a table of returns (from RunIterativeStrategy) and runs a function/metric over each set of returns.
• PerformanceTable calls CalculatePerformanceMetric for lots of different metric and compiles the results into a table
• OrderPerformanceTable lets us order the performance table by a given metric, ie order by highest sharpe ratio
• SelectTopNStrategies selects the best N strategies for a specified performance metric (charts.PerformanceSummary can only plot ~20 strategies, hence this function to select a sample)
• FindOptimumStrategy does what it says on the tin
Note that when performing the out of sample test, you will need to manual specify the parameter set that you wish to use.
?View Code RSPLUS
 ```  library("quantmod") library("PerformanceAnalytics")     nameOfStrategy <- "GSPC Moving Average Strategy"   #Specify dates for downloading data, training models and running simulation trainingStartDate = as.Date("2000-01-01") trainingEndDate = as.Date("2010-01-01") outofSampleStartDate = as.Date("2010-01-02")     #Download the data symbolData <- new.env() #Make a new environment for quantmod to store data in getSymbols("^GSPC", env = symbolData, src = "yahoo", from = trainingStartDate) trainingData <- window(symbolData\$GSPC, start = trainingStartDate, end = trainingEndDate) testData <- window(symbolData\$GSPC, start = outofSampleStartDate) indexReturns <- Delt(Cl(window(symbolData\$GSPC, start = outofSampleStartDate))) colnames(indexReturns) <- "GSPC Buy&Hold"   TradingStrategy <- function(mktdata,mavga_period,mavgb_period){ #This is where we define the trading strategy #Check moving averages at start of the day and use as the direciton signal #Enter trade at the start of the day and exit at the close   #Lets print the name of whats running runName <- paste("MAVGa",mavga_period,".b",mavgb_period,sep="") print(paste("Running Strategy: ",runName))   #Calculate the Open Close return returns <- (Cl(mktdata)/Op(mktdata))-1   #Calculate the moving averages mavga <- SMA(Op(mktdata),n=mavga_period) mavgb <- SMA(Op(mktdata),n=mavgb_period)   signal <- mavga / mavgb #If mavga > mavgb go long signal <- apply(signal,1,function (x) { if(is.na(x)){ return (0) } else { if(x>1){return (1)} else {return (-1)}}})   tradingreturns <- signal * returns colnames(tradingreturns) <- runName   return (tradingreturns) }   RunIterativeStrategy <- function(mktdata){ #This function will run the TradingStrategy #It will iterate over a given set of input variables #In this case we try lots of different periods for the moving average firstRun <- TRUE for(a in 1:10) { for(b in 1:10) {   runResult <- TradingStrategy(mktdata,a,b)   if(firstRun){ firstRun <- FALSE results <- runResult } else { results <- cbind(results,runResult) } } }   return(results) }   CalculatePerformanceMetric <- function(returns,metric){ #Get given some returns in columns #Apply the function metric to the data   print (paste("Calculating Performance Metric:",metric))   metricFunction <- match.fun(metric) metricData <- as.matrix(metricFunction(returns)) #Some functions return the data the wrong way round #Hence cant label columns to need to check and transpose it if(nrow(metricData) == 1){ metricData <- t(metricData) } colnames(metricData) <- metric   return (metricData) }       PerformanceTable <- function(returns){ pMetric <- CalculatePerformanceMetric(returns,"colSums") pMetric <- cbind(pMetric,CalculatePerformanceMetric(returns,"SharpeRatio.annualized")) pMetric <- cbind(pMetric,CalculatePerformanceMetric(returns,"maxDrawdown")) colnames(pMetric) <- c("Profit","SharpeRatio","MaxDrawDown")   print("Performance Table") print(pMetric) return (pMetric) }   OrderPerformanceTable <- function(performanceTable,metric){ return (performanceTable[order(performanceTable[,metric],decreasing=TRUE),]) }   SelectTopNStrategies <- function(returns,performanceTable,metric,n){ #Metric is the name of the function to apply to the column to select the Top N #n is the number of strategies to select pTab <- OrderPerformanceTable(performanceTable,metric)   if(n > ncol(returns)){ n <- ncol(returns) } strategyNames <- rownames(pTab)[1:n] topNMetrics <- returns[,strategyNames] return (topNMetrics) }   FindOptimumStrategy <- function(trainingData){ #Optimise the strategy trainingReturns <- RunIterativeStrategy(trainingData) pTab <- PerformanceTable(trainingReturns) toptrainingReturns <- SelectTopNStrategies(trainingReturns,pTab,"SharpeRatio",5) charts.PerformanceSummary(toptrainingReturns,main=paste(nameOfStrategy,"- Training"),geometric=FALSE) return (pTab) }   pTab <- FindOptimumStrategy(trainingData) #pTab is the performance table of the various parameters tested   #Test out of sample dev.new() #Manually specify the parameter that we want to trade here, just because a strategy is at the top of #pTab it might not be good (maybe due to overfit) outOfSampleReturns <- TradingStrategy(testData,mavga_period=9,mavgb_period=6) finalReturns <- cbind(outOfSampleReturns,indexReturns) charts.PerformanceSummary(finalReturns,main=paste(nameOfStrategy,"- Out of Sample"),geometric=FALSE)```

# Parameter Optimisation & Backtesting – Part 1

When developing a strategy that uses a technical indicator, such as a moving average, it is often difficult to decide on what period to use with the indicator. Should it look at the last 10, 20, or n days? The simple solution is just to iterate over many different parameters and look for the parameter combination that optimises a performance metric perhaps the Sharpe Ratio or minimises the Max Drawdown or some combination of both of them.

This method must be approached in a sensible manner, essentially we are fitting the strategy to the data and must be careful not to over fit and capture spurious profits. It is likely that any parameter that is trained will vary over time, the strategy must be able to deal with changes to this parameter.

For example say we are tuning the variable A, and we find that the optimum solution is A=n in the training data (it is very profitable with high sharpe ratio). How well does the strategy perform when we set A=n+1, or A=n-1 do we still get good results? If you don’t it’s an indicator that maybe you have over fitted.

For more discussion on this see http://epchan.blogspot.co.uk/2010/01/method-for-optimizing-parameters.html

Part two will contain a script to perform an optimisation on a simple moving average strategy.