053 - Martyn Tinsley - 2 of 2 - Walk Forward Correlation: A New Tool for Robust Strategy Design!

Regardless of whether you're using single stage or multi-walt-forward analysis, multi-stage walk-forward analysis, they both follow this principle, which is you choose a set of parameters, you see how they perform on data they've never seen before. There's a big problem with this approach, even for multi-stage walk-forward analysis, which is let's face it's considered the gold standard approach. And like I say, when I have sufficient trades, this is an approach I take myself. So a big advocate of Apollo, you know, and the work he's done, and this is a great approach, but it still uses this technique, which is potentially forward. Let me show you how walk-forward correlation, which is this new technique, is completely different. Welcome to the algorithmic advantage. We're here to expand the toolkit of the Quant Trading Community and introduce investors to the many advantages of systematic trading. Our goal is to educate and inspire as we embark on a captivating journey into the vast knowledge and experience of leading portfolio managers and other experts in the field. Alright Martin, welcome back. Part two, this is really good stuff. I propose we just jump straight in where we left off so that viewers who are halfway through it in part one can continue right along where we left off in part two. And we'll get through this presentation. And then, you know, I'm keen to know a little bit about what you're doing with AI, what's next for you as well. So the presentation that we've been going through is really about what constitutes best practice for strategy development and optimization. And obviously, you know, perhaps the most important topic in systematic trading. So a really interesting part one already. And well, maybe you could just remind us bullet point a couple of things we covered in part one, those I think three points and we've got another two major points to get through. And then let's dive into, you know, the next point which I believe is around looking at the performance metrics and parameter selection. Yeah, so so last time I think probably the main message that I was hopefully getting across was that optimizations can be very deceptive. Okay, and they can be deceptive because of the fact that they often get so affected. The results get so affected by major market news events, other random events that are non-repeatable. And that makes it extremely difficult for an algorithm trader to select what the best settings are for a strategy. And it really is a black art in terms of being able to do that. And one of the things that I proposed is this ability to filter out as best as possible those random events which hopefully leave behind the edge of the strategy. So if you can imagine you're looking at an optimization surface, what you'll see now is the genuine edge and it's that edge that is repeatable. So hopefully when you come to test that and validate that on out of sample and also when you trade it in a live account, it's much more representative and the results are much more representative of what you're actually seeing in your optimization. So I think that was the main kind of emphasis on the first session that we had. Yes, all right, awesome. Let's move right into the next point then. Okay, so if I just share my screen again, hopefully you can see that sign. It looks good, yep. So what I want to do here is really talk about performance metrics. So I'm talking about here things like the sharp ratio profit factor. Metrics that traders need to use in order to do what I was talking about in the past in the last episode. They're needing to select the parameters and the settings they're going to use in their live account and we typically use performance metrics to do this. And I continually get asked by other traders what is the best performance metric? Is it the sharp? Is it the return overdraw? Is it profit factor? What is it? And this is something that I myself have battled with for many years. You know, what is the best metric to use? And if I were to say there's one metric that helps to ascertain the edge of the strategy, which is what we're looking for, it's the profit factor. This is the performance metric I've had most joy with in terms of selecting good settings. Okay, however, recently over the last couple of years, I started to take a different approach. Okay, and that's what I want to talk about now. So let me take you through a bit of the rationale behind this. So if we've got, where are we? Here we go. The thing that you have to realize is that different performance metrics tell you different things about a trading strategy. They tell a different story about it. So there are certain metrics like the sharp ratio, like the compound annual growth rate over the drawdown. That focus their kind of attention on the equity curve. And they look at this balance between the reward that the equity curve delivers versus the risk. Okay, now sharp does they have to based on standard deviation? You know, if you're looking at drawdown, then you're looking at percentage drawdowns to give you that element of risk. And they're then combined to give you an assessment of what that equity curve effectively looks like. And the higher the performance metric, typically the better the equity curve looks. So that's one category of performance metric. And it's important. So it's an important category. But then you've got others which start to look at things from the perspective of the individual trades that constitute and build that equity curve. So I'm thinking here about things like van Thaap's or multiple. Okay, so this looks at every individual trade and attributes a maximum risk to the trade and says how well on average did the trades do in relation to that trade risk? So this is telling you something subtly different. This is telling you if you place a trade in the market, what can you expect to get return for a certain level of risk usually set by your stop loss? Okay, but again, it's incredibly important to know this, but it's different. It's different than the first type of metric we talked about. And then you've got kind of ratio based individual trade outcomes. So here profit factor fits into this. So profit factor looks at on average the gains from all of the individual trades compared to the losses of the loss making trades. How does that metric compare? And I found that this is really excellent at distinguishing how good the edge is from different parameters. The problem lies when you start to say which is the best metric? If you choose a metric, so you might choose van Thaap's or multiple, you're only seeing a small slither of the full story here. You're not getting the full picture. And if you choose another metric, that is in one of these other kind of categories. Again, you're seeing a different slither of information and intelligence about the optimizations and the back tests that you've performed. Okay. So I started to think, you know, really all of these different aspects are important. So how do we get all of the information from all of these into a single metric? And so that's when I started to come up with the concept of a combined performance metric. That includes all of these. And it took me a while to come up with the kind of the methodology around how you combine these because that can put you in. different, right? So profit factor, for example, the break even for profit factor is a value of 1. Anything above 1 is generally considered good, anything below 1 is bad. Whereas for others like fan thoughts are multiple, 0 is the kind of the cutoff point between good and bad. And likewise for compound annual growth rate, that's based around kind of a 0 level. And they have vastly different ranges of values. But eventually I came up with a solution to this using, you know, for anyone who's wanting to replicate this, really Z scores. It's a statistical technique and Z scores is what you want to be using in order to combine these things together. If you're alright, I'll just quickly show you how I've done this in my own software assignment. Yeah, absolutely. I'd love to see it. So I'll just briefly show you. So what we've got here is this is a table of results that have come from, from an optimization. And, you know, I have the ability to look at different individual metrics and kind of look at how they perform in this. And this is what I used to do for many years. You know, look at the compound annual growth rate over the max drawdown. Look at the sharp ratio, see how these compare. But it drives you crazy because they all give you different results. You know, one set of parameters will be best for one metric. And a completely different set of parameters for another metric. So because they're looking at different things, you expect that to happen. And so I came up with this concept of a combined ranking. And what this does is, if you look here, so in this particular case, I'm combining the performance here from sharp ratio, compound annual growth rate over the mean drawdown, which is one of my favorite metrics. Yeah. Rather than the max drawdown, I think it actually tells you more about this strategy. Also, run fast, expect and see in profit factor. And combining them into a single ranking. So what you then see here, and if your, you know, if your viewers take the same approach by producing these kind of metrics, what you see is a ranking that's more balanced. It's a balanced view of how, while these settings perform, looking at all of those different characteristics of a trading strategy. And I found this to be a far better way of getting intelligence about the process you've been through and making much more informed decisions. Does that allow you, Martin, to have a more intelligent objective function too if you are doing walk forward analysis or something like that? Could you, and would you use this as your objective function? So in terms of walk forward analysis, this is something that I used to use, you know, religiously. So by this, I assume you mean that kind of the Bob Pardo multi-stage walk forward analysis. So I used to use walk forward analysis all the time. And for some strategies that is absolutely the gold standard of, you know, the gold standard approach currently is walk forward analysis. Where walk forward analysis starts to not be as effective is when you have strategies that don't produce a sufficient number of trades to be statistically significant. Okay. So if you have, for example, a single stage, optimization with a walk forward out of sample stage, if you've got, you know, a huge number of trades that form part of that, it is then feasible to break it up into a multi-stage process, which has the advantage, of course, of staying in tune with different market regimes as they change throughout that time period. That's great. That is the way you should be doing it. But what I found is that there's a lot of trading strategies where you can't get sufficient sample size to make that approach valid. And so when that's the case, I found a single stage to be much more effective because it's this tradeoff. It's a tradeoff between statistical significance of your result, which you get in a single stage, and the ability to show how a strategy adapts to changing market regimes. And if you have sufficient sample size, great, do it. But if you don't, and most of the strategies I trade don't, so I tend to stick the vast majority of time with single stage. Yeah. Okay. But I will talk, in fact, the next thing I want to talk about, which is the real biggie. Yeah. I will talk about how a new technique that I've developed can be used in conjunction with Walk Forward Analysis. So I will talk about that a little bit more in the next stage. Awesome. All right. So anything else that you want to cover on that? Unless you've got any questions, silence? Well, no, not really. I like that. And by combining metrics, obviously, you can, you know, into the metric that works for you, you can streamline your process. And you could automate it easier as well. Like you can shuffle through ideas quicker because something just stands out. You've got a headline number that you trust that you like the look of that gives you enough feedback to iterate quickly to sort and rank over. So yeah, it's a really great idea. All right. Cool. No, we'll have it to keep moving because I'm very much looking forward to this next section. Yeah. Okay. So what I want to kind of take a little step back and look at what the standard approach or the kind of the traditional approach that traders have taken for forever, almost. And just to set the scene for this new technique that I've developed. And to show you where this would actually fit in in this in this process. Okay. So most of your viewers will already be familiar with this, but I think it's necessary for me to go through it to kind of position where this new technique fits. So if you look at this, we've, so this is from my perspective, when I am developing a trading strategy, I typically use 12 years of data. And so it's not moving forwards. There we go. And I split that in a ratio of three to one between my insample data and my, my out of sample data. Now, the insample data is where I do the vast majority of my analysis work, my devon work of a other strategy and so on. So any trials that I'm running, any back tests, any preliminary optimizations I perform are all done on this insample data. And I really consider the out of sample data to almost be sancers after, you know, I have to keep that untouched until the very final stage, where I typically perform the validation that I need to perform. So typically here, this is where traders would perform a walk forward back test on data that had never been seen before by the strategy. So you're really testing out all the work that you've done in this insample stage around the trials, the development, the optimizations and everything else. You're testing out how that then works on data it's never seen, which is of course completely analogous to what you will then do when you go live because when you go live, again, the strategy's never seen that data and so you're almost trying to replicate what will happen when you put your strategy live. So generally speaking, you should never use this data for optimizations. Okay. Now there is an exception to that, which I'll come onto in a moment, but you should never be using it for optimizations that you then make decisions about the best settings because you'll be introducing a look forward bias and to be quite frank, you cannot trust the results that you're seeing anymore. So keep this for your validation stage. So traditionally, after you've gone through this and you've done your final optimization, you'll then take what you consider to be the most robust parameters, and you'll test them on this walk forward validation out of sample data. What I think is a much better methodology, and this is the new technique that I've been developing over. I said to you in the first episode that we had, I've spent a decade really trying to hone the whole process of strategy development into a better process. And over the past kind of 12 to 24 months, this is something that has started to develop, and it's now at the stage where I've proved that this is a really, really effective technique that kind of improves dramatically the traditional approach. But what I want to say is regardless of whether you're using the traditional approach or this other methodology that I'll talk through with you today, the next stage here typically is that you've run what's called a pre-live optimization. And this optimization would use exactly the same settings that you used for your final optimization, and the only reason it's best practice to do this is that you're effectively now saying, "Okay, I've made my decision. I am going to trade this strategy." So let's now find the parameters that work best on more recent data. So you have to be at the point at which you either decided, "No, I'm going to throw it away, so I don't need to do this." Or, "Yes, I validated it on the out of sample. It is good enough to trade." And so now I perform a pre-live optimization. The difference now is that when you select your most robust parameters, these are what you're going to use in your live trading. Martin, can I just ask a quick question there. The pre-live optimization data looks like you're looking at, say, the more recent two thirds or nine years or whatever it is. Right, talk today. Yes. Would you not just at that point use all the data? So you could, and that is a technique that's used, I prefer not to. I prefer to keep everything, absolutely everything the same as I did in my final optimization. So I did my final optimization on nine years worth of data with a particular set of parameters that I optimized and values that I optimized. And I want to replicate everything to, just to make sure that I'm not introducing anything different that might change the way I then select those parameters. Everything has to be the same for me, which is why I shift it forward and use the most recent nine years, which is kind of more in tune with current market dynamics and you're incorporating those most recent three years now. So hopefully that would produce better parameters for your live trading and the parameters you've got from the in sample optimization previously. So with those two steps together there, does that mean that the out of sample data is always the more recent or is it sometimes relevant to take that three years and and select the out of sample period based on particular regime or, you know, put it in the middle of the of the time period or is it always just generally you leave it to the last section and that's that. And the reason again is for consistency between this development stage and the pre-live stage because in the development stage you did your optimization and then you tested that on the three years of data after that period. And so again for consistency, I want to use that approach on pre-live and it just so happens now that the data after is the live markets, but yeah, consistency again. So I'm not the great advocate in terms of using out of sample data that isn't the most recent. Because when you go live, you have no choice. You're always going to be using the most recent. So my view is you may as well be consistent in your development stage. Yeah. All right. Okay. Got it. So I mentioned what forward analysis there. So let's go into a little bit more detail about the differences between the traditional approach. Between walk forward analysis and why I believe walk forward analysis is better. Okay. So let's imagine with the traditional approach, you've performed an optimization. Okay. So you've got some kind of results that you're either viewing in tabular form or you're viewing as a surface or whatever. Doesn't really matter. You will make some kind of a judgment from that data about what the most robust area is of the surface. Now what a lot of novice traders would do is they just choose the highest value. Okay. That can be dangerous because the highest value isn't necessarily the most robust value. And we kind of looked a lot at this last time in terms of how different values can be influenced by noise. And if you look at this particular example, look at what this highest value is right next to. It's right next to this hugely lost making part of the surface. So it only takes a relatively small change in market regime in the live environment to kind of drop off a cliff in terms of performance. So for me, this is never this this point would never be something I would choose. It's just too close to this hugely lost making part of the surface. So for me, I'd always want to move into maybe this point, even if it's not got quite as high performance because this is further away in that parameter space from this negative area. So for me, this is a much more robust area. Now in the traditional approach, if I were selecting this point, what I'd now do is I would take those settings and I'd run them forward on the out of sample data and look at what the performance was. And so at this point, I get some kind of a back test accuracy chart that I then make a decision on an idea to be saying at this point, this is good enough to trade or no, it isn't good enough. So I either go back to the drawing board or I throw the strategy away. And this is this is the approach that's been taken regardless of whether you're using single stage or multi wall forward analysis multi stage walk forward analysis. They both follow this principle, which is you choose a set of parameters. You see how they perform on data they've never seen before. There's a big problem with this approach, okay. Even for multi stage walk forward analysis, which is let's face it, it's considered the gold standard approach. Big advocate of part O, you know, the work he's done and this is this is a great approach, but it still uses this technique, which is potentially forward. Let me show you how walk forward correlation, which is this new technique is completely different. So here you perform your example optimization in exactly the same way. So everything that's happened up until this point of performing your final optimization after your development, your trials, everything else is exactly the same. The difference then is that instead of comparing how this works with a single set of parameters on the out of sample, you look at the entire. And dimensional parameter space on the out of sample data, okay. Obviously it's going to kind of a bit different to the surface you're seeing on in sample, but the analysis that you then perform is that you take every single parameter value. That's important. Not just one. Hmm. You take. every parameter value. Yeah, so normally you're forward, you'd be taking the best parameters only forward. So yeah, just to really call it out again, you're talking about not just to walk forward process as normal, but looking at all or a huge range of the matrix of different parameter settings and running them again out of sample. Every single one, that's right. Okay, so for example, this one, which I think, so this, if we're looking at the chart here, we're looking at a parameter values of 120 and 250, okay, on two parameters. So you find the same parameters on the out of sample and you see how they perform. And this is the key bit. What you then start to do is to plot this data on the walk forward correlation chart. Okay, so you can see here you have on the x-axis, you have the in sample values and on the y-axis, you have the out of sample values. These are the performance values. So for example, if you were using sharp ratio, this is the in sample sharp values, the out of sample sharp values. Okay, so for this point, we've got here, just a single point, you'd then plot that on the chart. So hopefully you can, it's a little bit small, yeah, hopefully you can see the blue point that's appeared there. So you've plotted the in sample, result, the out of sample. Okay, don't look at the axes. This is just kind of just for illustrative purposes. So then you do that with the next point. Okay, I've just used the other corner here just, because it's easier to pick them out. And you plot that relationship on the chart also. And the next one and the next one. And you do it for every single point. And eventually you end up with a scatter chart that is the walk forward correlation chart. Okay, now this is an absolute gold mine of information. Now for superior to just looking at what the back test looks like on that single set of parameters. So let me take you through some different scenarios of what this chart is likely to look like. Okay, sorry, I should have said at this point, you need to perform a calculation of what the correlation is. Okay, so you can either use Pearson correlation or Spam and correlation. I prefer Pearson at the moment. That might change in the future. I'm doing some work on on different calculation techniques, but you'll come out with a value that will be between zero and one. One means perfect correlation, zero means no correlation. Okay. That's it. I just like looking at that, those charts there go back Martin. I mean, I was already speculating on where you were going, because you can see on the charts, you know, the two mountain landscapes, they're relatively similar, you know, the valley of the shadow of death is in the same place and the peaks, the victory peaks are in the same place roughly. So the result is a high correlation suggesting that the parameters performed in a similar way out of sample as they did in sample, which is really telling us something about the stability of the model. Exactly, exactly right. Okay. So in this particular case, that is the case. When that is in the case, you might get something that looks instead like this. Okay. So here, you kind of struggle to see any pattern whatsoever. So in terms of the measurement of the correlation in this particular case, this is zero or zero or close to zero correlation. Now, what does this tell you? First of all, it tells you one of several things. You've probably overfitted your model. And I'll just run through a different scenario here as an example. Look at this point here, top right. Okay, let me circle it. There you go. This point top right is a really important one, because in the in sample dataset, this is the, these are the parameter values that performed best. This is the peak. It's the highest peak on your optimization surface. Okay. Now, let's go back to what would have happened in the traditional approach. In the traditional approach, you'd have selected this and you'd have tested it on out of sample, but look on out of sample, this is positive. So you're going to get a decent looking back test curve. Okay. Fantastic. Most people would see this and say, we're getting somewhere here. This definitely got potential. And they'd either say, you know, I do the trade-it live now where I do a little bit more work on it on the strategy and then trade it. This has definitely got potential. But remember, what waterfall correlation is telling you is that this is completely random. This doesn't have any potential whatsoever. Okay. And in fact, if we chosen the second highest peak on the chart, which just happens to be this one down here, okay. So on the x axis, this is the second highest. If you'd, if that just happened to be a little bit higher and you tested that on your out of sample data, it would have produced a curve that looked like this and you'd have probably thrown the strategy away. So just because these two points were just slightly different, you've made a completely different decision. For the top point, you've said, great strategy works. It's past validation. Bottom point, nah, it doesn't work, throw it away. Okay. Complete opposites. Just because they were slightly different performances. Okay. But you will not make that mistake with waterfall correlation. So I need to be clear that waterfall correlation like this is not the method that you will use to select the parameters. First and foremost, you would still use the approach that you've used in the past. Okay. The difference is that walk forward correlation is giving you the go or no go to even get to that stage. Okay. So for example, if your walk forward correlation looked like this with zero correlation, this is a huge red flag that says to you as a trader, don't even bother trying to select the best parameter because the fact is that this strategy has no discernible edge. Okay. Everything's random and it has no predictive power. Okay. That's important. It's telling you that the process you've taken and the model you've developed has no predictive power. Because the in-sample results bear no relationship whatsoever to out-of-sample results. Yeah. There's no relationship there. There's no predictive power. And so why would you even then try to go back to an optimization chart to try to select the best parameters because you know that has no bearing on how they will perform in your live account. No bearing whatsoever. This is really quite an innovative approach, man. It is. I think it is. And I truly believe that this will become the de facto methodology. Okay. And just to be clear, this technique works both with single stage, validation, but also multi stage. So, but part of the goal is this, it could still operate with that and you could get one of these charts from every stage of the process to get a really good view of whether your model was being truly predictive at each stage of that process. So it actually works in combination. You heard it here first on the Elgo advantage. This is, I mean, if you're watching this show right now and you are not subscribed, what are you doing like subscribe? You know, the gosh, we are an underrated, Quantriding podcast. Let's just say that. But all right. So let's move on. Okay. So through some other scenarios here. So what we've got here is we're starting to see a pattern visually. Okay. You can measure the correlation here. The correlation of that 0.25. We're seen a pattern, but we're also getting, as you can see, some kind of outliers here. Okay. It's not a perfect model by any stretch. Okay. But what it's, so what it's telling you is that probably you have been subject to some overfitting in this model. But there are certain areas of the surface that are beginning to show genuine edge. If you select a value with a high in sample performance, you're likely to get a positive out of sample performance. This is really important because this now tells you, okay, the model works. So it might be worth moving forward to that next stage where you do now select the best parameters from the surface, the most robust parameters from the surface, okay. So it has some predictive power, but them also with a low correlation like this, a low positive correlation, it also indicates there might be a bit more work required first before you move forward. But if you get something like this, you know now you kind of onto a winner. You know that there's limited overfitting with this kind of pattern. And you can already see that because you've got quite a lot of points in this upper right quadrant, there are areas of the surface that have genuine edge, okay. So you've developed a strategy at this point that on the sum settings has genuine edge, which is great news. And this has very good now predictive power. If you're choosing high values in in sample, you're getting high values out of sample. This is really positive. You could be using better a correlation of point nine. This has extremely good predictive power, but you're unlikely to see this kind of results in reality. What kind of trade counts are you looking at in your strategies to make this statistical analysis valid? I think we did cover this on part one. Yeah, we did. Yeah. And particularly for these, presumably even in that out of sample period, one needs quite a few trades. Indeed. So, you know, when I covered this in part one, and I would, if people have come straight into part two, I would recommend you go back and watch that because one of the things I talk about is statistical significance and how important that is and how easy it is with low trade sample sizes to be misled by the information you're getting. So for me, an absolute kind of hard limit is a thousand trades. Anything less than that, and I, I won't really consider the strategy because I don't believe I can get statistically significant results from it. But this technique actually, walk forward correlation will either confirm or otherwise that assumption, okay? Because remember, the number of dots you're getting here aren't the number of trades. The number of dots you're getting on the walk forward correlation side is the number of different parameter combinations you test it. So even if you've only got ten trades, you could still have, you know, significant number of points on the scatter chart. So, what would typically happen with low numbers of trades is you would get this scatter chart you see on the left in my view. If you've only got a strategy that produces a hundred trades, you're probably going to get a walk forward correlation chart like the one on the left that has no predictive power because your results, your individual results don't have any significance from a statistical point of view. So this will tell you that for a start, it will actually help you determine because when you get this thing on the left, you don't know at the moment what the problem is. The problem could be statistical significance. It could be that you haven't got enough trades or it could be that you've just simply developed a strategy that despite what your instincts told you, it doesn't have an edge in the markets. So there's a whole range of things that will, that this chart on the left tells you and of course you then have to go away and investigate it. What is the problem? Is it the edge of the strategy or is it the sample size? What's the issue? Okay. But remember, with the old approach, you wouldn't know that. All that you would know is pick the top point, get good results, move forward, trade it in live. But guess what? If you traded this in live, it would almost certainly be break even or lost making, I would say, after you've added charges. Okay. So that's the difference between the two approaches. Makes sense? Yes, it does. And that's fascinating. Viewers should be reminded that I write a lengthy article with every podcast on sub-sex. So I'm looking forward to, I should get you to validate before I send it out. But looking forward to exploring this and writing and maybe that's actually a really great idea and just really delivering as much value as possible. This is a new technique and I want you to get it right. So get me to validate it. I'm sure it will be right. Yes. No, no, 100%. Let's do that. That sounds great. Really cool. So let me just quickly tell you where I'm kind of going with this. Okay. So in a nutshell, what does Walk Forward correlation do? It gives you that gate. It gives you the go or no go gate. Okay. And it will, I can guarantee it will save traders so much pain and money because you continually hear the stories where traders say, it performed great. Even out of sample validation, it performed really well. I put it live and it lost money. What's going on? This gives you the answer. It tells you what's going on. You just got lucky. You just got lucky that you chose a point that happened to perform well on out of sample data. But there was no real edge there. Okay. So it tells you what form correlation tells you have I overfitted my model and it tells you does my strategy have a discernible edge that I can then make valid decisions on to choose parameters that will work in out of sample and will work in the live account. That's what it does in a nutshell. Okay. So I know you're going to put a link to this Simon in your article. But so this preliminary research paper, I developed a paper as an independent researcher and this is published on a research website called SSRN. I've also put some kind of dialogue around this on my own site, which I've got the link for here. So it'd be great if people could take a look at that to get more information. But interestingly, to really take this whole approach further forward, I'm now working alongside the economics department of a really prominent UK university and working with the research team to really expand the scope of what World Forward correlation does. So we've been working on this now for some time and this is going to get published in a really prominent quantitative finance journal. So we jointly believe that this has huge potential and a Simon set. You heard it here first. So we truly believe this will become the, if you like, the de facto methodology that certainly once will use because we just believe that they will build this in as the new approach. But also in terms of the retail space because I'm sure many of your viewers are retail traders as well. One thing to bear in mind, the software that, as you know already Simon because you've taken a look at it, but the software that I've been showing you is a software package predominantly that I developed for myself, for my own trading strategy development. What I do then also make that available to other retail traders and, interestingly, this Walk Forward correlation technique that until now I've been doing manually. So basically, you know, through a combination of Python scripts and everything else, I've been doing this, this research manually, that will be built into that software, which makes this approach really, really accessible to retail traders. So just wanted to kind of make that point, you know, this isn't just for the quant space. I'm trying my best to make this accessible and accessible technique to help retail traders be better traders and to help turn that corner to become profitable because too many retail traders lose money, that's a fact. Yeah, 100% just for the technical help. Yeah, right. Well, that was definitely my next question, looking at the complexity of that exercise and asking how you did that in your software. So you're building that out of sample, in sample correlation analysis into your software. So this, the software that you've got, which by the way, I've been chatting to Martin for a while about this stuff. So he's happy to offer members, my members, a discount on this software. So we'll set that up as well. But tell links below, I guess. Tell me just a bit more about the software in general, Martin, you know, what does it do now? And obviously it'll be doing this forward correlation stuff in future. Give me a bit of an overview of what it does and how it works. Sure. Okay. So, I mean, I've shown different elements of this as we've gone through through the different techniques really. But, you know, we looked at in the previous, well, we looked at this ability to filter, filter out these random events. So, so the whole concept here is that this is a post optimization process. Okay. So at the moment, the trading platforms that I work with are just Metatrader, MT4 and Metatrader, MT5. Okay. I'm also working on development right now for Ninja Trader. And I am completely open to support any other platforms that have an optimization capability in the future. So the whole premise of this software is that you run your optimization in the platform where that you currently use. The information then gets sucked into the optimized strategy platform where you perform the analysis. So you run the optimization in your trading platform, but you don't analyze the data in your trading platform. You analyze your data here where the level of sophistication around the data science aspects and the intelligence you can get from that data is probably far superior to any individual trading platform. And remember, this is a decade worth of experience that I put into into this software. So, you know, you have this ability to smooth out that surface by taking away those random non-repeating events to retain what is the genuine edge of the strategy. Okay. To be able to make better decisions. In about three weeks time, by the way, you'll have a button down here just underneath this chart, which will say perform for correlation analysis. Okay. And that's how you will perform what we've just looked at. So that will tell you actually whether it's even worth looking at this chart or not to try and select those parameters. So you also get a whole host of different performance metrics that you can use to perform your analysis. Usually in trading platforms, you decide before you do the optimization what you're going to use. So you decide at the beginning, I'm going to use sharp ratio. And that's all you get. Without my strategy, it allows you to look at your results across all of those different metrics just from a single optimization. So you get the full picture. And as I said before, you know, you get this ability to use the combined metric as well. Okay. Now this table at the top, the smaller table, it's small for a reason. And that's just because it's there for comparison. You should never use this table. This table is the results that were taken directly from your trading platform. So these are the results, for example, the MT5 produced or MT4 produced. Okay. The table at the bottom is the one that's had the necessary filtering out of news, filtering, looking for those robust areas of the surface for you. So you don't actually need to manually say, is this point close to a clef edge? This does it for you. And it also looks at sample size. Okay. So how statistically significant are each of these results? And you can see those here. You've got the performance, the variance and the sample size. And they are all used to rank the results that you've got. So and as you can see, you know, it's chosen completely different parameters to what the trading platform chose as the best. You know, completely different. And and that's because you just cannot rely. I call them raw results. The results you get from your trading platform are raw. They're just the result, not taking into consideration anything else, not taking into consideration the variance of the surface, the sample size, the performance across multi metrics and so on. The table at the bottom here is what you would actually use to make your decisions. Okay. Because it takes account of all of those. But of course, you'd only do it if the waterfall correlation had said, yes, you've developed a predictive model. So three, which time the waterfall correlation will be that amazing. All right. Martin, gosh, yeah, there's a lot in this across both the shows. Obviously, we needed two episodes to get through it all. There's some very novel stuff in here. And you know, it's hugely interesting. I think my audience is going to find it very fascinating. I again will remind everyone that, you know, they should follow along on substack and read the articles that go along with this. And that's where I will also post, you know, the links to discounts for that software and, you know, to your paper, Martin, to your website and so on. And yeah, some, some, I'm just digesting all of this. So lots to think about. And this is why I love podcasting and being in this position to talk to so many of my trading colleagues doing cutting edge work and learning constantly, which is fantastic. And we get to share this now in the community. So for the Algo Collective Community, our paid members get to do a little bit more with Martin now, because we're going to go into a section that I've run from you, highly, highly proprietary information. I really pressed you because I wanted you to go through your multi-step process or, you know, to walk us through those steps that, you know, you don't normally talk about online of what your sort of building robust strategies process kind of looks like, which is, you know, the ultimate hot topic in systematic trading. So I really appreciate it. But what have I missed, Martin, anything else before we say farewell? I just like to thank you really for the opportunity of coming on the podcast. Anyone who goes to my website that see I will see I have a kind of a mantra that which is to be the best Algo trader I can be, but also to help other Algo traders succeed. You know, and I've got a real passion to help in particular retail traders be more successful. You know, as retail traders, you know, the odds are stacked against us. And so I like to do what I can to help provide tools and I've shown you provide information. So it's such a pleasure to come on to a podcast like your own Simon and be able to share some of that information. So it's been a real pleasure. Thank you for your time. Yeah, it really has. Thanks again, Martin. Yeah, I'm wearing the same boat there. We're both trying to give back and feed something into the community that is very different to a lot of the hype and, you know, nonsense that you can find online about trading. So and they're the kind of people that I'm attracting around me, which is really good. All right. Well, yeah, thanks once again, Martin and and let's talk again soon. Perfect. Thanks, Simon. (upbeat music)