How to define "what works" and "what doesn’t work"


Before we get into investing strategies that work, we need to objectively define what counts as “works” and what counts as “doesn’t work”. In investing as in life, only after accurately defining a problem can you solve it.
We use 2 criteria to define “what works” in the stock market.

#1: A benchmark

We need a benchmark against which we can compare various trading strategies. This benchmark becomes an OBJECTIVE percentage definition of “success”.
You can see this all the time online. “Gurus” tout their trading strategy that’s “profitable” in the stock market. “If you follow my strategy, I made $10k on my last trade!”  Then you see that their trade had a $1 million position size, which means that $10k was 1% (i.e. chicken scratch).
In the markets, absolute numbers mean nothing. What matters is the % profit.
As U.S. stock market investors, our benchmark should be “buy and hold the S&P 500”. Why? Because even someone who knows nothing about the stock market can do that. If an investor can’t even beat buy and hold over the long run, then he is wasting his time. What he is doing “doesn’t work”
For example, making an average of 5% a year trading currencies over the past 20 years when the S&P yields 7% is not “success”. It’s failure when looked through the objective lense of this benchmark.

#2: Focus on long term thinking and long term performance

One of the biggest problems with the investment industry is that everyone is so fixated on the short term results. This is a rather unique problem to the investment industry, probably because the wins and losses each day are so clear.
Instead, traders and investors should focus on long term performance.
This means that a strategy needs to work over the long term. I.e. a few years of benchmark-beating returns doesn’t mean anything. It could just be due to luck.
For example, everyone using technical analysis felt like a genius trading Bitcoin in 2017. But that’s because Bitcoin was in a bull market! It didn’t matter how you traded – you could have rolled a dice, and you still would have made money trading Bitcoin. But once Bitcoin entered into a bear market in 2018, it became clear that technical analysis for Bitcoin was not much better than rolling a dice.
So how long is “long enough”? How many years of backtesting do you need for a strategy before you can decide if it “works” or “doesn’t work”?
At least 30 years. That gives you plenty of bull markets, bear markets, economic expansion, recessions, etc.
Strategies need to be stress tested for various environments. That’s why the more data, the better.
Long Term Capital Management was one of the most famous hedge fund implosions during the 1990s (for a good read, see When Genius Failed). Here’s a brief summary:

  1. Long Term Capital Management was founded by geniuses. Literally geniuses. One of LTCM’s founders co-invented the standard options pricing model that everyone uses today. He won a Nobel prize for that.
  2. Long Term Capital Management had 4 years of exceptional returns from 1994 – 1998 with very little volatility.
  3. They blew up and lost everything in 1998.
  4. What happened?

Long Term Capital Management was a quant fund. They traded based on probability.
Their mistake was that they used LIMITED DATA when calculating their probability. This resulted in them underestimating the probability that the market would go down.

  1. For example, using data from 1994 – 1998, LTCM said “the probability of the market going down e.g. -10% is 2%”
  2. This probability is clearly inaccurate. 1994 – 1998 was a period of low volatility. Anyone who used more historical data could have seen that the “the probability of the market going down -10% is much higher than 2%”.

Using limited data is one of the biggest lessons that investors and traders should have learned from the 2008 crash. Never use limited data or limited time frame charts because if you do, you are not accounting for the full range of possibilities and probabilities. Always use as much data as possible and look at the data HOLISTICALLY. Don’t cherry-pick cases that match your existing market outlook.

Parallels between trading and business

I’d like to draw some parallels between trading and business.
No successful business focuses excessively on how much money they make day to day. In fact, the greatest businesses don’t even focus on quarter-to-quarter results. They focus on their long term results.
For example, look at Amazon. Amazon has an EXTREMELY LONG TERM mentality when building its business. It is arguably one of the most successful companies in the world right now. (Whether or not Amazon will crash in the next bear market is one question. But even if Amazon’s stock falls 90%, it’ll still be one of the biggest companies in the world.)
Like successful businesses, successful investors and traders shouldn’t focus on short term results. Successful investors who focus on short term results will forever have Shiny Object Syndrome. They will forever chase the hottest strategy AT THE MOMENT. Trading strategies are like everything else in life: they have ups and downs. People who chase the Shiny-Strategy-Of-The-Day will start using a strategy AFTER it has outperformed for a long time, just before it starts to underperform. Then they will switch to the next Shiny-Strategy-Of-The-Day and start using the new strategy AFTER it has outperformed for a long time, just before it starts to underperform. These people forever chase yesterday’s hottest strategy.
The following chart demonstrates how every single trading/investment strategy has its ups and downs. It fluctuates around the strategy’s “true mean” (average annual return).

This means that judging a strategy based on just a few years of performance is myopic. You might just be judging the strategy right now based on a period of relative outperformance. You might be judging the strategy based on a period of relative underperformance. Hence you will only see a small part of the big picture.
So if a model works great in backtesting, and then the first year you use it loses money or underperforms, don’t toss it aside. GIVE IT SOME TIME to adjust to the historical mean. Realizes that losses and underperformance are a part of the game. By definition, that’s how “averages” work. Averages = there are periods of underperformance and there are periods of outperformance.
Here’s another equally important point.
You need to follow a strategy (whether it be a quantitative model or discretionary strategy) from beginning to end. When you first start to use a strategy has a big impact on your initial profits:

  1. If you start to use a strategy during a period of outperformance, initial returns will appear terrific. This is misleading.
  2. If you start to use a strategy during a period of underperformance, initial returns will appear terrible. This is misleading.

Unfortunately, it’s hard to know without 20/20 hindsight in what phase of the strategy’s ups and downs you started using it in. That’s why you need to find something that works and stick to it.
So why is long term thinking hard? Because it goes against human nature. Human nature wants instant gratification. This is from Ray Dalio’s book Principles

“All great investors and investment approaches have bad patches. Losing faith in them at such tunes is as common a mistake as getting too enamoured of them when they do well. Because most people are more emotional than logical, they tend to overreact to short term results. They sell at lows when times are bad and buy too high when times are good. I find this is just as true for relationships as it is for investments – wise people stick with sound fundamentals through the ups and downs, while flighty people react emotionally to how things feel, jumping into rings when they’re hit and abandoning them when they’re not”

Other factors to consider

Of course there are other factors to consider when deciding if a strategy “works” or “doesn’t work”. This includes looking at the strategy’s drawdowns, risk:reward ratio, etc.
However, factors such as a strategy’s drawdowns are a matter of personal preferance and pain tolerance. But the 2 timeless and universal criteria are the ones I’ve mentioned, regardless of your personal preferance:

  1. A benchmark
  2. Focus on long term performance.

In later parts of this Membership Program I will show you how to know if a strategy isn’t working as it should. In other words, how much time is TOO MUCH time when giving a strategy “enough time” to work itself out to its true average performance.

Leave a Comment