Real vs. Hypothetical

It is stated in more than one place on this site that the generally excellent performance that is exhibited by the Traded Portfolio’s model programs is “hypothetical.” That is the traditional label for results that were never realized with real money, and it’s not an utterly bad choice of words. The meaning may seem to be very clear. Certainly it means that the charts aren’t a history of any actual financial performance of the programs that are offered here, as the programs were only recently created whereas the charts go back much farther in time.

    But anyone who does analyses of the Traded Portfolio kind and distributes the findings and has to label them hypothetical has to worry that some may suppose that the meaning of that simple characterization is that the shown good outcomes are the result of a good measure of wishful thinking, or even that there may have been some fudging of the programs’ specifications so as to maximize the returns over just the shown history, that consequently it would be merely fortuitous if in the real-world future any such happy outcomes were to actually be brought about by the programs. Otherwise, why would there be the need to provide something that certainly sounds like a disclaimer? Or so the reasoning may well go.

    Nothing in this note is to suggest that there is anything wrong with the old saw “past performance is no guarantee of future results”— except of course that it is a truism. But beyond that there are two things about past performance that are worth getting straight. One is whether or not past performance, in the particular way that it is wielded by the analyst, can at least be considered to be an unbiased estimator of future performance. The other is the matter of how often would the program have failed in the past. That is, we might find that the program would have finished with a splendid cumulative return over, say, 50 prior years— if only we had the program back then (we didn’t). But if in spite of that splendid finish there were a couple of really bad decades, that would be of interest to us. Even though we can’t assume that the program would perform badly in two out of the next five decades, we’re nonetheless thereby informed about the risk. That is, in a nutshell, how and why testing a program with historical data can be highly beneficial, if done properly— even though through it all we realize that future performance may deviate substantially from past performance.

    Imposition of the “hypothetical” label may elliptically suggest that “real-money” results are necessarily superior, that investors would be better off throwing their lot in with a good real-money performer. But is it as simple as that? If you’re an investor or advisor, exactly how then do you go about making your choices based on real-money results?

The Shown Results Aren’t Maximized

The basic approach of the Traded Portfolio to testing rules by which equity is to be allocated is discussed here. The asset allocation rule specification that is used on any given day is not determined by any use of data of that day or of subsequent days; it’s always a specification that has been determined using prior data, trailing data, with respect to which the price changes of the given day are “out-of-sample”.

    Out-of-sample testing is the main Traded Portfolio method for supplementing academic research and further qualifying a prospective rule specification. David R. Aronson states some of the advantages of out-of-sample testing in his book Evidence-Based Technical Analysis.

Out-of-sample testing is based on the valid notion that the performance of a data-mined rule, in out-of-sample data, provides an unbiased estimate of the rule’s future performance. Of course, due to sampling variation, the rule may perform differently in the future than it does in its out-of-sample test, but there is no reason to presume it will do worse.

    To reiterate, the out-of-sample data play no role in specifying, in hypothesis-testing parlance, the “data-mined” rule— the specification of the rule is in no way fudged so as to maximize the shown returns, all of which are out-of-sample returns.

Rules for Selecting Real-Money Performers

Now we can in principle do this with any kind of real-money performer but let’s assume at first that the only data that we have are the performance records— nothing about how the performers operate. The first thought might be to select the one that performed best in recent years. But that could be, say, the last three, five or ten years. So right away, we have three different rules for selecting a performer. Or we could say that it’s one rule with three values of the lookback period, the number of trailing years that will be considered.

    And of course it matters which lookback period we pick because we will rank past-performers radically differently with each and the different rankings can’t serve equally well to determine the best performer in future years, not if past performance matters. We could call the lookback period a “nuisance parameter” because we wish that we were not stuck with having to deal with it. So how do we know which lookback period is best? Yes, it sounds insane, but it’s inescapable reality— we just wanted to find the best performer but now we have to first find the best lookback period.

    The task of finding the best lookback period based on its past performance is often complicated by there being an insufficient amount of relevant data. Hardly anyone can keep a job in finance for decades, and it’s not as though financial institutions of any kind can stay the same for lengthy periods of time either. We are likely, particularly if we are seeking an actively managed fund, to run into the problem of there not being enough relevant data— not with a chosen range of lookback periods that extends to ten years which is longer than many such funds have been in existence under current management. With such a data insufficiency we could not begin to refute the null hypothesis, which is the claim that the past records of the real-money performers don’t matter when it comes to predicting their future performance.

    The data insufficiency problem is greatly eased if far shorter lookbacks periods are considered. The Traded Portfolio’s programs are not at all as simple as our example here. However, along with out-of-sample testing and null-hypothesis refutation they too involve a lookback period, looking back up to about a year and a half. That shorter time scale renders the programs’ management of asset allocations quite active, allowing timely responses— tests show that the dot-com collapse and the Lehman Brothers-subprime crisis would have been mainly avoided.

Academic Efforts

A number of academicians have taken up this subject of whether or not it pays to cast your lot with the best past-performers. Currently much of their work can be found by searching Google Scholar using the string “past performance repeat persist fund.”

    You’ll find variations among the studies. One says that there is persistence but it’s mainly due to managers who correctly select industry groups. Another cautions that what persistence there is is just due to some funds charging high fees (yielding persistently worse performance). Others refer to persistence only being present in the negative sense of bad-performing funds alone continuing to perform as in the past. And there is a paper that says that hedge funds had persistent performance only after bear market periods. So in all, the picture is not simple. You have your work cut out for you if you are to make effective use of real-money performance histories of any kind.

Hey Buddy… We’re On the Same Team!

The Traded Portfolio project makes use of real-money performance data! Since the project doesn’t operate a fund it does not have a real-money track record of its own. But presently the project is confined to the use of security price data, including that of funds, as the only inputs— other than categorizations according to value, market-capitalization and industry-group status. And of course the price histories are real-money performance records.

    So if you’re in the business of choosing funds that are invested in particular asset classes and you’re going to base your choices on past real-money performance, that’s truly what tTp now mainly does. And whatever rule that you use to make your choices, whatever lookback period you choose, any results that you project with it will be— brace yourself— “hypothetical.”

    But if you want to farm out the asset allocation responsibility to a fund it would only be among the performers who employ active management that you could hope to find a truly outstanding one. Active management is often rule-based. And with rules the question is not if the managers are good but is instead whether or not the rules are good.

    Using the methods of tTp the rules can be tested not only on securities held by a fund of interest that uses the rules but also on other securities of the same asset classes as those of the fund, using security price records that extend back in time well before the startup date of the fund. Testing the rules in that way is a better way to decide if the fund should be invested in, or emulated— better than trying to simply make use of the fund’s limited performance history.