Tree-based conditional portfolio sorts: The relation between past and future stock returns
The question it started with: which of the many variables really tell us something about future stock returns? The answer came from artificial intelligence.
By 2013, research had found more than 300 variables said to predict future stock returns. In his 2011 presidential address to the American Finance Association, John Cochrane called this a zoo of factors and asked which of them carry information of their own and which merely repeat others. He suspected that answering this would take different methods.
Double sort, as before
Cuts fixed in advance: 25 groups
Tree, using AI
Cuts chosen from the data: 10 groups
How it works, and what came out
The standard tools of the field could not go far. A portfolio sort splits stocks into groups by one variable, a double sort by two; with three or more, each group holds too few stocks. A Fama-MacBeth regression copes with many variables but hardly with their interplay: 50 variables with all pairwise interactions already need 1,275 coefficients, more than there were stocks in some months.
Tom Zimmermann and I answered in 2013 by bringing decision trees and random forests from machine learning, a branch of artificial intelligence, to the stock market. A tree sorts like a multiple sort, but it chooses for itself which variable to split on and where, over as many levels as the data support. The random forest averages 200 such trees, each grown on a random draw of data and variables, so that no chance pattern passes for a law. It was run on Harvard University’s supercomputer.
Applied to the past monthly returns of US stocks since 1963, the method found that almost all of the information sits in the last six months, often nonlinear and in interaction. A trading strategy built on it reached an information ratio, extra return per unit of risk, about twice that of a Fama-MacBeth regression with interactions; trading costs do not explain the gap. It was the first paper to read conditional portfolio sorts as a machine learning method, and in 2015 it received the Best Paper Award of the German Finance Association.
- 2013Presented at the Harvard research seminarSlides (PDF)
- 2014First working paper and extended presentationPaper (PDF) Slides (PDF)
- 2015Best Paper Award at the annual meeting of the German Finance Association
- 2016Cited by Cerniglia, Fabozzi and Kolm in The Journal of Portfolio Management 42(5)Article
- 2017Cited by Mullainathan and Spiess in the Journal of Economic Perspectives 31(2)Article
- 2018Cited by Gu, Kelly and Xiu in Empirical asset pricing via machine learning, later in The Review of Financial Studies 33(5)Paper
- 2019Discussed by Stefan Nagel in the Princeton Lectures in Finance, Machine Learning and Asset PricingDiscussions