Tree-based conditional portfolio sorts: The relation between past and future stock returns (opens in a new tab)
The question it started with: which of the many variables really tell us something about future stock returns? The answer came from artificial intelligence.
By 2013, research had found more than 300 variables said to predict future stock returns. In his 2011 presidential address to the American Finance Association, John Cochrane called this a “factor zoo” and asked which of them carry information of their own and which merely repeat others. He suspected that answering this would take different methods.
Double sort, as before
Cuts fixed in advance: 25 groups
Tree, using AI
Cuts chosen from the data: 10 groups
Chart: Dr Benjamin Moritz, benjaminmoritz.de/en/cv/
The standard tools of the field could not go far. A portfolio sort splits stocks into groups by one or two variables; with three or more, each group holds too few stocks. A Fama-MacBeth regression copes with many variables but hardly with their interplay: 50 variables with all pairwise interactions need 1,275 coefficients, more than there were stocks in some months.
Tom Zimmermann and I answered in 2013 with decision trees and random forests from machine learning, a branch of artificial intelligence. A tree sorts like a multiple sort but chooses for itself which variable to split on and where. The random forest averages 200 such trees, each grown on a random draw of data and variables, so that no chance pattern passes for a law. It ran on Harvard University’s supercomputer.
Applied to the monthly returns of US stocks since 1963, it showed that almost all of the information in past returns sits in the last six months, often nonlinear and in interaction. A trading strategy built on it reached an information ratio, extra return per unit of risk, about twice that of a Fama-MacBeth regression with interactions; trading costs do not explain the gap. It was among the first papers to read conditional portfolio sorts as machine learning and received the Best Paper Award of the German Finance Association in 2015.
- 2013Presented at the Harvard research seminarSlides (PDF)
- 2014First working paper and extended presentationPaper (PDF) Slides (PDF)
- 2015Best Paper Award at the annual meeting of the German Finance Association
- 2016Cited by Cerniglia, Fabozzi and Kolm in The Journal of Portfolio Management 42(5)Article (opens in a new tab)
- 2017Cited by Mullainathan and Spiess in the Journal of Economic Perspectives 31(2)Article (opens in a new tab)
- 2018Cited by Gu, Kelly and Xiu in Empirical asset pricing via machine learning, later in The Review of Financial Studies 33(5)Paper (opens in a new tab)
- 2019Discussed by Stefan Nagel in the Princeton Lectures in Finance, Machine Learning and Asset PricingDiscussions (opens in a new tab)



