Abstract
This paper proposes a simple back testing procedure that is shown to dramatically improve a panel data model's ability to produce out of sample forecasts. Here the procedure is used to forecast mutual fund alphas. Using monthly data with an OLS model it has been difficult to consistently predict which portfolio managers will produce above market returns for their investors. This paper provides empirical evidence that sorting on the estimated alphas populates the top and bottom deciles not with the best and worst funds, but with those having the greatest estimation error. This problem can be attenuated by back testing the statistical model fund by fund. The back test used here requires a statistical model to exhibit some past predictive success for a particular fund before it is allowed to make predictions about that fund in the current period. Another estimation problem concerns the use of a single statistical model for all available mutual funds. Since mutual funds often, but not always, employ dynamic trading strategies their betas move over time in ways that differ from fund to fund. Since no one statistical model is likely to fit every fund, the result is a great deal of misspecification error. This paper shows that the combined use of an OLS and Kalman filter model increases the number of funds with predictable out of sample alphas by about 60%. Overall, a strategy that uses very modest ex-ante filters to eliminate funds whose parameters likely derive primarily from estimation errors produces an out of sample risk adjusted return of over 4% per annum.