This paper introduces the model confidence set (MCS) and applies it to the selection of models. An MCS is a set of models that is constructed so that it will contain the best model with a given level of confidence. The MCS is in this sense analogous to a confidence interval for a parameter. The MCS acknowledges the limitations of the data; uninformative data yield an MCS with many models whereas informative data yield an MCS with only a few models. The MCS procedure does not assume that a particular model is the true model; in fact, the MCS procedure can be used to compare more general objects, beyond the comparison of models. We apply the MCS procedure to two empirical problems. First, we revisit the inflation forecasting problem posed by Stock and Watson (1999) and compute the MCS for their set of inflation forecasts. Second, we compare a number of Taylor rule regressions and determine the MCS of the best in terms of in-sample likelihood criteria.
JEL classification: C12, C19, C44, C52, C53
Key words: model confidence set, forecasting, model selection, multiple comparisons
The authors thank Joe Romano, Barbara Rossi, Jim Stock, Michael Wolf, and seminar participants at several institutions and the NBER Summer Institute for valuable comments, and Thomas Trimbur for sharing his code for the Baxter-King filter. The Ox language of Doornik (2006) was used to perform the calculations performed hereHansen and Lunde are grateful for financial support from the Danish Research Agency, grant no. 24-00-0363, and thank the Federal Reserve Bank of Atlanta for its support and hospitality during several visits. The Center for Research in Econometric Analysis of Time Series (CREATES) is a research center at Aarhus University, funded by the Danish National Research Foundation. The views expressed here are the authors’ and not necessarily those of the Federal Reserve Bank of Atlanta or the Federal Reserve System. Any remaining errors are the authors’ responsibility.
Please address questions regarding content to Peter R. Hansen (corresponding author), Department of Economics, Stanford University, 579 Serra Mall, Stanford, CA 94305-6072, 650-725-1869, firstname.lastname@example.org, and CREATES; Asger Lunde, Department of Marketing and Statistics, Aarhus School of Business, University of Aarhus, Haslegaardsvej 10, 8210 Aarhus V., Denmark, and CREATES; and James M. Nason, Research Department, Federal Reserve Bank of Atlanta, 1000 Peachtree Street, N.E., Atlanta, GA 30309-4470, 404-498-8891, .
For further information, contact the Public Affairs Department, Federal Reserve Bank of Atlanta, 1000 Peachtree Street, N.E., Atlanta, Georgia 30309-4470, 404-498-8020.