COVID-19 RESOURCES AND INFORMATION: See the Atlanta Fed's list of publications, information, and resources; listen to our Pandemic Response webinar series.

About


The Atlanta Fed's macroblog provides commentary and analysis on economic topics including monetary policy, macroeconomic developments, inflation, labor economics, and financial issues.

Authors for macroblog are Dave Altig, John Robertson, and other Atlanta Fed economists and researchers.

Comment Standards:
Comments are moderated and will not appear until the moderator has approved them.

Please submit appropriate comments. Inappropriate comments include content that is abusive, harassing, or threatening; obscene, vulgar, or profane; an attack of a personal nature; or overtly political.

In addition, no off-topic remarks or spam is permitted.

February 7, 2017

Net Exports Continue to Bedevil GDPNow

Real gross domestic product (GDP) grew at an annualized rate of 1.9 percent in the fourth quarter, according to the advance estimate from the U.S. Bureau of Economic Analysis (BEA), 1.0 percentage point below the Atlanta Fed's final GDPNow model projection. This was a sizable miss relative to other forecasts. Both the consensus estimate from the January Wall Street Journal Economic Forecasting Survey and the January 20 staff nowcast from the New York Fed were expecting 2.1 percent growth last quarter.

The miss was also large relative to the historical accuracy of the GDPNow model. As the table below shows, almost all of GDPNow's error for fourth quarter growth was concentrated in real net exports. For the other broad subcomponents, GDPNow was more accurate than usual, as the last two columns of the table show. But net exports subtracted 1.70 percentage points from real GDP growth last quarter, whereas GDPNow forecasted they would only reduce growth by 0.64 percentage points. All but 0.02 percentage points of this error was in the "goods" category as opposed to services.

Three months ago, I wrote a macroblog post showing that nearly all of GDPNow's 0.8 percentage point error for third-quarter growth was concentrated in goods net exports. That analysis explained how GDPNow's goods net exports forecast is a weighted average of two forecasts. One of these forecasts is a "bean counting" model that uses monthly source data on nominal values and price deflators for goods imports and exports. The other is a quarterly econometric model that uses subcomponents of real GDP for prior quarters. In the GDPNow model, the "bean counting" model gets nearly 60 percent of the weight just before the advance GDP release.

To see how this approach matters for the GDP forecast, the following chart shows the "real-time" forecasts of the contribution of goods net exports to growth just before BEA's advance GDP estimate from the two models alongside the advance estimate of the contribution and the final GDPNow forecast.

We see that the "bean counting" forecast has been much more accurate than the quarterly econometric forecast, particularly for the last two quarters of 2016. Not surprisingly given its name, the "bean counting" model was able to largely capture the 0.75 percentage points that soybean exports contributed to third-quarter real GDP growth and the just over 0.5 percentage points they likely subtracted from fourth-quarter growth. The econometric model was not.

The final forecasts of goods net exports from the "bean counting" model have also been more accurate than GDPNow since forecasts were first posted online in mid-2014. Does this imply that an alternative "bean counting" version of GDPNow would be preferable? The answer is less obvious than you might think. Not putting any weight on the quarterly econometric model for any GDP subcomponents yields an average error for GDP growth (without regard to sign) of 0.635 percentage points, and the same statistic for GDPNow is 0.589 percentage points. This is despite the fact that the "bean counting" approach has been more accurate than GDPNow in its forecasts of net exports and about as accurate, on balance, for the other GDP subcomponents.

The final forecast of real GDP growth last quarter of this alternative "bean counting" model was 2.8 percent—only slightly more accurate than GDPNow. (For each GDP subcomponent, I include the "bean counting" and quarterly econometric model forecasts in this excel spreadsheet.)

However, if variants like the aforementioned "bean counting" approach continue to outperform the GDPNow model in one or more dimensions, we may consider regularly reporting their forecasts along with the GDPNow forecast.

December 5, 2016

Using Judgment in Forecasting: Does It Matter?

​​

Many professional forecasters use statistical models when making their near-term projections for real gross domestic product (GDP) growth. A 2013 special survey on the forecasting methods of the Survey of Professional Forecasters  found that 18 out of 21 respondents featured a statistical model prominently in their current-quarter economic projections. Nevertheless, there is fairly compelling evidence that many professional forecasters incorporate judgment in their forecasts of the first estimate of real GDP growth for a quarter—even when much of the source data used to construct the GDP estimate are available.

In the October 2016 Wall Street Journal Economic Forecasting Survey (WSJ), the most common panelist projection for annualized third-quarter real GDP growth was 2.5 percent, and the second most common one was 3.0 percent. The first digit after the decimal point, or tenths digit, of these two numbers are "5" and "0." Of the 58 individual forecasts of third-quarter growth in the survey, 21 had a tenths digit of "0" or "5," a total that is almost twice as large as we would expect if all tenths digits were equally likely to be submitted.

This pattern isn't unique to the most recent quarter's GDP forecast. The following chart shows the historical frequency of the tenths digit in past WSJ surveys for first estimates of real GDP growth over the period from the first quarter of 2003 to the third quarter of 2016, made about three weeks before the release.

Almost 40 percent of these 2,390 forecasts have a tenths digit of "0" or "5." In contrast, the historical distribution of published first estimates of real GDP growth from the fourth quarter of 1991 to the third quarter of 2016 and real gross national product (the most common measure of U.S. production in an earlier era) growth from the third quarter of 1965 to the third quarter of 1991 has a tenths digit of either "0" or "5" only 18 percent of the time. The historical Atlanta Fed's GDPNow forecasts have a "0" or a "5" tenths digit only 15 percent of the time.

More formally, one easily can reject the hypothesis at the 1 percent significance level that the tenths digit of the WSJ panelist forecasts are either uniformly distributed or follow the Benford distribution for tenths digits after rounding to the nearest tenth (see this paper by economists Stefan Gunnel and Karl-Heinz Todter, who found similar relative frequencies of "0s" and "5s" in professional forecasts of German GDP growth and consumer price index inflation).

If we assume that near-term GDP growth forecasts with a tenths digit of "0" or "5" typically involve more judgment than forecasts with another tenths digit, a natural question is whether these more judgmental forecasts are less accurate than others. Of the 2,390 WSJ growth forecasts mentioned above, the ones with a tenths digit of "0" or "5" (after rounding to the nearest tenth) had an average error of 0.786 percentage points without regard to sign, and the others had an average error of 0.743 percentage points. These accuracy metrics are not statistically different at even the 10 percent significance level. Moreover, because of the panel nature of WSJ forecasts, we can measure how often a forecaster has a tenths digit of "0" or "5" (after rounding). Of the 44 panelists who submitted at least 30 three-week-ahead GDP forecasts during the period of the first quarter of 2003 through the third quarter of 2016, the correlation of the panelists "0" or "5" tenth digit frequency and their average error without regard to sign is only 0.13 and not significantly different from 0.

Although at least some professional forecasters appear to make judgmental adjustments to their near-term GDP projections, the evidence presented here does not suggest it comes, on average, at the cost of accuracy.

November 7, 2016

The Price Isn't Right: On GDPNow's Third Quarter Miss

The U.S. Bureau of Economic Analysis's (BEA) first estimate of third quarter annualized real gross domestic product (GDP) growth released on October 28 was 2.9 percent. A number of nowcasts were quite close to this number, including the median forecast of 3.0 percent from the CNBC Rapid Update surveyOff-site link of roughly 10 economists. The Atlanta Fed's GDPNow model forecast of 2.1 percent? Not so close.

What accounted for GDPNow's miss? The table below shows the GDPNow forecasts and BEA estimates of the percentage point contributions to third quarter growth of six subcomponents that together make up real GDP. The largest forecast error, both in absolute terms and relative to the historical accuracy of the projections, was for the contribution of real net exports to growth. The published contribution of 0.83 percentage points was much higher than the model's estimate of 0.07 percentage points.

Why did GDPNow miss so badly on net exports? For both goods and services real net exports, the GDPNow forecast is a weighted average of two forecasts. The "bean counting" forecast uses the monthly source data on the nominal values and price deflators of exports and imports. The econometric model forecast uses published values of 13 subcomponents of real GDP for the last five quarters to predict real net exports for both goods and services. The statistically determined weights on the bean counting forecast increase as we get closer to the first GDP release and accumulate more monthly source data. (More details are provided here.)

For real net exports of services, 89 percent of the weight was given to the bean counting forecast. This weighting worked out well last quarter as the forecasts of the contribution of real services net exports to third quarter growth from both the bean counting and combined models were within 0.01 percentage points of the published value of −0.14 percentage points. But for real net exports of goods, the bean counting forecast received only 59 percent of the weight in the final GDPNow forecast. It projected that real net exports of goods would add 0.76 percentage points to growth—reasonably close to the BEA estimate. In contrast, the econometric model projected a subtraction of 0.58 percentage points from growth.

Since GDPNow had monthly price and nominal spending data through September on goods imports and exports, why didn't it place more weight on the source data? One of the important reasons is that it's difficult to match the quarterly inflation rate of the BEA's import price deflator for goods. The BEA constructs its price deflator  with detailed price indices from the Bureau of Labor Statistics (BLS) producer price index and import/export price index programs as well as a few other sources. GDPNow uses the BLS's import price data at higher levels of aggregation than the BEA uses and also differs in the manner that it handles seasonality. The chart below plots the difference between the BEA's quarterly goods import price inflation rate and the GDPNow proxy. These inflation measures have differed by 5 percentage points or more on a number of occasions. Since goods imports are 12 percent of GDP, a miss of this magnitude on the price deflator would lead to a miss on the real net exports of goods contribution to growth of 0.5 percentage points or more, even if the other ingredients in the calculation were all correct.

Are there any lessons here for improving GDPNow? Ideally, GDPNow would be able to closely map the monthly source data to real goods net exports so that most of the weight would go to the bean counting forecast once all of the data are in—much as it does with nonresidential structures and residential investment. The BEA's estimates of real petroleum imports are based on similar data in the monthly international trade data publication. Because petroleum imports account for so much of the volatility of inflation for goods imports, it may be better to use the monthly real petroleum imports data directly and only worry about replicating the price index for nonpetroleum goods.

That said, a previous macroblog post illustrated that the method GDPNow currently uses has a reasonable forecasting track record for net exports when compared with several consensus estimates from professional forecasters. Net exports may remain difficult to nowcast even with refinements to GDPNow's methodology.

GDPNow has established a commendable track record. But sometimes when it misses the mark, an analysis of the error can provide insight into how GDPNow works and the limitations of the model.

May 23, 2016

Can Two Wrongs Make a Right?

In a recent macroblog post, I showed that forecasts from the Atlanta Fed's real gross domestic product (GDP) nowcasting model—GDPNow—have been about as accurate a forecast of the U.S. Bureau of Economic Analysis's (BEA) first estimate of real GDP growth as the consensus from the Wall Street Journal Economic Forecasting Survey. Because GDPNow essentially uses a "bean-counting" approach that tallies the forecasts of the various main subcomponents of GDP, the total GDP forecast error can be broken up into the forecast errors coming from each piece of GDP. For most of the subcomponents of GDP, the contribution to total GDP growth is approximately its real growth rate multiplied by its expenditure share of nominal GDP (the exact formulas are in the working paper for GDPNow). The following chart shows the subcomponent contributions to the GDPNow forecast errors since the third quarter of 2011. (I want to note that the forecast errors are based on the final GDPNow forecasts formed before the BEA's first estimates of GDP are released.)

The forecast errors for the subcomponents can sometimes be quite large. For example, for the fourth quarter of 2013, GDPNow underestimated the combined contributions of net exports and inventory investment by nearly 2 percentage points. However, these misses were nearly offset by overestimates of the other contributions to growth (consumption, business and residential fixed investment, and government spending).

The pattern of large but largely offsetting GDP subcomponent errors has been attributed to the work of a fictional "Saint Offset," as former Fed Governor Laurence Meyer noted in a 1998 speech. Unfortunately, "Saint Offset" doesn't always come to the forecaster's aid. For example, in the fourth quarter of 2011, GDPNow predicted 5.2 percent growth—well above the BEA's first estimate of 2.8 percent—and the subcomponent errors were predominantly on the high side.

A closer look at the chart also reveals that GDPNow has had a tendency to overestimate the contribution of business fixed investment to growth and underestimate the growth contribution of inventory investment. Although these subcomponent biases have nearly offset one another on average, we really don't want to have to rely on "Saint Offset." We would like the subcomponent forecasts to be reasonably accurate because the subcomponents of GDP are of interest in their own right.

Have the subcomponent biases been a unique feature of GDPNow forecasts? It appears not. Both the Survey of Professional of Forecasters (SPF), conducted about 11 weeks prior to the first GDP release, and Blue Chip Economic Indicators, conducted as close as three weeks prior to the first release, provide consensus forecasts for some GDP subcomponents. The following table provides an average forecast error (as a measure of bias) and average absolute forecast error (as a measure of accuracy) of the subcomponent growth contributions for the two surveys and comparably timed GDPNow forecasts.

We see that the biases in GDPNow's subcomponents have been fairly similar to those in the two surveys. For example, all three sources have underestimated the average inventory investment contribution to growth by fairly similar magnitudes.

The relative accuracy of GDPNow's subcomponent and overall GDP forecasts has also been similar to the accuracy of the two surveys. "Saint Offset" has helped all three forecasters; the standard errors of the real GDP forecasts are 20 percent to 40 percent lower than they would be if the forecast errors of the subcomponents did not cancel each other out.

Finally, notice that some GDP subcomponents appear to be much more difficult to forecast than others. For instance, the bias and accuracy metrics for consumer spending are smaller than they are for inventory investment. This differential is not really that surprising, because more monthly source data are available prior to the first GDP release for consumer spending than for inventory investment.

Can we take any comfort in knowing that private forecasters have mirrored the biases in GDPNow's subcomponent forecasts? An optimistic interpretation is that the string of one-sided misses are the result of bad luck—an atypical sequence of shocks that neither GDPNow nor private forecasters could account for. A more troubling interpretation is that there have been structural changes in the economy that neither GDPNow nor the consensus of private forecasters have identified. Irrespective of the reason, though, optimal forecasts should be unbiased. If biases in some of the subcomponents continue, then forecasters will need to look for a robust way to eliminate them.