Fed @ Issue banner with Melinda Pitts inset
Melinda Pitts is a research economist and associate policy adviser in the Atlanta Fed's research department. When Things Still Don't Add Up

Concerns often surface about the reliability of data used to determine monetary policy. In the fourth quarter 2006 issue of EconSouth, this column focused on how the real-time payroll employment estimates from the U.S. Bureau of Labor Statistics (BLS) in 2006 were underestimating the growth in employment. Data watchers are again questioning the reliability of BLS payroll estimates, but this time the focus has been on whether employment growth was being overestimated.

Tallying up
To understand how errors in payroll estimates could arise, it helps to understand how the estimates are created. Every month the BLS surveys 160,000 businesses to measure the number of workers on payroll, hours worked, and earnings. The time window for firms to respond is very small, and not all firms report their data in time to be included in the initial report. Thus, the BLS continues to adjust the estimate for any given month for two more months as late responses trickle in.

The sample includes only firms that responded to the survey in both the previous and the current month. No data are included for firms that went out of business during the month. The exclusion of jobs lost from these "firm deaths" is used to partially offset the absence of data on jobs created from "firm births" during the month. However, the relationship between job gains from firm births and job losses from firm deaths is not one-to-one. To extrapolate current net job creation, the BLS uses a statistical model to capture the birth/death relationship over the previous five years.

When is a trend not a trend?
It is this firm birth/death model adjustment that has attracted attention in the past year. Until 2007, the share of employment growth from net job creation had been relatively small and stable. From 2003 through 2006, the net birth/death adjustment accounted for 42.5 percent, on average, of the growth in the payroll estimate. But in 2007, net job creation accounted for 75 percent of the growth.

While the estimate of job growth derived from the birth/death model continued on an upward trend throughout 2007, the growth in employment in the real-time sample has dropped since April 2006 (see the chart). If this 2007 trend truly reflected changes in the composition of employment growth, then the share of growth attributed to the birth/death model would not be a concern. But the model's reliance on the past five years of data to extrapolate the birth/death adjustment causes current employment growth trends to be dominated by earlier patterns. Thus, the birth/death model continued to indicate the strong employment growth in earlier years while the sample was reflecting the more current, weaker level of activity in 2007.

The goal of the firm birth/death model is to provide a more accurate estimate of U.S. employment levels. Whether this adjustment is beneficial to forecasters is open to debate. From a forecasting perspective, economists are more interested in recent deviations from earlier patterns, so using a methodology that obscures the effect of the most current trends seems counterproductive.

Chart 1
Employment Growth in the BLS Real-Time Payroll Sample and the Firm Birth/Death Model
Employment Growth Chart
Note: Data do not reflect February 2008 benchmark revisions.
Source: U.S. Bureau of Labor Statistics

A preponderance of the evidence
These questions reinforce the idea that one cannot rely on just one snapshot of the economy when developing forecasts. Concerns about the BLS payroll estimates arose mainly because they seemed out of line with other sources of information on employment. Three of these sources—the BLS Household Survey, the ADP (Automatic Data Processing) National Employment Report, and the Quarterly Census on Employment and Wages (QCEW)—indicated a slower rate of employment growth than the BLS payroll estimates going back to the second half of 2006.

The BLS Household Survey, although not as accurate in terms of levels, tends to pick up changes in trends more quickly than the payroll estimates do. The ADP report, based on a sample of ADP's business clients, was developed to provide another timely payroll estimate. The QCEW, also from the BLS and released six months after the payroll estimates, is an almost complete census of U.S. employment derived from unemployment insurance records.

Yet another data source—the Business Employment Dynamics statistics, derived from the QCEW and also released six months after the payroll estimates—provides information on job gains at expanding and opening firms as well as job losses at contracting and closing firms. Unlike the payroll estimate's birth/death model, the most recent QCEW statistics show no signs that the net contribution of jobs gained from firm births and deaths has increased over time.

All of these alternative sources indicated that the BLS payroll estimate was overstating employment growth. This conclusion was borne out by the annual payroll estimate revisions released in February 2008, when the BLS benchmarked payroll estimates to the QCEW from March 2007. Final adjustments were made to payroll estimates from April 2006 through March 2007, and preliminary adjustments were made to estimates from April 2007 through December 2007. The benchmark is used to correct for such things as sampling error and changes in the seasonal adjustment factor and the birth/death model.

In the February 2008 benchmarking, the seasonally adjusted payroll estimates for 2007 were revised downward by 273,500 jobs, which represent an average of 127,250 jobs added per month in 2007 versus the 150,000 originally reported. Because the revisions are benchmarked to March 2007, it remains to be seen if payroll employment for the remaining three quarters of 2007 will be adjusted even more with the February 2009 benchmark revisions.