Economy Matters podcast series

Tom Heintjes: Welcome to another Economy Matters podcast episode. I'm Tom Heintjes, managing editor of the Atlanta Fed's Economy Matters magazine. I'm really glad you're here today—I've been looking forward to this conversation for a while now, because today we're joined by Brent Meyer, an economist here at the Atlanta Fed. Among the numerous projects Brent is involved with, he's one of the people behind our Survey of Business Uncertainty, and he’s agreed to sit down with me today and discuss the SBU. Brent, thanks for joining us.

Brent Meyer: Thanks for having me. How's it going?

Heintjes: Well, fine. I want to talk to you first off about the initial concept behind the Survey of Business Uncertainty. What was the impetus behind its creation?

Meyer: We're really interested in what drives businesses to make the decisions that they make. It's a crucial component to the formation of sound monetary policy, and it's actually something that we’ve been pretty interested in here at the Atlanta Fed for quite some time, with the development of our REIN network, our Regional Economic Information Network. So this sort of dovetails with that, in trying to get at—really, what are businesses reacting to?

Heintjes: So obviously, then, the concept of this survey has been in the works for quite some time before its debut here at the Atlanta Fed?

Meyer:Yes, and the idea behind the survey really is to track the uncertainty that businesses perceive in their external environments, and the uncertainty they perceive about their own future outcomes—say, for hiring or capex [capital expenditures]. For example, there can be discrete events—like 9/11, like Brexit, or the financial crisis, or something maybe even more mundane like changes in trade policy—and that can profoundly alter the outlook for decision makers and impact their business plans going forward.

Heintjes: Right. Well, we’re going to get into some of those things, so don’t get ahead of me here, but I wanted to ask you about how you actually constructed such a survey. I imagine it was quite complex. What went into building it? And also, I want to ask you who else is involved in it? I know it’s not just you.

Meyer: Yes. So in order to answer that question, it's kind of a little story. The story starts with our Business Inflation Expectations survey, which [former Atlanta Fed economist] Mike Bryan started back in 2010. Nick Parker and myself helped develop that survey, and by 2013, when that rolled around, we were approached by a couple of preeminent economists—Nick Bloom from Stanford and Steve Davis from the University of Chicago Booth School—and they were really interested in the methods we were using on our Business Inflation Expectation Survey, and they wanted to expand on that. So right now it's really a rather large group of researchers—including those two, and then we added one of Nick Bloom's PhD students at the time, who is José Maria Barreiro. He's been a tremendous addition to the team. And really making this whole thing run and happen is our survey director Nick Parker, who manages the panel, cultivates the survey, and leads a team of undergraduate interns in our recruiting efforts. I mean, it is really complex.

Heintjes: I guess it truly does take a village to do this kind of thing.

Meyer: Honestly, it really does. And think about where we're at now—it's the middle of 2019, and we're just really starting to socialize our Survey of Business Uncertainty with the public. We really started testing and cognitive interviews—these things—back in late 2013, early 2014, and we’ve been refining our survey for the last five years or so.

Heintjes: Wow. Well, you mentioned that a couple of your economist peers reached out to you about developing this sort of survey, so let me ask you: What informational need does the SBU fill that wasn't being filled by other existing measures? In other words, what makes it distinct?

Meyer: When you’re thinking about uncertainty, there are a couple of different ways to measure that and a couple of different approaches that economists have taken in the past. One is just looking at professional forecasters and how much they disagree with each other. When there’s a lot of disagreement, that can be a proxy for living in an uncertain time. Another measure—which, actually, Nick Bloom and Steve Davis put together along with another researcher—uses newspapers and news mentions of things like policy uncertainty or trade uncertainty to use as a proxy for trying to gauge the uncertainty that businesses are facing.

And there are other survey methods that try to get at this in a certain way, but really the niche in this field that the Survey of Business Uncertainty tries to get at it—what we’re really doing is we're really providing this innovative procedure and methodology for eliciting subjective probability distributions. And we're not focusing on aggregate expectations like, say, for GDP [gross domestic product] growth or the S&P 500. We’re really focusing on things that we think firms care about: their own outcomes for employment, for capital spending, for sales revenue. And this we do on a monthly basis. We have broad coverage of the U.S. nonfarm private sector. And in addition to these core questions that really try to get at not only a firm’s past and current outcomes with regard to employment, capex, and sales revenue. We're also looking at their probability distributions or their expected outcomes.

Heintjes: We recently published a research paper by you and some of your colleagues about the Survey of Business Uncertainty. And in your paper you note—and I’m quoting you here—“uncertainty is a fact of economic life,” which we all know to be true. So what led you to think you could provide clarity on a topic with a certain inherent degree of uncertainty?

Meyer: Well, uncertainty is a tough topic to grapple with—especially for households and businesses, really, in today's fast-moving global environment.

Heintjes: You say that with such certainty.

Meyer: [laughs] All you have to do is have a chat with one of these guys, right? So we do this a lot in our policy preparations where we're talking with contacts, business contacts in our district, and it became very clear, at least over time, in just chatting with firms that uncertainty and certain discrete events, and how they’re going to impact their business, is something that weighs on their minds on a daily basis—probably an hourly basis. So we're hoping that the methods that we’ve developed—and again, over more than five years of testing—can accurately measure and make use of firms' expectations and uncertainty to better understand how businesses behave in certain environments and whether or not we can use these findings to help formulate better monetary policy.

Heintjes: Well, you just said something that leads me nicely into my next question: Without getting too into the weeds—you mentioned the methods behind the survey. Can you briefly describe the methodology behind it?

Meyer: Yes. And if you want, I can go as deep into the weeds, or the swamp, as you want to venture with me.

Heintjes: I like to stay in the shallow end.

Meyer: All right. So staying in the shallow end, what we’re really trying to do is—let's take a specific topic, say employment. We’re trying to elicit basically their expectations for employment over the next year ahead, let’s say. And we think that firms form these expectations in a way that we could get them to respond to a question that says, "Okay, what's your middle case for your employment levels over the year ahead? How about the lowest case—maybe a low case—or a high case, and the highest case?" So we have them fill out basically five support points in the expectational space, from lowest to highest expected outcomes, and then we have them assign probabilities to those outcomes. And we do this not only for employment, but we also do this for capital investment and sales revenue.

And then there's a sort of complicated process. Basically, we try to make sure all these things are standardized in a certain way, and then we aggregate it, we add them up across the different indices. And what we're really getting out of this is an index of business expectations, based on their answers to those subcomponents, and we're also getting an index of uncertainty—again, based on how they've responded to those previous questions.

Heintjes: So what is a typical sample size that we're talking about here?

Meyer: Well, sample size has been growing. We have a panel that's over 1,000 firms now, probably 1,250 to 1,300 firms, depending on how well our interns have been doing, and typically we'll get around 450 or so responses per month.

Heintjes: Got it—that's a good size. You know, Brent, obviously everyone—individuals and firms—likes some measure of certainty, and businesses value certainty a great deal. But I want to ask you: Is there a commonly accepted notion of "certainty" that you see across firms and industries, as far as certainty about prices, supply chains, et cetera?

Meyer: Yes. So let me start with this answer: I think that it's true that everyone likes certainty, and I think that that's particularly true for firms—especially when it comes to profitability in an accounting sense. The more certain an environment firms are operating in, the more likely they are to hit their profit targets—the happier they are and the easier they sleep, right? Less moving pieces, and fewer unknowns. So basically when the economic environment that these firms are operating in is stable, they're pretty happy. When uncertainty spikes and that environment becomes less stable, this is often described as pushing firms to the sidelines, in terms of decision making. An economist might think about this as providing some sort of option value to waiting, basically, until the smoke clears. Until you get some certainty, you kind of sit on the sidelines. Now, can we say this operates in the same way for all firms, over all dimensions? No, I don't think we can say that. But I suspect that there are differences between maybe small and large firms, or regional firms and global firms, or across different industries. I suspect that there are differences along these firm characteristics in how they process and actually react to uncertainty, and it's something we're working on, trying to understand.

Heintjes: It's interesting you say that, because I wanted to ask you—a survey heavy on manufacturing might give you a different perspective from a survey heavy on healthcare, for example.

Meyer: Yes, absolutely. What we're really attempting to do is provide a sample that roughly matches the nation, the population of businesses out there. So we're really trying to work on—in our recruiting methods and retention methods—getting a broad, diverse, nationally representative sample.

Heintjes: So to try to reconcile all these disparate viewpoints into one "snapshot of uncertainty" —that's I guess where the composition of the survey respondents comes in?

Meyer: Yes, absolutely. And ex post, there could be some weighting techniques that we could use if we find that we're missing the mark. But at least right now it looks like we have a broad—relatively broad—nationally representative sample. So imagine you've overweighted the mining sector, and we get a big spike in oil prices. Well, our survey might end up saying that uncertainty has increased quite a bit and expectations have dropped, even though two-thirds of firms in the U.S., at least, are services and probably react much less strongly to a spike in, say, oil prices. So it's something that we care about, and we work diligently to try and make our sample representative.

Heintjes: You mentioned a few minutes ago that the survey looks at past and present levels of uncertainty to attempt to look ahead over the horizon—or at least toward the horizon. How does this sort of forecasting work, and what window does the SBU attempt to forecast?

Meyer: Most of our questions are in a year-ahead format—so say, over the next 12 months or next four quarters. What we're really trying to do is see if whether or not firms—by the way they respond in aggregate, looking at our sample—say their employment expectations are increasing. Does that correspond to the employment outcomes that we would see a year from now? That's one of the ways we’re trying to understand this survey, and some of our work points to having this predictability, or predictive content, in these expectations for future macro variables. So that's something we point out in our paper. The other thing that we point out is, firms tend to know what they don't know—or firms that are in uncertain environments, they respond with a very diffuse probability distribution, and they tend to have higher forecasting errors going forward. So this is really an area of research that we’re trying to understand a bit more about.

Heintjes: Your survey includes what you call a "special question." What sort of information do these special questions attempt to capture, and how do you vary the special questions?

Meyer: Special questions are really just that: they're one-off. Sometimes they're just really special topics. They either allow us an opportunity to get the probabilistic expectations or uncertainties about other variables outside of our core questions. So maybe one month we want to really understand more about pricing, for example. Then we would ask a specific question on pricing. But more often than not, these special questions give us insight into policy-relevant topics. So imagine we can go back to the end of 2017, when the ink was just drying on the Tax Cut and Jobs Act bill. We were in the field trying to figure out how firms were responding to that. Are they making changes at the margin on how they're planning to invest in the year ahead? Or when trade policy started changing and trade policy uncertainty spiked—how were firms reacting to that, especially in regards to capital expenditures? There are really two types of special questions that we can employ: one that helps us understand firm decision making on a more academic level, and others that maybe better inform our policy narrative—so really trying to understand the current situation—and it's just a balancing act between the two, of which way we’re going to go.

Heintjes: You mentioned trade policy, and of course I can't let you leave here today without talking about tariffs and trade policy. There was a lot of talk about uncertainty when tariffs and trade wars entered the headlines. Did you see this sort of topic move the needle in terms of what you were observing in the survey?

Meyer: So this is a great question. We’ve asked now special questions on the topic, and we just finished up our third foray into this—actually, we're in the midst of our third foray into this—special question on tariffs and trade tensions and how they’re impacting capital investment. And what we’ve found, at least initially over the first two forays into this, is that these trade policy changes and tensions with other countries—the increase in tariffs—have had a modest impact—not a huge impact, just a modest impact—on firms' capital expenditure expectations. And it’s something that we can see in our subcomponent on capital expenditures.

And even now, even when we've seen the trade policy uncertainty as measured by the number of news mentions—so in the newspaper, trade policy and uncertainty have spiked—we're out there. And we don't necessarily see increasing or intensifying impact on capex, as far as firm expectations are concerned. So their forward-looking capital expenditure expectations are still in a favorable spot. Uncertainty has actually gone down in our July survey for capital expenditures, and our special question isn't really picking up any more—or a more aggressive, or more impactful—effect from trade tensions and tariffs on capex right now.

Heintjes: Interesting. It's interesting to separate the survey results from the headlines we see. Well, Brent, I know the art and science, if you will, of surveying involves ongoing refinements, but I want to ask you: How has the survey changed since you first began conducting it, and do you have a sense of how it might change more down the road?

Meyer: Yes. So the survey has changed a lot, and that's part of the reason why we've spent five years refining it. When we started out, the way we were asking questions, we’d have respondents respond to drop-down menus for their percentage change for a variable on an expectation. And we found that that had impacts, like a framing impact, or an impact on the way that they're answering the question. So one change we made is we now just provide open-ended boxes—basically they're just boxes where they respond what that actual level is, or what that actual percent change is, for a different support point or for a different expectation.

We've also varied the number of questions and how frequently we're asking the questions. We started off by asking six core questions. In addition to the employment, the sales revenue, and the capex, we had unit costs, average prices, and profit margins. Really, what we found—this was some back-and-forth between our team, but we really wanted to lessen the number of core questions we had, so you have more responses or what you think is maybe more validity in any given month. And we changed the way our panel is set up, so now we've whittled our core questions down to three, and we only ask one question per panel right now. So we have three separate panels, and basically if you're a respondent in the Survey of Business Uncertainty, you'll only see one of our core questions once a quarter. That’s sort of born out of a little give and take, understanding the frequency of decision making. So for employment, that might change on a monthly basis, but for capex, it doesn’t appear that most firms really alter their plans on a month-to-month basis. So trying to get at things at a quarterly frequency for an individual respondent seems to be a pretty decent trade-off, and it gives us a higher number of respondents responding to each core question.

Heintjes: I guess that also has the benefit of making it less burdensome for them to respond to?

Meyer: Yes, absolutely. When you’re trying to get in front of C-suite folks and business owners, you really want to limit the amount of time that you’re asking them for, because really, they're doing this out of the goodness of their own heart.

Heintjes: There's no reward in it for them.

Meyer: Well...we've given them, I think, mugs and water bottles before, [laughter] but I'm not so sure that that's a benefit. So we really try to keep the survey response time to a minimum, so we’re talking somewhere around five minutes or so to fill out a survey.

Heintjes: Right, the sweet spot. Well, Brent, we're about out of time, but I want to note to people listening that we'll have a link to the Survey of Business Uncertainty on our website, frbatlanta.org, as well as your research paper that we recently published about it. I encourage everyone listening to check it out periodically, as the survey's obviously updated with new results as we receive new information. But Brent, thanks to you for spending some time with us today. I've looked forward to this conversation, and I really enjoyed it.

Meyer: Thank you very much, Tom.

Meyer: Yes, or a coffee mug. [laughter] I have a Stanford mug on my desk you’re welcome to.

Heintjes: Excellent. And that's all for this episode of the Economy Matters podcast. I'm Tom Heintjes, managing editor of the Atlanta Fed's Economy Matters magazine, and I hope you'll check out Economy Matters on our website at frbatlanta.org. I also hope you'll join me here next month for a new episode. Thanks for being here.