The Evolution of Rotation Group Bias: Will the Real Unemployment Rate Please Stand Up?
This paper documents that rotation group bias — the tendency for labor force statistics to vary systematically by month in sample in labor force surveys — in the Current Population Survey (CPS) has worsened considerably over time. The estimated unemployment rate for earlier rotation groups has grown sharply relative to the unemployment rate for later rotation groups; both should be nationally representative samples. The rise in rotation group bias is driven by a growing tendency for respondents to report job search in earlier rotations relative to later rotations. We investigate explanations for the change in bias. We find that rotation group bias increased discretely after the 1994 CPS redesign and that rising nonresponse is likely a significant contributor. Survey nonresponse increased after the redesign, and subsequently trended upward, mirroring the time pattern of rotation group bias. Consistent with this explanation, there is only a small increase in rotation group bias for households that responded in all eight interviews. An analysis of rotation group bias in Canada and the U.K. reveal no rotation group bias in Canada and a modest and declining bias in the U.K. There is not a “Heisenberg Principle” of rotation group bias, whereby the bias is an inherent feature of repeated interviewing. We explore alternative weightings of the unemployment rate by rotation group and find that, despite the rise in rotation group bias, the official unemployment does no worse than these other measures in predicting alternative measures of economic slack or fitting key macroeconomic relationships. Read the RESEARCH BRIEF below.
America's unemployment rate — most recently reported as 6.1 percent — has long been used to gauge the country's economic well-being. But a new working paper released by Princeton University's Woodrow Wilson School of Public and International Affairs highlights the difficulty in estimating the exact unemployment rate, though changes in the official measure still signal important movements in the economy.
The research, published by the National Bureau of Economic Research, finds that the true unemployment rate may be higher or lower than recent reports indicate. In fact, the authors write that published unemployment rates have gradually become more difficult to interpret over time, especially in the last two decades. The researchers cite survey design changes as a likely culprit, in large part because the changes corresponded with an increase in nonresponse rates by Americans.
"It is potentially a huge issue," said Alan Krueger, the Bendheim Professor in Economics and Public Policy at the Wilson School and former chairman of President Barack Obama's Council of Economic Advisers. "But our results do suggest several important avenues for future research and improvements in the data."
To calculate the unemployment rate, the U.S. Census Bureau and U.S. Bureau of Labor Statistics administer the Current Population Survey (CPS), which collects extensive demographic data to better understand labor market fluctuations and economic conditions. The CPS consists of a sample of nearly 60,000 households in all U.S. states and the District of Columbia. Households are surveyed monthly for four consecutive months, left alone for eight months and then surveyed again monthly during the final four months. In any given month, there are eight "rotation groups," depending on how many months the households have been in the survey so far. Each rotation group is designed to be representative of the population. The official unemployment rate is a weighted average of the eight groups.
Krueger and his collaborators — Alexandre Mas, professor of economics and public affairs at Princeton's Wilson School, and Xiaotong Niu, an analyst at the Congressional Budget Office — found that in the first half of 2014, the unemployment rate among people in the first month of being interviewed was 7.5 percent. However, for those in the final month of being interviewed, it was only 6.1 percent. Because the Bureau of Labor Statistics weights the first interview more heavily, the official unemployment rate for this period was 6.5 percent.
This disparity between interview groups, known technically as "rotation group bias," isn't new. But Krueger's paper is the first since 1975 to explore the growing magnitude and evolution of such discrepancies. This new paper shows that rotation-group bias has doubled since 1994, when the CPS underwent a major redesign.
"It is unclear which rotation group provides the most accurate measure of the unemployment rate," said Krueger. "The unemployment rate for each rotation group tends to rise or fall together over time."
Two decades ago, the CPS moved from pen-and-paper questionnaires to computer-assisted phone interviews. They also changed the language and logic behind some of the questions. Shortly after the redesign, CPS began seeing a large rise in nonresponse rates.
Interestingly, the study shows that this same kind of bias isn't found when looking at data from the United Kingdom and Canada, which also use rotating groups to measure their unemployment rates. This finding suggests several avenues for future research including examining the details of the surveys themselves, like the nature of the questionnaire, the interviewing methods and the survey response rates. All, Krueger said, play a pivotal role in shaping the unemployment figure outcomes.
The paper, "The Evolution of Rotation Group Bias: Will the Real Unemployment Rate Please Stand Up?," was published online in August by NBER.
To read more Wilson School research briefs, click here.