The household survey most commonly used to track participation rates in workplace retirement plans has been put under further scrutiny in a new study by the Investment Company Institute.
The Current Population Survey, administered by the U.S. Census Bureau, is the country's "primary source" for labor force statistics, according to the agency's website.
It also tracks how many Americans are saving—or not—in workplace retirement plans, data that's used, in part, to craft retirement policy at the federal and state levels.
According to ICI, a trade group that represents the interests of mutual fund companies, the Census has historically under-reported the number of Americans participating in a retirement plan.
But recent updates to the CPS, designed to more accurately capture retirees' income, resulted in even shoddier data on participation rates, it says, raising the question of whether or not retirement policy initiatives are being advanced on bad information.
"To do policy analysis accurately it is absolutely crucial to know the facts," said Peter Brady, a senior economist at ICI. "If data are incorrect it will lead to bad policy decisions."
For the past decade, Census data on plan participation rates tracked lower than data from the IRS. CPS data showed between 49 and 51 percent of Americans age 26 to 64 were participating in workplace plans. IRS data generally put the figure at 55 percent, according to a study co-authored by Brady.
But in 2014, when the CPS was redesigned, participation rates inexplicably plummeted, first to 44 percent, and then to 39 percent by 2016. By comparison, IRS data shows 56 percent of taxpayers between ages 26 and 64 participated in workplace plans in 2014; 63 percent of workers either participated in a plan or had a spouse that did.
To the agency's credit, the Census Bureau was aware of the CPS's failure to accurately report retirees' income and set out to fix that, says Brady. New questions distinguishing 401(k) and pension income were among the changes, as was a new question on annuities. The result was a longer survey that more accurately captured retirees' income and its sources.
But for non-retirees, the changes resulted in what Brady describes as survey fatigue. Federal law requires participation in the Census when the government comes calling; it is less clear on the failure to pay full attention. The best explanation for the dramatic under reporting of plan participation is respondents losing focus when navigating a series of pension questions that don't apply to them, according to Brady and other economists.
Participation rate the wrong number to look at
"The growing consensus is we need better sources of data on retirement," says Brady, who has also served as an economist at the Treasury Department and Federal Reserve Board.
But even when total participation rates are accurately documented, Brady thinks that measurement falls short when assessing the health of the country's private retirement system.