Investors Prioritize Retirement, but Not Planning

First, they may be struggling to find a place for it among their other financial priorities. Respondents said the No. 1 reason they didn't have a financial plan was because they had other, more important priorities.

Furthermore, the retirement industry may be failing these consumers. Sixty percent said that in the past two years, they've had no interaction with any financial institutions about their retirement needs. Even when financial institutions do communicate with consumers, according to the report, they often don't help them integrate their retirement planning needs into their larger financial plan.

A third barrier is consumers' lack of understanding about financial products. Nearly 40% don't know about or understand annuities. Many are simply unaware of the number of options available to them.

Another barrier is a holdover from the recession. Respondents had a remarkably low level of trust for financial institutions in general. Just 20% said they had a high degree of trust for financial institutions of any type.

"After the crisis, investors started looking closer at their portfolios and that hasn't ebbed," Tracy said. "Folks are leery about proprietary products and assets allocations that don't tie to their individual goals." Tracy said that "intertwined with that mistrust" is investors' disappointment with a lack of tailored advice. That sense of mistrust and disappoint gave rise to the fifth barrier Deloitte identified: the DIY investor. Many respondents said they simply didn't need any help with planning. About 40% said they preferred to manage their own portfolio.

Trust, or a lack of it, was one of the top reasons those DIY investors gave for wanting to take on their financial planning by themselves, Tracy said. While it may seem like investors who are trying to plan on their own are at least a little better off than those who aren't planning at all, the report found just 17% of respondents are managing their portfolio on their own and also have a retirement plan in place.

Tracy noted, though, that just because there's a large segment that doesn't have a plan, that doesn't mean there isn't a lot of activity in terms of investments. "Wealth managers need to rethink what they need to be providing," he said.