Many people are positioning their bond portfolios for a rising interest rate environment, as the Federal Reserve continues to wind down its stimulus program. But portfolio managers at Envestnet’s advisor summit in Chicago said the bigger concern—and more likely scenario—is that rates continue to stay low.

“We possibly have Japaned ourselves,” said Morgan Neff, senior portfolio manager at SMH Capital Advisors.

Japan has had low interest rates for a prolonged period, and the results have been troubling, Neff said. The Nikkei index, for example, is just a fraction of what it once was.

The U.S. may not be far off from looking like Japan. Japan currently has about $2.10 of debt for every GDP, while the U.S. is at $1.02 of debt per GDP.

The possibility of rising rates is not as foreseeable as people think, Neff added. The Fed originally said it was going to look at unemployment rates to make decisions about rates, but they have since changed that tack. Because even the Fed is confused on the issue, it’s hard to predict what will happen to rates.

Instead of unemployment, Neff believes we should look instead at labor force participation, which is at a 34-year low. Also, median household income continues to drop.

There are risks to rates staying low, said Thomas Marthaler, portfolio manager at Neuberger Berman. The risk is that investors will get too defensive and act irrationally, in response to the fear of rising rates. His recommendation: don’t go too short in duration. Because if you do, you’re timing the market.

Neff is more concerned about clients being too aggressive, however, because if rates have a correction, that would be damaging to an aggressive portfolio. Being too aggressive is what caused large pension funds to experience huge funding gaps in the crisis of 2008. Stay the course in bonds, he suggests.

Adam Backman, portfolio specialist at Lord Abbett, said that if rates stay low, it will mean that the economy is continuing to grow slowly, and debtors will be able to refinance, as they have been. That means that the real return will be in taking credit risk. Credit-oriented segments of the bond market have historically outperformed in periods of rising rates.