The need to stay on top of data—and answer to regulators—is keeping financial services firms on their toes, with compliance budgets ballooning, according to a recent Thomson-Reuters annual compliance survey.

In fact, 65 percent of firms say they’re going to be spending more on their compliance team budgets in the next 12 months, according to the March report. In this tougher regulatory environment, with rules from Dodd-Frank to new cost-basis reporting to handle, firms have to mine deeper into data holdings for answers.

“When there are inquiries, the length of the inquiry and the follow-ups have increased,” says Don Runkle, chief compliance officer at Raymond James Financial Services. “The initial and the follow often require data to be supplied. And it used to be 10 initial questions with three follows. Now it’s 20 questions, followed with 20 questions, with another 24 question follow-up.”

The key for broker/dealers and RIAs is knowing where to look for data that will answer regulators’ questions. The problem is, data is piling up, as firms have to keep more information to satisfy regulators. But while keeping it is relatively simple, finding what is needed can be another matter.

Financial services firms are among the biggest offenders when it comes to hoarding data. When adding in banking, securities and investment services, this sector has the most digital data stored on average for a firm, according to a report from the McKinsey Global Institute in 2011. That’s a lot of chaff to sort when looking for wheat. Many firms are now starting to clean up their data—to dig themselves out and make sense of what they have, and reorganize their holdings to find things faster and more efficiently.

“I see regulatory concerns as being the biggest driver for increased storage and retention,” says Meri-Ellen Cain, vice president of business technology for Raymond James Financial in St. Petersburg, Fla. “Now with the financial implications and the burden to respond and adhere to regulatory obligations, it’s a business imperative. We have to cast a fairly large net, and then we have to work to refine it.”

Without that investment, firms are going to be looking at headaches down the road as regulators themselves feel the pinch and continue to push for more information—and firms struggle to respond. The good news is, if they do it right, there are ways to use the data to potentially grow their business.

Rebuilding The Data Farm

Raymond James considers itself no exception to that challenge, and Cain is helping to reorganize the firm’s data storage, and design an “enterprise-wide data repository,” she says, where information will be cleaned, refined and organized with an eye to easier retrieval.

The company has historically stored data within different areas—legacy applications that keep information in separate silos. In some cases, these are different software programs and languages, meaning it takes more time to locate and retrieve data. Think of it as filing cabinets that have different filing rules, some alphabetical, some numeric, with different locks to get into them, and perhaps even written in different languages.

For advisors, the process is fairly invisible, but the results fruitful, potentially. Cain says that with a more efficient process in place to find data, reps will likely be able to see ways to save money and also discover new revenue streams. Examples could include shifting clients into different investments that might provide them with better tax advantages—details reps might be able to see more clearly with data that’s more streamlined.

 “We can do this in partnership with our marketing side,” she says. “Not all regulation closes a door. Some may open a door.”

Of course, the primary concern is making sure that Raymond James has a faster and easier time in returning a regulator’s demand for information, not to mention the followups. In many instances, oversight requests cannot be answered in a simple way, particularly if a firm doesn’t store data in a way that can easily appease queries.

“It never ceases to amaze me; you think something has been answered and they come back with something new,” says Runkle. “‘Sometimes you have to go back and say, ‘We don’t store data that way.’”

Storage Overload

Storing the data isn’t the problem since keeping data is relatively cheap, and getting cheaper. As of 2011, the cost of collecting, housing and keeping digital information is just one-sixth of the cost it was in 2005, according to the 2011 Digital Universe study from research firm IDC. The problem is, the amount of data is growing exponentially—to about 35 trillion gigabytes of data by 2020—making it more difficult for firms to sift through and find what they need. According to IDC, enterprises have “some liability” for 80 percent of that information.

Even though data costs are going down, firms are still investing more to manage this barrage of information. Enterprises invested $2.7 trillion in global information technology in 2005, and IDC expects that to increase to $5.2 trillion by 2015.

The rise of social media among firms is also contributing to increased data retention. Although its use is still tentative among the bigger Wall Street firms, companies are rolling the channels out to reps, albeit slowly. And data being retained in the forms of tweets, comments, and posts is sizeable. Firms can keep anything they want. But logging social interactions—and then locating them as regulators continue to hover over social channels—is the challenge.

“There are risks of overretention,” Martha Mazzone, vice president and associate general counsel with Fidelity Investments, told a room of bankers at the SIFMA Social Media Seminar in New York in June. “The biggest challenge is actually the design of scalable retrieval mechanisms from social media that help you avoid the enormous costs and failures downstream when you’re trying to get information out.”

Yet because of financial crisis after crisis, from the subprime mortgage concerns to bailouts across the globe, regulators are under their own pressures from governments to prove they are doing their job in overseeing financial markets. No one is quite sure when another blow-up will occur, like Knight Capital’s recent skirt with demise. And so, regulators are pushing for more answers, more insight—and often firms are asked to answer questions that can appear so subjective, they seem almost impossible to answer.

“Have you treated your customer well, yes or no?” poses Susannah Hammond by example, senior regulatory intelligence expert, in the governance, risk and compliance business for Thomson Reuters and author of its recent compliance survey. “That is a tough thing to prove definitively. What data have they kept around product development? Why did they develop this product at this price, and why is that a good idea? How do they track afterwards? All of this data around the customer requires in-depth data sets. And if they’re dealing with a lot of data sets and uncleaned data, that makes the process much harder.”

Opportunities Ahead

While Raymond James had already started to streamline their approach to collecting data, Cain says the company has pursued it more as an enterprise discipline in the last two to three years. Cain, who joined Raymond James in October, believes the shift in the way data is now stored will not just streamline the regulatory process, but also open doors for new revenue opportunities. The hope is that the firm will see more quickly and more directly when (and even which) clients want to know about a new service, and then deliver.

But first the data has to be retrievable. And while finding new businesses is crucial for a firm’s long-term viability, so too is answering demands from regulators who feel their own pressure and are pushing that concern on to the financial services market. While Cain says she doesn’t have a crystal ball, she’s pretty sure that the current regulatory environment won’t be easing up anytime soon.

That’s why financial services firms are investing the capital to clean, sort, sift and absorb the trillions of new gigabytes that analysts believe are coming down the road, not just from e-mail but also from sources that haven’t even been invented. Firms need it to appease oversight concerns—and they need it to stay competitive. It’s a race most agree will never really end.

“The job is never 100 percent done,” says Cain. “We still have more to do. I’m not comfortable projecting that in three years we’re done. But we have made significant progress. And you get the benefit along the way if you do it thoughtfully.”