You’ve decided to survey your plan-sponsor clients. Before you jump in, though, it’s important to understand the basics of effective survey design and the associated cost drivers.
Keep It Short, Simple and Focused
The first rule of thumb for any survey is that the questions must be easy to answer, says Jack Semler, president of Readex Research in Stillwater, Minnesota. That generally means that the questions are short, to the point, and use scaled responses or yes/no answers.
Although the terms are often used interchangeably, ratings questions are not the same as ranking questions, Semler emphasizes. Ranking questions ask respondents to rank a list’s elements from highest to lowest. That approach provides valuable information, but it can backfire if the list is too long. “How can you possibly rank-order, with any reliability, 15 different options?” he asks. “Instead, it’s better to ask people, how do you rate these in terms of their importance to you? You can say, ‘It’s very important,’ ‘It’s moderately important’ or ‘It’s not important at all.’”
Robin Green, senior vice president at Ann Schleck & Co. in Woodbury, Minnesota, recommends keeping the estimated completion time for online surveys to five minutes or less. Sarah Cho, director of research with San Mateo, California–based SurveyMonkey, emphasizes the need to avoid obscure jargon and to make the survey mobile-device friendly. Her firm’s experience indicates that about one-third of U.S. respondents and almost half of international respondents participate on mobile devices. “People are responding on the go, so surveys that are too long or with images or videos will be tough to complete for people on mobile,” Cho noted in an email response.
Semler maintains that creating surveys to address the concerns that plan sponsors consider most important can be difficult because it’s a matter of understanding the respondent’s perspective. “When we’re asked to engage in a customer-experience survey, the first thing that I ask my prospective client is this: What have you done to make sure that what we ask about in this survey are truly the most important issues to your customer?” he says.
It’s also a good practice to have someone outside the survey-design team take the survey before using it, Semler adds. That feedback can identify unclear questions and help avoid problematic structures that might reduce response rates.
Controlling Costs
If you hire an outside research firm, you’ll incur their fees plus the indirect cost of spending your staff time on the project. Conducting a do-it-yourself (DIY) survey avoids the research-firm expense but is likely to require more staff time. Regardless of the design method, it’s important to know the cost drivers.
Survey length is the most important factor, says Semler. Longer surveys take longer to develop and consequently are more expensive, whether the survey is developed internally or externally. The good news is that customer-satisfaction surveys don’t need to be extensive, he stresses: “If you have something that’s shorter, it doesn’t take as much time (to develop), and so the meter doesn’t run as long.”
Question complexity is the second cost driver. Questions with yes/no responses or that include lists of possible responses are easier to tabulate and analyze. More-complex question formats require more time, Semler explains. He points to open-ended questions or those that ask the user to “Please specify” their answer outside a given list of responses. Those cases need to be transcribed, coded and grouped to allow analysis—and each step requires time. “Those kinds of things can really make the meter run a little bit longer and a little bit quicker,” he says.
The other cost driver is methodology. Nowadays most surveys are conducted online, but some target groups respond better to mailed surveys, which cost more to distribute and process than their online counterparts. These three factors drive total cost, says Semler: “The easier it is, the less the cost. The more complicated, the more time (required). That will make it more expensive.”
Tips for Survey Success
Green shared several additional suggestions to increase the odds of a successful survey:
- Don’t broadcast an email asking for feedback. A personal note from the relationship manager will generate far more responses.
- If you are working with a third-party like a survey firm, ask them to be brutally honest. Then develop tough skin, because sometimes the feedback is uncomfortable, and be prepared to make changes.
- Follow up and tell clients what you’ve heard. A critical element often overlooked in client-feedback programs is to report back collective program findings and how your business is making changes based on the client’s feedback. When people know you’ve listened and are making changes, they are more likely to continue giving feedback in the future.
