How good are those life insurance and retirement surveys that cross your desk just about every week? Have you noticed that surveys by life insurance companies almost always find that people are unprepared for retirement and don’t have enough life insurance?
Here are the findings of just a few of the life insurance company-conducted surveys over the past several months:
- Despite the fact that half of U.S. households say they need more life insurance coverage, only 22 percent seriously shopped in 2011.
- Fifty-three percent of more than 1,000 adults say that they are financially under-protected.
- Since the economic downturn in 2008 and 2009, people are more inclined to buy life insurance products due to the guarantees.
- Fortunately, yet another poll indicated that most retirees are happy and optimistic.
Are these polls truly accurate reflections of consumers nationwide? Or are they ways for insurance companies, investment companies and financial planning firms to drum up business by issuing subtle reminders about investors’ sorry financial shape?
The only way to tell is to look closely at the survey’s methodology, suggests Mathew Greenwald, Ph.D., president of Greenwald Research, Washington, D.C. Greenwald’s company has conducted numerous surveys for financial services companies over the years.
The guts of a survey are its statistical design, its sampling procedures and questions. Good probability sampling selects people to respond to questions in proportion to the degree that they comprise the overall population. The quality of the interviews and questions put to respondents are of paramount importance.
Greenwald recommends that financial advisors who are interested in a survey call the sponsoring company and get detailed information about its methodology. Only then can they determine if the survey is accurate and reflects the attitude of the investment public.
There are several variables to consider when evaluating a survey’s results, according to Greenwald. Here are a few:
- Reasonable sample size. The sample size should be over 200, but 500 is better. A large sample size will more accurately reflect the attitudes of the population that served as the focus of the survey.
- The sample should not be skewed or distorted. For example, if the survey reveals consumer attitudes about investing, you don’t want the sample to come from one area of the country in which people are more affluent than the rest of the country.
- How were the survey respondents contacted? Telephone interviews are best because the surveyor can get more specific information compared with written survey requests in which respondents are asked to check off items. In addition, respondents should be contacted through an automated “random digit dialing” program, rather than from specific lists. If names come from a list, the list could be biased. Be suspect of surveys conducted over the internet, because there is no control over who answers the questions. Also be suspect when a publication reports surveys of its readership. Although there are some good readership surveys, results may be biased due to the limited demographics of those who respond.
- Be wary of “push” surveys, Greenwald says. These surveys are designed to elicit a specific type of response to prove a particular group’s viewpoint. Examine the type of questions and the way they were asked.
- Check to see if a statistical analysis was conducted to ensure that the questions were not highly correlated. Typically, survey firms conduct a “regression analysis” or “factor analysis” to insure the questions are independent from one another.
- How were the results interpreted? Does the person interpreting results have a good understanding of the subject matter? What are the points being made? Based on your experience, are their comments knowledgeable? Did you learn something from the survey results that you did not know beforehand?
- Consider the company that conducted the survey. Is the company a well-known entity that has a good reputation for conducting surveys?
- What is the sponsoring firm’s goal in releasing the survey? Many insurance and investment companies have well-designed surveys that draw attention to financial behavior and attitudes. These surveys still could be helpful as long as you consider the company’s interest.
- Was there a statistical test conducted to make certain the responses were not due to random chance? You want to know, for example, that 95 of 100 times, the responses represent 45 percent to 50 percent of the population’s viewpoint on a specific issue.
- Ask the survey sponsor for past surveys so you can identify any important trends.