The research process...
High confidence results demand a rigorous survey development process. How many research projects have been rendered useless, in part or whole, because a step was left out or ignored in the research process? Here are the steps we adhere to for Wilson Research Group projects:
- Define goals, sample, response rates, method and schedule
- Sample development and preparation
- Survey design: survey questions and programming
- Pretest survey in "live" format
- Review and signoff by client and WRG
- Prep for deployment: administrative, list management, printing, postage, letters, assembly, and tracking as needed
- Deploy real-time online reports
- Survey fielding by mail, Web, or telephone
- Data entry, data cumulating, and data cleaning
- Programming and tabulation setup for prelim and final reports
- Final offline data analysis and report
SURVEY GOALS: Properly conducted market research helps to improve the quality of products, services, customer satisfaction, and business decisions. Market research provides you as a decision-maker with a more informed basis upon which to lay out strategies, alter courses, and get in tune with the needs and suggestions of your customers.
Strict methodologies are required to achieve these goals. To gather valid and accurate information, survey market researchers use special statistical methods. Sampling is key. A sample of those to be surveyed must be scientifically selected to represent the total population of the target group. Then the research planners must select the survey method and survey instrument that will best reach the target group within the time frame (or schedule) and budget needed. High response rates are also part of the equation. Goal setting identifies the incentives and strategies needed to maximize respondent participation in the survey.
SAMPLES: Sampling is perhaps the single most important component of high confidence market research. A sample must represent the larger target audience (see our confidence levels and sample size chart). To be representative, the sample must be selected randomly or with a scientific method that insures members of the sample are represented in the same ratio as they are in the total population of the audience, or that each member has an equal chance of being selected from the universe. Samples come from customer lists, industry SIC code lists, magazine subscriber lists, Web-visitor lists, government sources, newsletter subscriptions, privately built groups, and many other sources.
Customer samples, if selected randomly from all customers, will reflect the entire customer base. Generally, this is the best source for understanding your customers' needs and opinions. Live samples, such as conference attendees, Web-site visitors, or technical support callers can be "intercepted" and provided an opportunity to give feedback on their experiences on a spot basis or on a continued basis. Convenience samples are lists of people who are conveniently available, who are members of your target group, but may not be fully representative of the entire group. Industry samples can be derived from generally available lists of various market segments. How well these lists represent the industry varies as to how the lists were constructed.
RESPONSE RATES: Another key to having high-confidence surveys is the response rate. Not only must the sample be randomly or scientifically selected, but the number responding to the survey as a percentage of how many were sent the survey determines in part how representative the results will be.
It takes only about 384 respondents (assuming a random sample and a reasonable response rate) to project to a universe of 1 million people at 95% +/- 5% confidence (a rule of thumb standard). This is why sampling is a cost efficient and highly productive way to learn about broad characteristics of a population.
Here's an easy way to think about response rates: 500 returned surveys from a random sample of 1,000 people yields much greater confidence than 500 returned surveys from a sample of 2,000 people. The first case is a 50% response rate; the second case is a 25% response rate. In the second case, you are LESS certain that there is NOT a bias in those who did not respond.
Many strategies (and incentives) for achieving a high response rate are available. Success in obtaining high response rates revolves around a few principles:
- Disclosing the full purpose and reasons for the research
- Consideration of the respondent first and foremost
- Attention to every detail in the process
- Understanding the respondents' interests and motivations
- Placing trust in the respondent
Strategies are varied, complex, and each has an associated cost. Client and researcher must jointly determine the expected response rate based on cost and confidence factors. Usually, researchers strive for a 95% confidence level. This means, for example, that if 20 scientifically selected samples were chosen, 19 samples would yield answers that are within five percentage points of each other. One of the 20 samples would yield results outside these tolerances.
THE RESEARCH SCHEDULE Each research methodology requires different time allowances to complete.
Below is a rough estimate of typical schedules for each type of survey. A host of factors affect the
development, deployment, and analysis of each specific survey.
Web survey schedule is typically 2 to 8 weeks
Paper survey schedule is typically 8 to 10 weeks
Disk survey schedule is typically 7 to 8 weeks
Phone survey schedule is typically 4 to 5 weeks
