One of the earliest and most critical decisions in the survey development process is to determine how to gather your data. This is referred to as modality or mode. Common modes include inperson/face-to-face, telephone, mail, web, or mixed mode surveys that blend these formats. This tipsheet covers the strengths and weaknesses of the various modes, and compares the tradeoffs involved in choosing one mode over another. Given the increasing ease of conducting web-based surveys, many researchers use web surveys as a default. However, as this document will show, online surveys are not always the most desirable for a given survey project.
Deciding on a Mode
What factors go into deciding on a mode? Some to consider include:
Population – Know your population and consider whether certain modalities would better suit its characteristics. Many senior citizens, for example, are not as adept at the Internet. Thus, an online survey may not be the best mode for a survey of older individuals (e.g. a survey of Medicare recipients).
Types of questions – A survey with simple questions and response scales will likely fit any mode well. However, if a survey has complicated or lengthy questions, a selfadministered survey via mail or the web might be better than an interviewer-administered survey over the phone or in person. In a self-administered format, respondents can read the text multiple times without having to ask for a question to be repeated.
Skip logic – If a survey has complicated skip logic, a web or interviewer-administered survey takes the burden of correctly following skip instructions off of the respondent. This will reduce errors that stem from incorrectly following skip logic.
Question topics – Surveys with sensitive questions might benefit from using a selfadministered format. Respondents are less likely to be influenced by social desirability bias in those modes.
Response rate – If your population is characterized by low response rates (e.g. poorer, less educated, minority, English as a second language groups, etc.), intervieweradministered surveys will typically yield higher response rates. Trained interviewers can be effective at converting refusals and encouraging respondents to complete a survey once they start it, whereas self-administered surveys place that burden on the respondent. This typically yields significantly lower response rates in mail and web surveys.
Cost – Self-administered surveys yield lower response rates and poorer data quality, but they cost less than interviewer-administered surveys that demand greater overhead and labor costs.
Time – Self-administered surveys, especially those on the web, typically have quicker turnaround time than interviewer-administered surveys.
Choosing a mode necessarily involves tradeoffs between all of these factors. If a researcher needs data quickly and at low cost, a web-based survey may be desirable even though it will yield lower response rates and poorer data quality than a phone or in-person survey where a trained interviewer can use quality control techniques to get better responses.
- Generally yields highest cooperation and lowest refusal rates
- Allows for longer and more complex interviews
- Higher response quality
- Interviewer presence can be advantageous
- Multi-method data collection easier
- Most costly mode
- Longer data collection period
- Interviewer presence can be disadvantageous
- Less expensive than personal interviews, but still expensive
- RDD samples of the general population allow easiest sampling method
- Shorter data collection period than interviews
- Interviewer can administer standardized survey (v. mail or web)
- Better control and supervision of interviewers (v. in-person)
- Better response rate than in-person or web
- Coverage bias
- Nonresponse bias increasing
- Questionnaire constraints
- Social desirability
- Lower cost (v. in-person and mail)
- Less human effort needed
- Access to otherwise difficult to contact, busy populations
- Respondents can look up information or consult with others
- Difficult to obtain cooperation
- No interviewer involved
- Sampling can be difficult
- Greater need for an incentive
- Slow data collection
- Lowest cost
- Can survey globally with ease
- Fastest data collection period
- Complex surveys can be easily implemented
- Sample size can be greater
- Multimedia easily incorporated
- Lack of interviewer
- Internet penetration and familiarity
- Representative samples of general population “impossible”
- Technology constraints on the user end
- Lack of interviewer
Author: S Patrick R. Miller, DISM Survey Research Associate