Release Date: 7/10/2014
Survey researchers have been increasingly utilizing multiple interview modes to engage respondents
by whatever mode respondents deem most desirable – mail, telephone, in-person,
or mobile/web. The use of multiple modes has arisen because of the impact of computer/mobile
technology and decreasing response rates to traditional survey modes, such as telephone.
One major issue with multiple mode utilization is that data collected from different survey modes
may introduce mode-related measurement error and reduce data quality. Interview mode itself
may influence responses. In other words, respondents to a telephone survey may offer different
answers than if they were completing the interview online.
To address this challenge of adjusting for mode differences, two members of the Abt SRBI
Advanced Methods Group, Stas Kolenikov and Courtney Kennedy, have just published an
article in the Journal of Survey Statistics and Methodology (2014, 2, 126-158) assessing the
strengths and weaknesses of various mode adjustment approaches . Adjustment approaches
A Regression modeling approach, where adjustments are computed by regressing survey
responses on mode, demogrphics and other relevant variables.
A multiple imputation approach, where mode effects are conceptualized as a missing- data
And a proposed new imputation approach based on an econometric framework of
implied utilities in logistic regression modeling.
They evaluated all three approaches using data from the second wave of the Portraits of
American Life Survey sponsored by Rice University's Kinder Institute for Urban Research.
This survey featured online and CATI interviewing with a national sample of adults, with
random assignment to either CATI-only or web with CATI follow-up for non-respondents.
To evaluate the three approaches, they developed a workflow to determine which variables
require a mode effects adjustment based on standard false discovery rate multiple
hypothesis testing procedures. They detected a significant mode effect on four survey
outcomes after controlling for demographics and risk of type I error. The mode effects
adjustments were then applied to these variables.
They concluded that the multiple imputation approach produced estimates with better
apparent accuracy, as evidenced by better internal consistency of the estimates and a
moderate increase in the standard errors. Unlike the regression adjustment approach, which
can only produce aggregated estimates for the whole study, the multiple imputation
approach can be used for disaggregated analysis with mode-adjusted estimates as well.
About the authors
Dr. Stas Kolenikov is Principal Survey Scientist in the Advanced Methods Group at Abt
SRBI. His key responsibilities include sampling design, power analysis, weighting and
Dr. Courtney Kennedy is Vice President in the Advanced Methods Group at Abt SRBI,
specializing in complex survey designs, particularly those featuring multiple sampling
The full article is accessible free of charge at:
About ABT SRBI
Abt SRBI, a leading public policy and market research organization, is a subsidiary of
Abt Associates, a mission-driven, global leader in research and program implementation in the
fields of health, social and environmental policy, and international development. Abt SRBI is
headquartered in New York City, with offices in the Washington, D.C. area, Chicago, IL,
Cambridge, MA, Fort Myers, FL, Durham, NC and other cities.www.srbi.com