Release Date: 8/12/2013
Abt SRBI’s statistical team presented three papers at the just completed Joint Statistical Meetings (JSM), August 3-8, in Montreal. The JSM is the largest gathering of statisticians in North America.
Attended by more than 6,000 people, the meetings are sponsored jointly by eight leading North American and international statistical organizations, including the American Statistical Society, the Institute of Mathematical Statistics, the Statistical Society of Canada, and statistical associations of China, India, and Korea.
JSM presentations by Abt SRBI’s Advanced Methods Group members included:
• Blending Probability and Non-Probability Samples Using Calibration Techniques : examines the extent to which appropriate quota sample drawn from one or more non-probability opt-in Internet panels may be used to supplement the probability sample and combined data calibrated to approximate the probability sample's estimates. (DiSogra and Cobb)
• Mode Effect Analysis and Adjustment in a Split-Sample Mixed-Mode Web/CATI Survey: applies regression and implied utility-multiple imputation mode effect adjustments to adjust for mode effects. A randomized split-sample design was built into the study to allow for a rigorous comparison of the item response distributions in the two modes. (Kolenikov and Kennedy)
• Design Effects in Surveys That Require Oversampling of Certain Subpopulations: In a large-scale random population sample with oversamples, this paper examines the variance under simple random sampling with oversampling of subpopulations. The paper looks at design effects with and without oversampling requirement under simple random sampling. (K.P. Srinath)
Full abstracts are appended below.
The Joint Statistical Association meeting activities include oral presentations, panel sessions, poster presentations, continuing education courses, society and section business meetings, committee meetings, social activities and networking opportunities.
Abt SRBI Presentation Abstracts:
Title: Blending Probability and Non-Probability Samples Using Calibration Techniques
Author(s): Charles DiSogra* and Curtiss L. Cobb
Companies: Abt SRBI and GfK
Keywords: Web panels ; Non-probability samples ; calibration
Web panels recruited through probability samples (e.g., ABS) are generally limited in size due to recruitment costs. Properly constructed, these panels can be an ideal generalizable sample for rapid web surveys. To overcome their small size to be able to meet larger sample requirements, it has been demonstrated that an appropriate quota sample drawn from one or more non-probability opt-in Internet panels may be used to supplement the probability sample and combined data calibrated to approximate the probability sample's estimates. Variables that effectively discriminate between these probability and non-probability samples are used in the calibration weighting. Based on experience at GfK/Knowledge Networks with KnowledgePanel, the speaker will discuss the methodology and feasibility of calibrating blended samples from different sources. This methodology can be used with any single-purpose probability-recruited web panel. Alternatively, if mode effects are ignorable, any probability telephone, in-person, or mail sample might also potentially benefit from a non-probability web sample supplement as a cost-saving function. These applications are for discussion.
Title: Mode Effect Analysis and Adjustment in a Split-Sample Mixed-Mode Web/CATI Survey
Author(s): Stanislav Kolenikov* and Courtney Kennedy-Shea
Companies: Abt SRBI
Keywords: mode effect ; mixed mode survey ; multiple imputation ; non-response ; social desirability bias ; split-sample design
We analyze the results of a national survey collected in two modes: SAQ on the web, followed by personal CATI of web non-respondents. We apply regression and implied utility-multiple imputation mode effect adjustments. Since some items may exhibit mode effects, such as social desirability bias, a randomized split-sample design has been built into the study to allow for a rigorous comparison of the item response distributions in the two modes. A logistic model for Yes/No responses or an ordinal logistic model for Likert scales was fit to the data with explanatory variables that included demographic variables and the mode indicator. The regression mode effect adjustments zeroed out the mode variable and formed predictions using the estimated regressions coefficients. Another mode adjustment is based on econometric framework of implied utilities in logistic regression modeling, in which the alternative is chosen with the greatest utility. The latter was treated as a version of multiple imputation. We found the community involvement items and experiencing a major financial crisis recently to exhibit the strongest mode effects.
Title: Design Effects in Surveys That Require Oversampling of Certain Subpopulations
Author(s): Kadaba Srinath* +
Companies: Abt SRBI
Keywords: Complex sampling designs ; Simple random sample
The design effect is the ratio of the variance of an estimated parameter from a survey using a complex sampling design to the variance of the same estimate under simple random sampling with the same total sample size. This is a useful tool estimate sample sizes for future surveys that use complex sampling designs. It also provides a measure of the precision of the design. To compute the design effect we need to estimate both the numerator and the denominator of the ratio. For computing the design effect of estimates in surveys which require oversampling of certain subpopulations, what is the appropriate variance of the estimate under simple random sampling. Since oversampling is a basic objective of the survey, we look at the variance under simple random sampling with oversampling of subpopulations. We look at design effects with and without oversampling requirement under simple random sampling.