Research Impacts - Articles

Do Advance Letters Help Response Rates for Cell Phone Samples?

Release Date: 10/26/2016
 
Health Policy Group Blog

The percentage of cell phone-only households continues to rise, and the characteristics of these households differ from landline-only and combined landline/cell phone households. The 2015 National Health Interview Survey found that 47% of American homes were cell phone-only. The survey also found that cell phone-only adults were more likely to live in poverty, be younger, be Hispanic, live with unrelated adults, and have a higher frequency of heavy drinking and smoking. Given the increase in incidence and the characteristic differences of cell phone-only households, it is important to identify new methods to increase response rates and enhance the representativeness of cell phone samples. While advance letters are commonly used to increase response rates in landline Random Digit Dial (RDD) studies, the ability to attach billing addresses to cell phone numbers is a recent development. 


Abt SRBI, Inc. collaborated with the University of Colorado, Denver on an experiment in the first wave of the 2015 iteration of the dual frame RDD study, “The Attitudes and Behaviors Survey (TABS) of Colorado” (Colorado TABS) to measure the effect of advance letters on the response rates and data quality under three cell phone sample conditions. The conditions were: (1) cell phone numbers with an appended billing address that were mailed an advance letter (denoted as treatment group); (2) cell phone numbers with an appended billing address that were not mailed an advance letter (denoted as control group); and (3) cell phone numbers with no appended billing address (no-address match group).
While 69% of the cell phone sample had a matched address and only 12% were returned as undeliverable, just 20% of cell phone sample respondents remembered getting the letter compared with 42% of the landline sample respondents. We were unable to determine how recently the matched cell phone address had been updated, and assuming that recall is a rough proxy for having received the advance letter, the difference in the cell phone versus landline sample recall rates may reflect, at least in part, out-of-date cell phone addresses. This interpretation is consistent with the 2013 National Immunization Survey, where Skalland, Zhao, and Jeyarajah found that the accuracy of respondent-provided cell phone address zip code matching increased from 46% to 64% for addresses updated within 90 days.


We compared response rates between the cell phone sample groups that did and did not receive an advance letter. The AAPOR response rates (RR6) were: 85.6% for the treatment group, 84.3% for the control group, and 74.4% for the no-address match group. Although the matched address conditions yielded higher response rates than the no-address match condition, the advance letter did not increase the response rate among cell phone respondents where we had a matched address. Here, we used RR6 which assumes no eligible cases among those with unknown eligibility because the high percentage of records with unknown eligibility in the no-address match condition skewed the other response rate calculations.
We also looked at whether or not the advance letter improved the representativeness of the sample. While we found small differences between the treatment and control groups, almost all significant differences were between the no-address versus address match conditions (treatment and control groups combined). Among cell phone respondents, the no-address match group had lower socioeconomic status (lower income and lower educational levels), a higher percentage who self-identified as Hispanic, and a higher percentage of renters.  


Our study found that mailing an advance letter to matched cell phone sample addresses had no effect on the response rate or representativeness of the cell phone sample compared with no advance mailing. However, key demographic differences between cell phone numbers with and without matched addresses suggest that we should focus our efforts on finding methods to increase the response among the no-address match cell phone group.