I recently attended the annual International Workshop on Household Survey Non-response, hosted this year by the Basque Statistics Office. I would like to share with you three broad themes that I took away from the workshop that directly challenge current practices of dealing with survey non-response.
First of all, it was highlighted that most attempts to increase survey response rates tend to follow the line of least resistance. For example, when reissuing unproductive cases to another interviewer, it is common practice to select those cases that are most likely to be converted rather than the least likely. However, this could potentially increase rather than reduce non-response bias if those who are the least likely to be converted are under-represented in the survey and differ with respect to what the survey is trying to measure. In other words, any attempt to increase response rates should be based on an understanding of who is not responding, why they are not responding and whether their non-response will bias survey estimates.
Secondly, the main obstacle for measuring, preventing and adjusting for non-response bias is the lack of good quality information about non-respondents as well as respondents. Sometimes the sampling frame includes information which could be informative, or the sample could be linked to other data sources (e.g. administrative records). More often than not, this information is not readily available and extra efforts need to be made to collect it. The challenge we currently face is to identify information that can be recorded for all sampled cases and is related to what the survey is trying to measure.
Finally, non-response bias is only one source of survey error. Other sources include coverage error, sampling error and measurement error. It is possible that some methods for increasing response rates could increase other types of error, potentially increasing total survey error. So any attempts to increase response rates should also consider the impact that this may have on other error sources.NatCen has stepped up to these challenges. We are developing an improved strategy for reissuing unproductive cases, evaluating the collection and use of interviewer observations, exploring measures of interviewer-respondent interaction, and promoting a Total Survey Error framework to the design and implementation of our surveys. If you would like more information, or would like to explore opportunities for collaborative research, please contact Gerry Nicolaas.