Is it ok to use differential incentives to encourage survey participation in under-represented groups?

The article was first published in SRA's Research Matters – September 2024 edition, click here to read the article.

Despite the increasing use of administrative records and alternative data sources, government departments and other research commissioners continue to rely on surveys to help them understand the reality of people’s lives, and to develop and improve services to the public. However, there is growing awareness that surveys struggle to reflect the diversity of the population, and more needs to be done to improve the inclusion of the most vulnerable and disadvantaged groups in society. ^{1 2}

Can differential incentives produce more inclusive survey data?

One promising practice for improving the inclusiveness of survey data is the use of differential incentives with survey participants incentivised differently based on (a) characteristics that are either available on the sampling frame or collected during fieldwork, or (b) their behaviour (for example reluctance/refusal to take part). This can take different forms: incentives versus no incentives, given to all or only those who co-operate, different values, different types and so on.

The use of differential incentives is common practice in the USA where they have been used and evaluated since the 1990s. Overall, these US studies show that differential incentives are effective at persuading reluctant respondents, decreasing non-response bias, and are cost effective because they are only given to a small sub-sample.

Differential incentives are also being used in the UK, but the evidence base is limited, and its practice remains controversial. There is widespread concern among research ethics committees, survey sponsors and survey providers that giving incentives to some and not others will violate expectations of equity. However, it can also be argued that treating everyone equally does not necessarily achieve equity.

Differential incentives are not inherently unethical

In June 2019, a workshop funded by the Economic and Social Research Council (ESRC) considered the ethics of using differential incentives in survey research. The main conclusion was that differential incentives are not inherently good or bad. Whether their use is acceptable depends on the importance of including hard-to-persuade groups, the reasons why these groups are less likely to take part, and evidence that differential incentives will achieve the desired objective of reducing non-response bias and/or improving inclusiveness.

The workshop report ³  includes eight draft principles to help guide researchers in their decision on whether to use differential incentives:

1.  The use of differential incentives in surveys is not inherently good or bad but depends on the context and how they are implemented.
2.  Researchers must provide a clear justification for using differential incentives.
3.  The justification should balance the interests of the survey sponsor, the organisation collecting the survey data, the participant and society.
4.  The reasons for using differential incentives should be based on evidence that these are likely to reduce non-response bias and/or improve inclusiveness, while controlling costs.
5.  The impact of differential incentives on participants’ rights, dignity and autonomy needs to be considered.
6. The value of differential incentives should be proportionate to the perceived burden imposed on those who are eligible to receive the incentive and limited to what is necessary to achieve the desired effect.
7.  Receipt of the differential incentive should be based on the characteristics and/or behaviour of the participant and not dependent on others (for example, it cannot be conditional on whole household co-operation).
8. The use of differential incentives should be transparent unless thiswill undermine participants’ rights(for example confidentiality) and dignity (for example stigmatization).

 Time to revisit the practice and ethics of differential incentives

Given the commitment to produce more inclusive data, now is a good time to revisit the practice, evidence and ethics of using differential incentives in the UK. Under Phase 2 of the UKRI ESRC-funded Survey Futures programme, UCL and NatCen will soon be starting a new project to:

• review current practices in the UK
• carry out an experiment on a UK push-to-web survey
• develop a 'code of ethics' and advocate for its adoption by survey sponsors, professional research associations and research ethics committees.