Can tablets collect data from people with no internet or limited digital literacy?

Survey data collection is increasing moving online. A key driver has been cost, though other factors such as changing public attitudes and lifestyles and the recent COVID-19 pandemic have also played a part.

However, not everyone is able to take part online. Internet penetration varies between countries making web-only surveys challenging when conducting cross-national research. The latest estimates (PDF) suggest that 6% of UK households do not have access to the internet, with those without access more likely to be older, poorer and disabled.

We also know that even if people have access, they may not feel confident in using digital devices such as tablets, computers or smartphones. As survey designers, we need to ensure that in the move to online surveying, we don’t exclude these groups of people.

Typically, the way we try to include people who are not online is by offering a paper version of the questionnaire. This can be a good option when the questionnaire is fairly straightforward, but if the questionnaire structure is more complex then it may be more challenging for people to complete.

Another option would be to offer people the chance to complete the questionnaire on an electronic device that does not require internet connectivity and is simple to use. Could such a device be developed?

Working with colleagues at Centerdata in the Netherlands and funded by the European Social Survey European Research Infrastructure Consortium (ESS ERIC), we developed and tested such a device during 2020-21. Known as the ESS Electronic Questionnaire Device (ESS EQD), we customised an off-the-shelf Android tablet and installed an adapted version of the ESS core questionnaire for survey participants to complete in their own at home.

The device had to be easy to use: the ESS questionnaire should display automatically when the device is turned on, there should be no need for scrolling, and answer selection and input should be made as easy as possible (e.g. through the use of large buttons and the option to audio-record answers instead of typing them).

But would people with no internet access, who rarely use the internet, or who have limited experience of using a tablet or smartphone be willing and able to use it?

Results (PDF) from our small-scale, proof-of-concept test run in Portugal, Serbia, Slovakia, and the UK indicated that the target groups for the ESS EQD were able to use the tablet if some, minimal, training was provided. The target groups included people who never use the internet (or use it less than once a month).

However, the ability to use the device was not universal and a sub-set of people struggled to use it. These people reported feeling nervous about using the device and were not able to master the basic tasks necessary to complete the questionnaire, such as tapping buttons to select an answer and moving to the next question screen.

Is an electronic questionnaire device a viable alternative to a paper questionnaire? Our study demonstrated the proof of concept that participants with no internet connection and limited experience can be trained to complete a questionnaire on a customised tablet. More research is needed to assess the likely uptake of such a device in a random probability sample survey and to understand how such a device could be integrated in existing survey data collection systems.    

NatCen’s Jo D’Ardenne presented findings from this project at a survey methods seminar hosted by City, University of London, the European Social Survey and NatCen in November 2022. A recording of the event is available on YouTube.