Today we have published results from a survey of the British public’s voting intention in the EU Referendum, based on the country’s first ever probability-based research panel.
NatCen has been tracking changing political attitudes for over 30 years through British Social Attitudes but this is the first time we have run a survey in the run up to the actual polling day.
The new research panel we have set up uses a sequential web/telephone design that enables us to conduct surveys much more quickly than the traditional face to face survey approach.
However this isn’t a typical research panel – it is uses a random sampling method as recommended by Inquiry into the 2015 General Election Polls to improve the representativeness of the sample, and the quality of the research.
There are three key differences between our survey and most other web panels:
1) The sample members were originally selected at random from the British Social Attitudes survey (what is often called a ‘probability’ sample). In most online panels, participants are wholly selected from people who have volunteered to take part with subsequent controls to make the sample socio-demographically balanced.
2) Those who failed to respond to the survey over the internet have, where possible, been followed up by telephone. In our survey, this was particularly important in representing those without internet access who have a different view on the EU Referendum - 14% of the public have not been online in the past three months.
3) The survey has been conducted over an extended period of 4 weeks and has made multiple attempts to reach hard to contact panel members. As a result, the response rate to this particular survey was almost three-fifths of those invited to take part.
Despite a high level of response by those on the panel, there is obviously a concern that the cumulative non-response is quite high and could introduce bias (the effective net response rate is 19%). However, we have been reassured by the raw sample quality, and we also have the advantage of being able to use information collected in the very thorough BSA questionnaire to weight for non-response.
This first published research using the new panel has been self-funded with the purpose of contributing to the methodological debate around opinion polls in the EU Referendum. Since the May 2015 polling miss, there has been a fascinating and important review led by Patrick Sturgis, as well as ongoing refinements in the methods used by polling organisations. Interestingly, our estimate lies in between the average level of support for Remain and Leave being produced by internet and phone polls when most of the interviewing for the NatCen survey was conducted. We have also tried to follow the spirit of the inquiry too, for example by publishing detailed information on weighting (including the adjustment for turnout) and the inclusion of confidence intervals.
The set up of the panel was made possible through a feasibility project into researching attitudes among low income groups, commissioned by the Joseph Rowntree Foundation.
We now propose to further increase the size and scope of the panel so it can complement the British Social Attitudes series and other surveys. I’d be interested in hearing from those who want to work with NatCen on this exciting new methodological development.
Kirby Swales is Director of NatCen’s Survey Centre and can be contacted at: kirby.swales@natCen.ac.uk