What makes British Social Attitudes a gold standard social survey?

British Social Attitudes (BSA) is the National Centre for Social Research (NatCen)’s flagship study tracking social and political attitudes. The survey is often referred to as the ‘gold standard’ of attitudinal research, but what does this mean and why is the methodology used for BSA so important?

For the last 40 years the BSA has asked the public about a wide range of topics including attitudes to public spending, the NHS and the monarchy. This time series data is a major strength of the survey and allows us to track changes over time to see how views, attitudes and opinions have (or haven’t) changed. The recently published BSA 40 report focuses on how society has changed over the last four decades, exploring gender roles and work and sexual relationships among other important social issues.

However, the survey is not limited to time series data and every year we include a number of new questions on BSA, allowing us to respond to emerging issues and explore new topics. Importantly, any new questions are thoroughly tested to ensure that respondents understand them and that they capture the intended attitudes. This rigorous testing process is another advantage of BSA and consists of three stages:

• Initial review of questions: working with funders to understand their aims and objectives; checking whether existing BSA questions could be used and ensuring that wording of new questions is not leading or likely to cause offence. 
• Cognitively testing: any new questions that are considered to be particularly complex or sensitive. During this process we ask respondents the sub-set of questions to be tested who are then probed about how they answered the question, to explore any issues with wording, response options and concepts. We use the feedback from these interviews to review and refine questions.  
• Quantitative pilot of questions: NatCen telephone interviewers speak to around 50 respondents recruited from the NatCen Opinion Panel. We use quotas on age, sex, region and education to make sure that a range of different people are included. Interviewers record questions that appear to cause difficulties for respondents, or where there are issues. Final changes to the questions are made before they are included on the main questionnaire.

A further strength of BSA is its use of random probability sampling. Addresses  are selected from the Postcode Address File (PAF), a list of all the known “delivery points” and postcodes in the UK, and ask up to two adults aged 18 and above to complete the survey. The random probability approach means that everyone has an equal chance of being selected to take part in BSA and that the results are representative of the British population.

From 1983 to 2020, BSA was conducted in person by NatCen interviewers. However, during the COVID-19 pandemic we needed to move to a ‘push-to-web’ approach, asking the BSA questionnaire online. Respondents that were unable to complete the survey online were offered the opportunity to participate over the phone. Care was taken during this transition to ensure that BSA maintained its robust methodological principles. As with the face-to-face survey, a random probability sample of addresses in Britain was selected from the PAF. Questions for the web survey also underwent the same thorough development process as those included on the face-to-face survey. As push-to-web surveys have lower response rates than face-to-face surveys, non-response weights were applied to the data.

After fieldwork was completed, checks are run on the data to ensure that any changes were the result of genuine shifts in attitudes rather than the change in method. Analysis was also run to compare the samples collected on the face-to-face and push-to-web versions of the surveys. After weights were applied, it was found that the populations for the two modes were demographically similar. BSA has been run as a push-to-web survey since 2020 and we are confident in the quality of the data collected using this method.

We are constantly looking at ways to improve BSA and to make sure that the findings from the survey continue to stay relevant.  Whilst we have successfully moved the survey from face-to-face to push-to-web, we may also consider other changes to methodology. Looking to the future we could explore conducting interviews via video rather than telephone or using AI or bots to help respondents answer questions online. It will also be really important for us to think carefully about the issues that society will face in the coming years and how we will ask about these on BSA.

British Social Attitudes (BSA) is made possible by the funding we receive from a variety of charitable and governmental sources each year. Each year the BSA collects data on a variety of topics covering attitudes on a breadth of social issues this includes repeat questions and new questions that are relevant to society today.

You can fund anywhere from 5 to 50 questions on the survey with a sample of between 1,000 and 3,000 people. You can also access more than 40 years’ worth of trend data, an array of demographics and have the opportunity to be part of the British Social Attitudes report. If you are interested in funding a module of questions within a particular topic area, please contact the BSA team at BSA@natcen.ac.uk