Switching to an alternative survey method to assess crime levels in Scotland during the COVID-19 pandemic

Emily Gray and Chris Martin of Ipsos Scotland explain the alternative methodological approach we took so that evidence to inform crime and justice decision-making in Scotland could still be collected during the pandemic.

The author(s)
  • Dr. Emily Gray Managing Director, Scotland
  • Chris Martin Ipsos Scotland
Get in touch

Since 2008 the Scottish Crime and Justice Survey has provided invaluable evidence about experiences and perceptions of crime, the police and the justice system in Scotland. The study measures the level of crime in Scotland and how that has changed over time. This is essential information, not least since much crime goes unreported to the police. The latest SCJS National Statistics tell us that the police became aware of 40% of crime in Scotland in 2019/20, a similar proportion to previous years.

The SCJS and its predecessor surveys (see the latest technical report for a short history of crime surveys in Scotland) have in the main been conducted using a gold standard in-home face-to-face survey methodology. The one exception was in 2004, when the then Scottish Crime and Victimisation Survey underwent a major methodological change moving to telephone interviewing. However, after it was shown that the robustness of the data produced by the 2004 telephone survey could not be substantiated (Hope 2005), the survey returned to face-to-face interviewing and has been conducted this way ever since.

That was, until COVID-19 hit. On 17th March 2020, the Scottish Government took the decision to suspend all face-to-face fieldwork to help prevent the spread of the virus in Scotland. The country went into lockdown less than a week later. This meant the study data could not be collected in the usual way.

The suspension of the SCJS left a gap in the evidence base at a crucial time, when the pandemic made it more important than ever to assess how crime in Scotland was changing. Recorded crime statistics from the police in Scotland have shown that while volumes of some types of crime were lower in December 2020 than in December 2019, such as shoplifting, housebreaking, other theft, and sexual assault, other types of crimes such as fraud were higher. But without robust survey evidence to complement the recorded crime statistics, we would have no way of measuring adults’ experience of crime during the pandemic.

The SCJS team at the Scottish Government, together with the survey contractors Ipsos and ScotCen, therefore assessed the different options available for collecting evidence on crime and victimisation during the pandemic. We rapidly came to the conclusion that it would not be possible to collect evidence that was directly comparable with previous years of the SCJS. A different survey mode would have to be used as face-to-face in-home interviewing was not permitted in the pandemic, and it would be impossible to trial any such method in parallel to the existing face-to-face methodology – work which would be needed in order to understand the impact of the change of survey mode on the survey estimates.

We worked at speed to assess the different methodological options and agree the preferred approach. What the team concluded would be possible was a new and unique survey which would contribute to the evidence base by providing a reliable measure of adults’ experience of crime in Scotland during the pandemic, as well as providing information on public perceptions of crime, policing and safety issues related to the pandemic. The result was the Scottish Victimisation Telephone Survey (SVTS) conducted in September and October 2020, with findings published in February 2021.

Sampling was a key consideration for the SVTS design. We elected to invite participants who had previously taken part in the SCJS in person in 2018 or 2019 and agreed to be recontacted for research purposes to take part in the survey. This had a number of advantages: fieldwork could start more quickly, the response rate would be likely to be higher than if contacting people who had not previously taken part in the SCJS, and the survey did not need to collect certain information again (such as people’s address details, if they had not moved since the original interview). It also meant that we would be using a sample based on a random selection of addresses and adults, and that we had information on the characteristics both of those participants who had agreed to be recontacted and those who had not. This benefited the survey quality since adjustments to the data to reflect the Scottish adult population could then be made at the weighting stage. 

Questionnaire content was an important challenge we faced. Face-to-face SCJS interviews take around 45 minutes on average. Switching the mode to telephone meant that a shorter questionnaire was needed to keep participants engaged and minimise the risk of them terminating interviews.

The team worked quickly to agree the survey objectives and questionnaire content. This included adapting existing demographic questions to include COVID-19 concepts, such as key worker status and furlough.

Much of the questionnaire content focused on collecting the detailed questions that collect information on incidents of crime, to allow these to be coded in a way consistent with police recorded crime statistics so that comparisons can be drawn. This meant that the space available for new questions relating to the pandemic was limited. However, we were able to include a small number of new questions on perceptions of crime, safety and policing since the coronavirus outbreak, which meant we were able to draw comparisons between Scotland and England and Wales since the Office for National Statistics’ work on the Telephone-Operated Crime Survey for England and Wales (TCSEW).

One limitation of the SVTS is that it was not possible to include questions on more sensitive topics which are asked through a self-completion element on the SCJS, such as drug use, sexual victimisation, stalking and harassment, and partner abuse, for ethical and safeguarding reasons.

The SVTS results were published in February 2021 and provided the first official attempt to assess the total level of crime in Scotland during the pandemic. Crime fell significantly since the start of the first national lockdown, by around 35%. While the vast majority of adults (91%) said that the pandemic had not changed how worried they felt about being a victim of crime, previous victims of crime felt more worried about this since the virus outbreak compared with non-victims.

At the time of writing, lockdown restrictions in Scotland are set to ease soon. But the virus has not gone away, and it is clear that there will be no return to face-to-face interviewing in Scotland in the short term. In this context, the SCJS team are currently taking stock of the SVTS and examining the options for collecting further critical evidence on crime, justice and policing later in 2021.

The author(s)
  • Dr. Emily Gray Managing Director, Scotland
  • Chris Martin Ipsos Scotland

More insights about Public Sector