Between 2007 and 2010, on behalf of the Scottish Government and General Register Office for Scotland (GROS), we conducted a series of studies testing a range of questions that had been developed for Scotland’s Census 2011. Following testing, we made a series of recommendations on how the questions could be improved.
The focus of each study was as follows:
Cognitive testing of the whole census questionnaire
The testing had two main aims: to test section H1 to H5 (the household composition section) and the accompanying guidance to ensure that it was easy to understand and answer to test that the information and questions contained in the whole questionnaire were presented as effectively as possible, within existing design constraints.
Overall, 70 cognitive interviews were conducted with a broad range of respondents. The sample was not intended to be statistically representative of the Scottish population, but was designed to include people from the main groups that we anticipated might:
- have trouble understanding the concepts of usual residents and ‘visitors’
- have difficulty correctly navigating through the questionnaire
- have difficultly understanding and following particular guidance and instructions included in the questionnaire.
Click here to view the report (PDF)
Cognitive testing of the ethnicity question for the 2011 Census
The overall aim of the research was to provide evidence on how the different draft questions were interpreted by respondents and whether the questions elicited the required information on ethnicity.
Eighty-eight cognitive interviews were conducted with respondents from a wide range of ethnic and religious groups.
Click here to view the report (PDF)
Cognitive testing of the national identity question for the 2011 Census
Three separate waves of testing were carried out, in 2007 (as part of the project testing the ethnicity question), 2008 and 2009. The overall aim of the research was to provide evidence on how the national identity question was interpreted by respondents and whether respondents answered it in the way intended.
Over the course of the three projects, the question was tested with a total of 208 participants. The sample was designed to include people from the main groups that we anticipated might:
- have trouble working out the meaning or intention of the question
- interpret the meaning of the question incorrectly and, as a consequence, answer differently than they might otherwise have done
- feel that having both the national identity and the ethnicity question allowed them to describe themselves more fully
- have an objection to the inclusion or phrasing of the question or inclusion or exclusion of individual response options.
Click here to view the report (PDF)
Cognitive testing of the language question for the 2011 Census
Between 2008 and 2009, on behalf of the General Register Office for Scotland (GROS), Ipsos Scotland conducted two projects which tested draft questions for the language section of the 2011 Scottish Census. The aim was to test whether respondents’ answers to whether they can understand, speak, read or write Scots provide meaningful data; to explore how respondents interpret Scots; and to understand people’s decision making process when answering the English proficiency question.
Over the course of the two projects, we tested the section with a total of 167 participants. The sample was not intended to be statistically representative of the Scottish population, but was designed to include people we anticipated might:
- have trouble working out the meaning or intention of the question
- have difficulty selecting the accurate response option
- have difficulty locating an appropriate response option that described their situation
Specifically the sample included:
- Scottish Gaelic speakers
- BSL users
- ethnic minority language speakers. This included Bengali speakers, Urdu speakers and Polish speakers
- ‘Scots’ speakers.
Click here to view the report (PDF)
Cognitive testing of the health and disability questions for the 2011 Census
The General Register Office for Scotland (GROS) commissioned Ipsos Scotland to undertake cognitive question testing of the question on long term health conditions (Q20). As part of this, the related questions on general health (Q19), and on whether day-to-day activities are limited by long term health problems or disabilities (Q21) were also explored.
In total, the questions were tested with 102 respondents: 30 respondents with health conditions who were recruited specifically for the project; and 72 other respondents (who may or may not have any health conditions) recruited as part of two separate projects that were being conducted at the same time.
Click here to view the report (PDF)
Cognitive testing of the income and qualifications questions for the 2011 Census
The General Register Office for Scotland (GROS) commissioned Ipsos Scotland to undertake cognitive question testing on a provisional set of questions for household income and qualifications. In light of information received from previous user consultations and the 2006 Census Test, GROS wished to test how the questions would work in practice to help them to determine if, and how, questions should be amended for the 2011 Census.
65 cognitive interviews were conducted with a broad range of respondents in terms of ethnic background, age, sex, household living arrangements, educational attainment level and language capabilities.
