Preparing the questionnaire 

The format, question selection, wording and order used in the Community Child and Youth Well-being Survey questionnaire have been carefully designed and thoroughly tested to deliver the best possible results. The survey aims to present a multi-dimensional and balanced overview of the state of children and youth based on their perspectives. It was developed through extensive consultation with experts, including young people, to create the right balance of dimensions, ensure question clarity, align with validated surveys of young people and keep the time for completion to a manageable 20 minutes for most participants. UNICEF Canada may revise the survey questions from time to time, but the intent is to sustain a survey that can be repeated across communities and over time to compare progress. Therefore, any future iterations will be as limited as possible, and communities are not permitted to alter the survey questions except for certain demographic questions.

Core questions

In all communities, the full set of core questions must be used and the corresponding data publicly reported. This helps ensure fidelity and gives the community the opportunity to compare results with other communities and with national and regional data, to learn from and help each other create conditions for young people to thrive.

Choosing module questions

In addition to the core questions, optional module questions are available to add to the survey questionnaire. Each community may decide which, if any, of these module questions they want to include based on the particular interests and concerns in their area, including those of young people. It is generally recommended that no more than five module questions be added to the survey to maintain a reasonable completion time and, by extension, the likelihood that most young people will complete it.

Sample size

The larger the final sample, the more certain you can be that you are accurately reflecting the youth population. However, a larger sample size does not guarantee that the survey data will be “representative” of the youth population ages 9 to 18 in your community. The more you work to include diverse young people (by gender, gender identity, ethnicity, socioeconomic conditions, age, etc.), the more representative your survey findings are likely to be. If your survey method is not based on random/probabilistic sampling but on convenience sampling (e.g., an open invitation) and/or snowball sampling, which is often the case for the Community Child and Youth Well-being Survey, then your aim is to achieve a profile of young people that is based on a good demographic and geographic cross-section and of sufficient size so that the data can be used with confidence and some caution in drawing conclusions.   

Your research/data partner should be able to advise you on an overall target sample size and the sizes of demographic sub-groups needed to disaggregate the sample. You should try to get as large a final sample of youth as possible, depending on your resources. This may require additional effort if you want to compare one demographic subset to another. For example, if you are interested in identifying disparities between one neighbourhood and another, or between groups of young people according to age cohorts, ethnicity or gender, you will need to be sure that at least 100 surveys are completed by each sub-group within the final sample. As a starting point, you should be able to identify the size of the total youth population in your community using census data (e.g., community profiles) or local community data to help decide on a target total sample size and the sizes of each desired sub-group.

Demographic profile

The main purposes of asking young survey respondents about certain demographic identifiers are to understand how well the survey responses represent the community youth population and to use the data to identify where there are inequalities between them. There is a minimum set of required demographic questions that enable communities to compare survey results between groups of children according to these demographic identifiers (unless the research/data partner finds upon the close of the survey that privacy would be violated due to small response sizes for particular demographic groups). These identifiers align with the dimensions of equity and non-discrimination outlined in article 2 of the Convention on the Rights of the Child. They include gender/gender identity, disability, ethnicity, language, age and other appropriate identifiers for a children’s survey. 

Communities can decide to add or modify demographic questions based on their context and equity concerns. The total number of these questions should be expansive enough so that young people feel included and validated, but limited enough that the length is manageable for young survey respondents (or risk young people abandoning the survey at this stage) and the sample sizes are likely to be large enough to report distinctions. The chosen demographic questions could be aligned with census data for the community and adjusted with advice from youth, the broader community and considerations raised by the research/data partner, including relevance, acceptability, clarity and survey sampling. The choices are important, not least because they influence young survey respondents’ perceptions of inclusion and identity validation. Broadly speaking, while demographic survey questions are continually evolving and surveys vary from each other in important ways, the Community Child and Youth Well-being Survey has been carefully attuned to be as sensitive and clear as possible for young people. It should not be modified without careful consideration. Below are some considerations for deciding on demographic questions.

Based on survey experience, some young people may have difficulty interpreting the meaning of a question asking about “gender identity” and may interpret it as a question about sexual orientation. Since gender identity is an important mediator of many aspects of well-being and is typically included in a set of demographic and identity-related questions, providing a clear, youth-friendly explanation of this question is important. There is also potential difficulty in distinguishing between “country of origin” and “ethnicity” , and a clear explanation of these questions will be helpful.

Keep in mind that open-ended demographic response choices may be important to be respectful and inclusive, since close-ended response options, such as check-boxes and drop-down menus, have inherent limitations if they cannot be exhaustive. Including options such as “Prefer not to answer” and “Prefer to describe as” (with an open response) might be appropriate. However, open-ended responses increase the likelihood that responses cannot be included in the data due to interpretive and coding challenges. 

Should the survey ask for the name of the respondent’s school? This is a challenging consideration. On the positive side, asking this question can help schools secure important insights that could address their individual conditions, particularly if they do not administer their own school climate surveys, and school conditions frequently emerge as a dominant theme in relation to well-being. On the negative side, asking this question runs a material risk of increasing confidentiality concerns among young people; schools may not welcome this question if public data are used to make unfair comparisons between schools; and securing an adequate sample size by school will be a considerable challenge. This question can be explored by speaking with school leaders and youth advisors about the implications of this choice.

Refer to the section on Respecting diversity to consider if protocols are warranted to potentially distinguish and represent certain groups of children. Provincial and territorial human rights bodies provide some guidance, and Statistics Canada’s surveys offer models to consider, for example:

• Disaggregated demographic data collection in British Columbia: The grandmother perspective. British Columbia Office of the Human Rights Commissioner, September 2020
• Data Standards for the Identification and Monitoring of Systemic Racism. Government of Ontario: Anti-Racism Directorate, 2020 
• Ontario Nonprofit Network: A Framework for Nonprofit Data Strategies, 2023
• #DataBack: Asserting and supporting Indigenous data sovereignty
• Statistics Canada: Balancing the Protection of Confidentiality with the Needs for Disaggregated Census Data
• Powered by Data: Doing Data Ethically
• Urban Institute: Taking an Equity Lens to our Data Practice
• McBride, Kate. Data Resources and Challenges for First Nations Communities, 2018

ALIGNING WITH CANADIAN CENSUS DATA

Some communities may be able to access a population count for the total potential survey age population (9 to 18 years old) from census tables (see https://www12.statcan.gc.ca/census-recensement/2016/dp-pd/prof/index.cfm?Lang=E). If the community is not a “census geography,” the survey team can define the boundaries for the survey and apply the closest census geography match to yield a total potential survey population for the desired age range. Census population data can enable the survey team working with the research/data partner to select specific geographic boundaries for sampling design, extrapolate an estimate of the current year survey population, and compare survey response results (according to demographic profiles) to the census profile of the community to determine how well the sample has represented young people in the community by age, gender and other selected variables.

For example, Ottawa has data tables of different geography levels, broken down by single years of age, age group and gender. 

From Table 98-400-X2016004, select Ottawa as geography and add the age cohorts from 9 to 18 to yield a total potential survey population of 108,210 in the census base year.

Ethics approval

The survey questionnaire and Consent and Information Release Forms were reviewed by and received ethics clearance through the University of Waterloo, Office of Research Ethics (in 2021). Your community will decide whether you need to submit these and/or additional materials you develop (such as the survey completion instructions) through your own ethics clearance procedure or that of your research/data partner. If you are promoting or delivering the survey through your schools, some school boards/districts require their own ethics review and the age of consent may be different than 13 years old.

Obtaining participant consent

Before beginning the survey, participants must be asked to acknowledge that they have read the information in the Consent and Information Release Forms and understand and accept the proposed uses of the survey information they will be providing. For children ages 9 to 13, a parent or guardian will need to consent (and the children provide their assent), while youth ages 14 to 18 can consent for themselves. Each of these two groups has a distinct form (available upon execution of the UNICEF Canada Licensing Agreement). The research/data partner will need to exclude survey responses by respondents that did not sign and submit the form. The forms can be delivered in different ways depending on the survey delivery method. For instance, they can be posted on a survey questionnaire platform, and when the young person clicks an “accept” button, they can access the survey. For more information about young people’s consent, assent and other resources, see UNICEF’s resources for Ethical Research Involving Children.

Setting up the survey platform

This stage involves core functions to deliver the survey to young people: 

• Provide information about the survey, including its purpose and who is hosting the survey.
• Provide the two versions of the Consent and Information Release Forms (for the two distinct age groups) and a means to submit completed forms prior to starting the survey.
• Provide a link or other means to receive the survey.
• Provide information to access youth-friendly help if the survey raises issues or distress. 

The survey is typically delivered through a community website, web page or other type of host platform (see Appendix for recommended web content elements and sample structure). Functionally, web design should be cellphone-friendly and support accessibility, and the format and language should be clear and youth-friendly. 

Alternatively, the survey and consent forms might be delivered through an online survey platform, SMS on cellphones, in schools or in person – even paper-and-pencil approaches might be options in some community contexts (e.g., a door-to-door or community Street Team method). The survey might also be provided in accessible versions. Some survey software packages have applications to increase accessibility, such as audio formats. It is also good practice to make the survey available on paper for young people who may not have Internet access or prefer pencil and paper, and be prepared to upload the responses into the database. Any delivery method must preserve confidentiality and privacy.

Young people consistently say they like these aspects of the survey design:

• Wide and comprehensive range of topics (“No one ever asked me before”)
• Varied format with many multiple choice questions and a good range of answer options
• Relevance of the questions
• Anonymity of the responses
• Non-judgmental design of the questions
• Ability to skip questions
• Use of colour and emojis
• Completion rate bar or count

UNICEF Canada provides the survey questions and the Consent and Information Release Forms in both English and French. The community may also translate these documents and supporting content into other languages relevant to young people in the community using a high-quality service provider. 

Some of the questions might make some children or youth anxious, for example, because they ask about experiences of being bullied, perceptions of safety in their neighbourhood and feeling socially isolated. The survey delivery method must include relevant contact information for support should participation in the survey create distress, such as a helpline or local counselling service. All of the questions have been used in other research studies without having any known short- or long-term impact on participants. 

Consider other key information to include (without over-complicating it), such as emphasizing the confidentiality of the survey; privacy protection; and the ability to skip any question or choose not to submit the survey or stop completing it at any time. Encourage adults to provide young people with privacy as they complete the survey. Some communities also provide incentives (such as a prize draw upon survey completion); contact information for a support person to answer questions; links to helpful resources in the community; an optional follow-up “survey of the survey” to ask for young people’s feedback on the survey experience after completion; and a QR code to easily share and access the survey. Young people can help shape the content and the delivery option(s) and test the functionality of the survey presentation.

NOTE: A “survey of the survey” can be offered as an option to young people who have completed the survey to ask them about their survey experience – information that can be used to inform future planning and iteration. Questions can include what they liked best about the survey, how they would make it better, whether the demographic questions were inclusive, how they found out about the survey, why they completed the survey, how they think their data should be used by the community, and anything else they’d like you to know. See Appendix for an example. 

The research/data partner will most likely be involved in co-designing the survey “front-end” that the young respondents experience and will need to set up the “back-end” to host and compile responses for the pre-determined language versions. For instance, the survey can be set up using a software package, such as Survey Monkey or Qualtrics. Survey respondents must be able to withdraw from the survey by ending their participation without submitting the survey, and no record of their responses should be retained. The research/data partner must also provide raw data protection, storage, access and cleaning. 

NOTE: Delivering the survey to young people using SMS (text through cellphones) is an option that can be more accessible and inclusive, since young people typically have more access to a cellphone than a laptop and the Internet. The advantages also include automatic delivery of incentives (such as an electronic gift card upon survey completion), a feature to share with a friend through text, and contact information for (carefully managed) follow-up information and opportunities (though the contact list will be subject to non-response and attrition). SMS information must be subject to privacy regulation.

Preparing for data security, privacy and sharing

Having the appropriate data security and privacy standards in place is imperative to ensuring the identities and privacy of children and youth are protected and that there are no ensuing risks to other aspects of their rights and well-being. Following standards also protects all survey parties from risk and helps meet ethical guidelines. 

All survey responses are anonymous because participants must not be asked for their name or address at any point. But the research/data partner must also “clean” the raw data to ensure that responses are suppressed before sharing results if there is any chance that a response (particularly to open-ended questions) or a small sample number could identify a person. For instance, survey data based on a particular demographic identifier, such as gender identity or rural residence, are often not shared due to very low response rates. According to privacy legislation, data must be password-protected and stored on a secure server, with access limited to the principal investigator(s) and supervised research analysts. Any paper versions of completed surveys must be stored in a locked cabinet, and the plan to securely dispose of the data must also be identified. Your research/data partner must be able to ensure these standards and protections. It is recommended that these standards and protections are spelled out in a contract with the research/data partner.

The UNICEF Canada Licensing Agreement includes and requires a Data Sharing Agreement (DSA) between the community, the research/data partner and UNICEF Canada. All signatories are obligated by the same responsibilities outlined in the DSA. The basic goal of the DSA is to govern the ownership, sharing and use of protected and cleaned data so that children’s rights are protected (particularly privacy) and the data are shared and used in their best interests.

Basic principles for a DSA include:

• Protect the identities of all survey participants; only share data that have been de-identified and stripped of any variables that do not meet the threshold for ensuring participants’ privacy.
• Oblige parties to use the data only for the advancement of child and youth well-being, not for any commercialization or personal gain.
• Define the ownership and obligation of community partners to notify all parties of any products that they create using the data (e.g., fact sheets, reports).
• Commit parties to properly acknowledging the Community Child and Youth Well-being Survey partnership and recognize these partners and source of the data on any products that are released.

The DSA also specifically provides for aspects of data sharing and use, including transfer, storage, security and destruction of the collected data.

If the community plans to share the protected and cleaned data with other trusted parties, these parties should also sign the DSA. Sharing raw data should be limited to research partners who can secure and protect the data for analysis that will produce privacy-protected, cleaned data and information. For other users/signatories of the DSA, the shared data should already be privacy-protected and cleaned by the research/data partner as described above. 

If the research/data partner is required by their academic institution or provincial/territorial legislation to use a different DSA, UNICEF Canada may approve the use of an alternate DSA or the modification of the DSA in this toolkit, provided that requirements are at least as stringent.

The format in which the data are shared can be negotiated with the research/data partner and might be in an Excel worksheet or in a specific statistical package. 

The process to complete the DSA is as follows:

1. The community partner(s) sign.
2. UNICEF Canada signs.
3. The research/data provider signs.
4. The research/data provider returns the completed DSA to the community partner(s) and to UNICEF Canada.
5. The research/data provider arranges for the data to be transferred to the community partner and to UNICEF Canada in the preferred format(s).

If persons or organizations who did not sign the original DSA wish to receive the raw data, you must decide whether they are trusted and whether their proposed data use is in the best interests of children. If so, the requesting organization must sign a Data Request Application.

See Appendix for the Data Request Application.