Equity, diversity and inclusion considerations at each stage of the research process
NSERC is acting on the evidence that achieving a more equitable, diverse and inclusive Canadian research enterprise is essential to creating the excellent, innovative and impactful research necessary to advance knowledge and understanding, and respond to local, national and global challenges. This principle informs the commitments described in the Tri-Agency Statement on Equity, Diversity and Inclusion and is aligned with the objectives of the Tri-Agency Equity, Diversity and Inclusion Action Plan.
This document is designed to assist the research communities served by NSERC with embedding equity, diversity and inclusion (EDI) considerations relevant to each or any stage of the research process (figure 1): research questions, design of the study, methodology and data collection, analysis and interpretation, dissemination of results. It is an evolving document that will be enhanced and strengthened over time, and additional examples will be made available. We welcome your comments, examples and advice in this regard at firstname.lastname@example.org. For information regarding EDI in research teams, please refer to the Guide for Applicants: Considering equity, diversity and inclusion in your application.
Considering equity, diversity and inclusion in the research process, where relevant, promotes research excellence by making it more relevant to society as a whole, ethically sound, rigorous, reproducible, and useful (Tannenbaum et al. 2019). It also fuels innovation through scientific discovery and by opening up new areas of research. In this context, EDI considerations promote research excellence by:
- expanding the applicability of research findings and new technologies across a wider segment of society (examples 1, 4, 8);
- helping to reveal implicit assumptions related to research that otherwise might go unnoticed and unchallenged (example 2);
- helping to mitigate biases by conducting inclusive research and improving technologies (see examples 4, 8 and 11);
- supporting research outcomes that fairly benefit communities most impacted by the research (example 10);
- questioning biased norms and stereotypes (example 2);
- preventing overgeneralizations of findings (example 3, 8, 9) that can be harmful and/or misleading; and
- improving reproducibility of research findings. Reproducibility can be more difficult when diversity-related variables are relevant to research but are not reported (example 7 and 9).
When equity, diversity and inclusion are not considered in the research process where relevant, research results may be of lower quality and can lead to harmful outcomes. For example, when products are engineered based on a particular standard or on subjects assumed to be representative of the population as a whole, it can lead to serious consequences such as decreased safety and security or other inequities for some groups (see examples 1 and 8).
Example 1: The trouble with assuming a standard body size and type in auto safety testing
Crash test dummies used in auto safety testing commonly use what is assumed to be a standard adult male model that is simply scaled for varying heights and weights to account for people of all sizes. However, female bodies are not just a smaller version of a male body. An analysis of automotive crash data from 1998 to 2008 in the United States revealed that, even after controlling for weight and body mass, the odds of being severely injured were 47% higher in belt-restrained female drivers than in belt-restrained male drivers involved in a comparable crash (Bose and Segui-Gomez, 2011). The same is true for obese individuals and the elderly, who are at a greater risk of serious injuries in car crashes, as well as pregnant individuals who have a high risk of fetal injuries even in minor crashes (Schiebinger et al. 2011-2020). Thus, inadequate critical reflection on implicit assumptions embedded in auto-safety testing—the tendency to assume a “standard” male body adequately represents all human bodies—has resulted in inequities for the majority of car users. Missing from research are crash-test dummies that accurately model a diversity of bodies, in terms of their respective geometry, muscle and ligament strength, spinal alignment, dynamic response to trauma and mass distribution.
Example 2 illustrates how inequitable social norms have permeated into genetic research leading to unfounded assumptions which were later proven to be inaccurate.
Example 2: An overreliance on common assumptions in developmental biology has led to gaps in knowledge in ovarian development
In the 1990s, experts working on the key mechanisms of mammalian sex determination had come to a consensus that the mechanisms controlling testis development were the key to understanding sex determination overall. The SRY gene, located on the Y chromosome, found to control testis development, was declared the ‘master gene’ in control of sex determination. Ovarian development was considered a “passive” and “default” pathway and consequently the mechanisms of ovarian development received very little attention. Some members of the research community had been raising concerns about this lack of attention to ovarian development beginning in the mid-1980s, arguing that this lack of attention resulted from the imposition of gendered social norms about typical male and female roles in reproduction (Richardson 2008; 2013). By the mid-1990s, pathways of ovarian development were reconceptualised as “active” and research began to focus on the mechanisms involved in ovarian development and maintenance. By the early 2000s, the “master gene” theory controlling sex determination had largely been abandoned, replaced by a more comprehensive and complex understanding of sex determining pathways involving many factors.
You are invited to consider your work through a critical EDI lens from the initial framing of research questions to the dissemination of findings. The goal is to encourage greater reflection on how your research could be strengthened by integrating EDI considerations where relevant. Applying an EDI lens means systematically examining how diversity factors such as sex (biological), gender (socio cultural), race, ethnicity, age, disability, sexual orientation, geographic location, among additional possible relevant factors, and their intersections may affect the research questions, design, methodology, analysis, interpretation, and dissemination of results.
NSERC acknowledges that EDI considerations may not be applicable in the context of some proposed research, but nonetheless encourages you to fully consider their relevance, as they apply to more areas than one might think. Thoroughly reflecting on the type of data collected and who might be impacted by the research findings is critical before concluding that EDI considerations are not relevant. Generally, research that involves or impacts human subjects, organisms capable of differentiation, or their tissues or cells can benefit from such considerations.
Guiding questions for incorporating EDI considerations in your research
The intent of the following questions is to provide guidance on what it means to critically reflect using an EDI lens, by offering examples of considerations or best-practices for each stage of research. The questions are not exhaustive and some may not be applicable to your research; however we invite you to reflect on how your research may be strengthened by considering those that are relevant.
- Does your literature review address relevant EDI considerations?
- What keywords could be used in your literature review to gain deeper and broader knowledge of who might, or might not, be impacted by, or contribute to, the research?
- Are certain diversity factors and/or intersections known to affect the phenomenon of interest?
- What are the relevant knowledge gaps? Have previous studies failed to adequately incorporate relevant diversity factors and/or omitted investigating their intersections?
- How will your research questions and the subsequent findings from your study apply to the needs or experiences of various groups? Who benefits from the findings and/or product developed? Have you considered which populations may experience significant unintended impacts (positive or negative) as a result of the planned research?
- Who should be consulted about the needs and wishes of the group under study? (subjects and/or users)
- What contextual factors are relevant and important, and what may be overlooked without a conscious, intersectional integration of these considerations?
- Have you made assumptions regarding certain diversity factors? Are these based on empirical evidence?
Example 3: Important gaps in knowledge can lead to an inaccurate extrapolation of findings
Over-generalization of research findings is an issue in many disciplines. In psychology research, the ease of access to subjects where the research is conducted has led to the vast majority of psychology research being done on Western populations and often on undergraduate student populations. Indeed, a survey of some of the top psychology journals from 2003 to 2007 found that 96% of subjects used in studies were from Western countries, while representing only 12% of the world’s population (Arnett, 2008). Research articles routinely assume that their results are broadly representative while no evidence supports this assumption (Henrich et al. 2010). More research on diverse and inconvenient subject pools and careful thought on how broadly specific results apply are needed to put theories of psychology on a firmer empirical footing.
Example 4: Taking women into account in the design of energy-efficient buildings
The regulation of temperatures inside buildings is based on a model of thermal comfort for which the primary variable is the metabolic rate of building occupants. This variable has been based on a standard value of metabolic rate, which represents the average male. By showing the effects of not considering female metabolic rate on thermal demand, Kingma and van Marken Lichtenbelt (2015) make a case for changing the current models of thermal comfort to one that is more inclusive. This work is a step forward not only to reducing the bias that overlooks female thermal comfort in office buildings but it can also help better predict building energy consumption. More work is needed to create thermal comfort models that are more representative of all occupants by taking into account additional diversity factors such body weight and age.
Design of the study
- Will members from the population/community of interest be invited to help shape the objectives of the study?
- Which diversity factor(s) could be embedded to strengthen the study? Why would you consider or not consider these factors and their intersections?
- What is your position relative to the context of the research problem or the subjects themselves? What biases related to identities, privileges and power imbalances could impact the study? How will they be mitigated?
- Does the proposed research follow relevant protocols and/or best practices on how, why and by whom research is to be conducted or with relevant or impacted communities and how knowledge is accessed and shared (such as in Indigenous communities)?
- In cases that involve a research site, have you determined which Indigenous government or community has jurisdiction over or interests in the research site? Have you genuinely engaged the community, considered their own research priorities and interests in the co-production of knowledge (even if you are from the community)? Are there opportunities for reciprocity in the design of the study such that both the community and the researcher benefit (see example 5 and key resource F)?
Example 5: Developing and mobilizing local knowledge
The Dehcho Collaborative on Permafrost (DCoP) is an initiative that combines scientific and Indigenous knowledge on permafrost to improve monitoring, adaptation, process understanding and prediction of permafrost thaw in the Dehcho region in the Northwest Territories. Members of the DCoP research team as well as community members are co-developing a number of knowledge-based resources including real-time data, interactive maps and modelling data demonstrating rates and patterns of permafrost thaw, land cover change and hydrograph response for different scenarios of warming. These resources are important for the communities to manage and respond to the disrupted hydrological cycle and ecosystems resulting from permafrost thaw and land cover change in the region.
Methodology and data collection
- How will you obtain information for each diversity factor under consideration? How will privacy be protected?
- How will you ensure that the research participants reflect the diversity categories that are included in the research design?
- If the analysis is based on existing data sets, is there potential for bias due to the cultural and/or institutional contexts in which the data were generated?
- For Indigenous research, how will data collection and monitoring be conducted using established guides for/by Indigenous Peoples?
- How will bias be monitored, mitigated, and recorded?
- Do EDI considerations impact relations between those conducting the research and those participating in it, in ways that affect data collection (see example 6)? How will this be identified and mitigated?
- Does your proposal consider the different forms of support required (e.g. financial, logistical, cultural, linguistic) to ensure that the individuals or communities involved in the research are able to meaningfully participate in it?
Example 6: Bias introduced by the sex of researcher
Growing evidence in research where research team members interact with research subjects shows that the sex, gender, or race of the team member can impact study outcomes with both human subjects (Davis et. al. 2010; Davis and Silver 2003) and non-human animals (Sorge et al. 2014). For example, research on pain experience demonstrated the presence of male researchers blunted pain behavior of laboratory mice and rats, a response that was not observed in the presence of female researchers (Sorge et al. 2014). This difference was found to be due to stress-induced analgesia caused by the scent of male researchers. The lack of awareness of this confounding variable may have resulted in numerous studies reporting inaccurate results, highlighting the importance of accounting for the sex of the person collecting the data in this context.
Analysis and interpretation
- Where appropriate, have you:
- Presented your data, disaggregated by diversity factors?
- Evaluated whether diversity factors and/or their intersections have an impact on outcomes?
- Statistically tested your data to determine whether the magnitude of effects is different for each diversity factor and their intersections?
- If diverse groups are involved in the research, will they have the opportunity to participate in the interpretation of the data and the review of research findings before the completion of the proposed research?
- If the results are inconclusive, will they be reported in a disaggregated format for future studies?
- Are you applying the findings of your research to the population as a whole, when your method and design were in fact limited to certain groups?
- Did you report the diversity factor(s) used in the study to ensure that experiments are reproducible and findings are not over-generalized (see example 3 and 9)? Have you considered including this information in the title, abstract, keywords?
- If relevant diversity factors were not included in the study, did you acknowledge that it is a limitation of the study? Did you discuss the implications of the lack of such analyses on the interpretation of the results?
Example 7: Copepod research highlights the importance of disaggregating data by sex
In copepods, a small aquatic crustacean, disaggregating respiration rate data by sex revealed different responses to increased partial pressure of carbon dioxide (pCO2) levels between male and females. In a study to further understand how these animals respond to ocean acidification, Cripps et al. (2016) found that respiration rates of male copepods decreased when exposed to high pCO2 levels while in females they increased under the same conditions. Failure to account for this sex difference by pooling the data would have led to the false interpretation that high pCO2 levels had no effect on the respiration rate. See figure 1 in the article by Tannenbaum et al. (2019) for a visual depiction of this example and of the hazards of pooling data from both sexes.
Example 8: Intersectional accuracy disparities of gender classification products
Concerns have been raised about the imagined neutrality of machines, data, and algorithms, with research findings suggesting that cultural bias can be built into such technologies. Buolamwini and Gebru (2018) evaluated the accuracy of well-known commercially available gender classification products on women and men of different skin types. They analyzed the data not only by gender and skin type separately but also their intersection, resulting in 4 intersectional subgroups: darker women, darker men, lighter women and lighter men. They found that, overall, women were misclassified at a higher rate than men, and darker-skinned individuals were also classified with greater error rates than individuals with lighter skin types. However, accuracy rates were lowest when classifying darker women. Error rates were up to 34% for darker-skinned women, 12% for darker-skinned men, and only 7% and 1% for lighter skin women and men, respectively. This study highlights the importance of inclusive product development and testing to reduce bias and to achieve more equitable systems. The Gender Shades video produced by the MIT Media Lab provides a good overview of this example.
Example 9: Reporting sex in animal research
Sex is still often not reported in animal research today in many disciplines, and when it is, females are often underrepresented (Beery and Zucker, 2011). A survey of journal articles within specific biomedical subfields reported that 22 to 42% of articles did not specify the sex of animal subjects in select neuroscience, physiology and interdisciplinary biology journals. In marine species, a review of ocean acidification studies on key taxonomic groups (Echinodermata, Crustacea, Mollusca and fish) reported that 85% of studies failed to consider sex at all even though sex-based differences in response to ocean acidification have been documented (Ellis et al. 2017). Failing to report on the sex of study animals may lead to inaccurate conclusions and decreases reproducibility. If sex determination cannot be made, this should be disclosed.
Dissemination of resultsFootnote 1
- What means of dissemination will be most effective in reaching those who will use and/or could benefit from the findings?
- How will inclusivity be integrated in dissemination? Will accessible formats be used? Will anyone who took part in the research receive a summary of the research findings and/or be invited to a presentation about the work?
- Does the dissemination plan consider the language of use (i.e., English and/or French) or other appropriate languages depending on the groups identified?
- Does the dissemination material take into account gender sensitive and inclusive communication (e.g. gender-neutral language or unbiased content)?
- Are the dissemination strategies the product of collaborative efforts with a diversity of input or have they been envisioned in a narrower focus?
Example 10: Effective dissemination strategies in an Arctic ecology research program
The Centre for northern studies (Centre d’études nordiques) has led to the production of films, articles and material for and with Indigenous northern communities on water and environmental resources. The program of research involves youth in Nunavik by mixing traditional and local knowledge with Western science to stimulate and nurture Inuit youth’s interest in science-related careers, to promote environmental stewardship, and to build better relationships between community members and research teams. Elders, local guides and coordinators, youth and members of research teams all worked together to train lnuit youth in environmental data collection. Materials are available in English and Inuktitut.
Example 11: Maximizing the impact of research through targeted dissemination of findings
As part of the Gender Shades project, a short video was developed explaining the research results of Buolamwini and Gebru (2018) and datasets were made available to companies that developed and commercialized the gender-classification products tested in the study in the interests of improving the accuracy of gender classification products for women and men of different skin types (see example 8 above).
Incorporating EDI considerations in NSERC research proposals
You are encouraged to incorporate relevant EDI considerations in the proposal section of your application as appropriate. The goal is not to be prescriptive about the design of your study; it is to encourage you to reflect on how EDI consideration can strengthen your research. NSERC acknowledges that these considerations may not be relevant to every field in the natural sciences and engineering. If EDI considerations do not apply to your research, depending on the funding opportunity you are applying to, you may be asked to explain why they are not relevant in a specific section of your application.
The goal of this document is to help illustrate what it means to apply a critical EDI lens to your research. You should refer to the instructions and information materials of the funding opportunity you are applying to for additional guidance on how these considerations should be reflected in your application and on how it will be evaluated.
- NSERC’s Guide for Applicants: Considering equity, diversity and inclusion in your application
- Stanford University’s Gendered Innovations (available in English only)
- Gendered Innovations 2: How Inclusive Analysis Contributes to Research and Innovation (available in English only)
- A review of Indigenous knowledge and participation in environmental monitoring: This review poses EDI-related research questions and defines the terminology related to the diversity factors at play. They also discuss how people in positions of power impact the Indigenous peoples’ participation in environmental monitoring. (Thompson, K.-L., T. Lantz, and N. C. Ban. 2020. A review of Indigenous knowledge and participation in environmental monitoring. Ecology and Society 25(2):10) (available in English only)
- Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans – TCPS 2 (2018), which includes a chapter on Research Involving the First Nations, Inuit and Métis Peoples of Canada (Chapter 9)
- Towards Reconciliation: 10 Calls to Action to natural scientists working in Canada, the authors “challenge the scientific community to recognize that reconciliation requires a new way of conducting natural science, one that includes and respects Indigenous communities, rights and knowledge, leading to better scientific and community outcomes.” Among these calls to action are the following items in particular relating to research design: 1) Understanding the socio-political landscapes around research sites; 2) Recognizing that generating knowledge about the land is a goal shared with Indigenous Peoples and to seek meaningful relationships and possible collaboration for better outcomes for all involved; 3) Calling on natural scientists to enable knowledge sharing and knowledge co-production; and 4) Calling on natural scientists studying animals to seek out advice from Elders for respectful ways of handling animals.(Wong, C.; Ballegooyen, K; Ignace, L.; Johnson M.J; Swanson H. (2020) Towards reconciliation: 10 Calls to Action to natural scientists working in Canada. FACETS 5(1):769-783) (available in English only)
- The Université du Québec en Abitibi-Témiscamingue has created an EDI resource site for researchers. The site includes training resources and practical guides on integrating principles of EDI and intersectionality into research (available in French only).
- Gender Shades website (available in English only) which includes a short video (English audio with French subtitles)
Arnett, J.J. (2008) The neglected 95%: Why American psychology needs to become less American. American Psychologist 63(7):602-614
Beery, A.K., and Zucker, I (2011) Sex bias in neuroscience and biomedical research. Neuroscience & Biobehavioral Reviews 35(3):565-572
Bose, D., and Segui-Gomez, M. (2011). Vulnerability of female drivers involved in motor vehicle crashes: an analysis of US population at risk. American Journal of Public Health, 101 (12), 2368–2373.
Buolamwini, J., and Gebru T. (2018) Gender shades: intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81:77–91.
Cripps G, Flynn KJ, Lindeque PK (2016) Ocean Acidification Affects the Phyto-Zoo Plankton Trophic Transfer Efficiency. PLoS ONE 11(4): e0151739. https://doi.org/10.1371/journal.pone.0151739
Davis, D.W., and Brian D. Silver, (2003), Stereotype threat and race of interviewer effects in a survey on political knowledge, American Journal of Political Science, 47 (1): 33-45.
Davis, R.E., Couper, M.P., Janz, N.K., Caldwell, C.H., and Resnicow K. (2010) ‘Interviewer effects in public health surveys’ Health Education Research, 25(1): 14-26.
Ellis, R. P., Davison, W., Queiros, A. M., Kroeker, K. J., Calosi, P., Dupont, S., Spicer, J. I., Wilson, R. W., Widdicombe, S. and Urbina, M. A. (2017), Does sex really matter? Explaining intraspecies variation in ocean acidification response, Biology Letters, 13(2) (https://doi.org/10.1098/rsbl.2016.0761).
Heenrich, J., Heine, S., and Norenzayan, A. (2010) Most people are not WEIRD. Nature 466, 29.
Kingma, B., and van Marken Lichtenbelt, W. (2015) Energy consumption in buildings and female thermal demand. Nature Clim Change 5, 1054–1056. https://doi.org/10.1038/nclimate2741
Richardson, Sarah. 2008. When gender criticism becomes standard scientific practice: The case of sex determination genetics in Gendered Innovations in science and engineering (edited by Londa Schiebinger), Stanford University Press: pages 22-42.
Richardson, S. (2013) Sex Itself: The search for male and female in the human genome. Chicago: University of Chicago press.
Schiebinger, L., Klinge, I., Paik, H. Y., Sánchez de Madariaga, I., Schraudner, M., and Stefanick, M. (Eds.) (2011-2020). Gendered Innovations in Science, Health & Medicine, Engineering, and Environment (genderedinnovations.stanford.edu).
Sorge, R.E. et al. (2014) Olfactory exposure to males, including men, causes stress and related analgesia in rodents. Nature Methods 11(6):629-32.
Tannenbaum, C., Ellis, R.P., Eyssel, F., Zou, J., and Schiebinger, L. (2019) Sex and gender analysis improves science and engineering. Nature 575: 137 – 146.