Diversity, Equity and Inclusion are Central to Good Research

KEY Findings:

Below are five ways Ipsos is translating diversity, equity and inclusion concepts into survey practice, examining case studies for young adults, Black Americans, Hispanic Americans and diverse political attitudes. 

1. Translating DEI concepts into survey practice;
2. Meeting young adults where they are to boost representation;
3. Addressing historical research mistreatment and underrepresentation of Black Americans;
4. Recognizing that Hispanic and Latino Americans are not a singular group; and
5. Identifying and capturing the voices of hard-to-reach political groups.

The U.S. is at an inflection point as the country turns from long-term convention and disrupts historic norms. With the country growing ever more diverse and previously marginalized groups taking seats at the table, fully representing the rich tapestry of America has never been more complicated.

The latest 2020 Census highlighted the problems with measurement and data collection that can occur when a more diverse population, political events and societal change collide. At the heart of the challenges experienced by the Census are also fundamental questions facing public opinion and market research. How do we represent a changing population? And, more pressingly, what is representation in surveys?

The success of a survey depends on the ability to capture a wide range of diverse opinions, attitudes, and behaviors. “A survey is supposed to represent all voices,” says Michael Link, Chief Research Officer at Ipsos U.S. Public Affairs, “including those in the front shouting the loudest but also those standing back and whispering – as well as those who may have stopped participating altogether because they are tired of not being heard. Diversity, equity and inclusion are central to being representative.”

Translating DEI concepts into survey practice

To translate the underlying spirit of the diversity, equity and inclusion framework into a survey, it is important to start by questioning who might be missing at every stage of the research process. This includes identifying who the researchers want to reach, as defined by the research questions being asked, and outlining the potential survey universe of these individuals in the sample frame. Additionally, recruiting respondents should be done with an emphasis on designing questionnaires that are accessible to all. Finally, using weights can give a bigger voice to smaller groups, and therefore, every stage of survey development needs to be examined.

While these are important questions researchers should be asking in the aggregate to create more inclusive research and data, there are some well-researched challenges with specific hard-to-reach groups that are worth investigating on their own terms.

Meeting young adults where they are to boost representation

Young adults have been historically hard for survey researchers to reach because they tend to be more mobile or transient or live in harder-to-reach group quarters, like dorms. However, the past two decades introduced new challenges to accurately representing this group in survey research. The current under-30 cohort is the first fully digital and mobile generation, communicating through visuals and videos, making them harder to reach and less tolerant of long survey formats.

Ipsos has employed several solutions to address the growing challenges of reaching the 18-to-29 crowd. Increasing incentives and additional reward points at random intervals to people in this age group allows Ipsos to encourage participation in surveys. But monetary enticements simply aren’t enough. For example, Ipsos’ KnowledgePanel – one of our flagship survey research services- has turned to research colleagues in marketing research, polling and other fields for solutions, using techniques such as user experience (UX) practices to learn how best to engage this age group. Additionally, Ipsos researchers are finding ways to leverage visuals and video communication to collect data and use text and SMS for survey invitations and reminders to meet the under-30 crowd where they are (which is likely on their phones).

Source: Ipsos Tech and Society polling April 20-24, 2023, among 1,008 U.S. adults

Addressing historical research mistreatment and underrepresentation of Black Americans

Black Americans are also a group that survey research and the Census Bureau struggle to reach. That’s in part because of the history of research abuses Black Americans have suffered. The trust that’s been lost in this community is tied directly to lower levels of engagement with research efforts and prevailing feelings of disenfranchisement and skepticism. Additionally, making sure that researchers are reaching a representative sample within the Black community is also a challenge for polling. In short, if you’ve been let down – to say the least – why engage?

Ipsos takes non-response and survey efforts to reach the Black community very seriously. The KnowledgePanel has taken a holistic view of addressing these challenges, using different levels of incentives and putting resources toward higher levels of incentives to harder-to-reach individuals within the Black community. Beyond that, Ipsos has also used research-on-research, investing in working groups to understand how best to measure race and ethnicity and what people’s experience with race in America is like. Additionally, Ipsos has reassessed communication strategies and tailored materials and appeals so that Black Americans are centered. Seeing yourself in your work matters, and Ipsos is putting money into ensuring Black voices are seen and heard in ours.

Source: American Communities Project/Ipsos poll

Hispanic and Latino Americans are not a singular group

Hispanic and Latino Americans are a heterogeneous group. Yet researchers and the media treat and often discuss this community in broad homogenizing brush strokes, often miscounting and misrepresenting them.

To ensure research is representative of all members of the Latino population in the U.S., it is essential to understand who historically has been missing or more difficult for researchers to reach. For example, new immigrants can be more mobile and harder to contact via email, with potential language barriers related to data collection. Relatedly, as new immigrants come to the U.S., the Latino population is now more diverse, with many different countries of origin, cultures, social and political attitudes, and behaviors represented. These factors can change how people within this community self-identify within race and ethnicity questions. To that end, the growing xenophobic political rhetoric from some quarters of the political and governmental ecosystem has had spillover effects in research, with some worrying that their data and information won’t be protected. All of these factors must be taken into account when considering how to reach and represent all members of this community in research.

Ipsos has been working on solutions to elevate Latino voices, investing in research to understand better ways of asking about race and ethnicity. Additionally, Ipsos’ communication with members of the Latino community has emphasized how respondents’ identities will be kept private and protected while also leveraging culturally attuned appeals to community interest versus just the individual benefits that participating in research can have for the individual.

Tailoring bilingual communications to geographies where primarily Spanish-speaking respondents are more likely to live allows Ipsos to obtain more Spanish-speaking Latinos too. Understanding that Hispanic and Latino Americans aren’t a homogenous group also means that statistical weighting across major subgroups within the Latino community is important to accurately represent this group and the country overall. Getting research right means employing multiple practices to consider where Hispanic respondents may not participate or respond at every stage of the research process, something Ipsos has been actively doing.

Source: Washington Post/Ipsos Latino poll

Identifying and capturing the voices of hard-to-reach political groups

Identifying diverse political opinions can be a challenge within polling. Take, for example, Make America Great Again (MAGA), a political slogan popularized by former President Trump. It can be easy to see in the news or on the street the famous red baseball hats the former president sells. But while this group can feel visible, systematically identifying and measuring this group rigorously is difficult.

Most Republicans who voted for Trump don’t identify with the MAGA label, which means that other demographic, behavioral and attitudinal variables need to be triangulated to define this group within the context of a survey.

There are several measures Ipsos has taken to improve sampling and outreach to this group. First, Ipsos in partnership with the New York Times worked on studies where outside-of-the-box questions were asked to help refine the identification of this group. In combination with a respondent’s past vote and political attitudes, these questions can be a creative way of zeroing in on a hard-to-pin-down group of Americans. In addition, we’ve worked backward and experimented with new weighting schemes to up MAGA representation in surveys successfully in the 2022 midterm cycle. It’s worth noting here that while identifying, sampling and weighting this subgroup has proved difficult for the polling industry writ large during presidential election cycles, the issues polling this group presents exist within the realm of election polling and typically don’t impact other areas of survey research.

Source: Reuters/Ipsos poll conducted Sept 6-7, 2022

Conclusion

Most of these practices are systems and methods to meet Americans where they are so they can best make their voice heard. Groups who are hard to reach have often felt let down and left out. To best make Ipsos research representative, here are five things to remember:

1. Proactively consider who might be missing at every stage of the research process.
2. For young adults, meet them where they are, using mobile, videos, and increased incentives to up representation.
3. Centering Black voices in research means investing in research to better ask questions about race and ethnicity, focus on Black stories in communication, and expand compensation for surveys.
4. Latino Americans aren’t a monolith; use weighting, outreach, translation, and better research questions to reflect that.
5. Political subgroups can be hard to identify. Ipsos is weighting, triangulating questions, and employing creative tools for identifying hard-to-reach groups.

Authors:

Michael Link
Chief Research Officer
Ipsos U.S. Public Affairs
[email protected]

Sarah Feldman
Senior Data Journalist
[email protected]

Bernard Mendez
Data Journalist
[email protected]

About Ipsos

At Ipsos we are passionately curious about people, markets, brands, and society. We deliver information and analysis that makes our complex world easier and faster to navigate and inspires our clients to make smarter decisions. With a strong presence in 90 countries, Ipsos employs more than 18,000 people and conducts research programs in more than 100 countries. Founded in France in 1975, Ipsos is controlled and managed by research professionals.