Image from Brian Dunaway.

Case Study

Rewriting a Graduate School Survey to be LGBTQ+ Inclusive

Overview

In 2020 the University of California Office of the President (UCOP) surveyed the graduate population on multiple items, including financial confidence, housing, and food security. The survey purpose was to compare the health and security of marginalized students to their peers. One of the cross sections was LGBTQ+ identity. Statistics show that 1 in 6 (15.9%) of young adults aged 18-23 identify as part of the LGBTQ+ community, and roughly 17% of college students identify as LGBTQ+. However, it is important to note that the data on the LGBTQ+ community is scarce, and LGBTQ+ identity is likely underrepresented in the few national surveys that are taken.

Problem

As a nonbinary graduate student attending University of California San Diego (UCSD) at this time, my experience was exclusionary, discriminatory, and offensive. Because this survey is exclusionary to many members of the LBGTQ+ community, the data and statistics from the survey would result in an inaccurate portrayal of the lived realities of LGBTQ+ students. The questions that were used created bias in the survey and had the potential to skew the final analysis.

The goal of this case study was to ensure that biases were removed from the questions, yielding a more accurate and inclusive survey for all members of the UCSD community, including LGBTQ+ students.

Users and Audience

The users targeted were LGBTQ+ individuals, specifically transgender and gender non-conforming (GNC) graduate students. Queer graduate students at UCSD made up 10% of surveyed population as of 2016, and transgender/gender non-conforming (GNC) graduate students made up 2% of surveyed population as of 2016.

Roles and Responsibilities

As a recipient of this survey during my time as a graduate student at UCSD, I contacted the Office of the President to present concerns on the inaccuracies of the gender and sexuality content. I was asked to propose solutions for future interactions of such surveys and work with the survey team to ensure a more inclusive experience. I served as the only UX Researcher for this project and did primary and secondary research to substantiate my proposal. Furthermore, I outlined a story board for a LGBTQ+ user completing this survey, highlighting pain points and exclusionary content. I then published my critique and proposed solutions on Medium for the UX Collective, as seen here.

Scope and Constraints

The creation of this template was limited in time and scope as it was originally crafted over emails to the Office of the President. Additional suggestions and proposals are included in the conclusions of this analysis.

Process

I notified the Office of the President and brought up my concerns about their research methodologies. The Office of the President requested if I had any specific solutions for the problems I presented. I looked at each question individually and found that two out of the four were obsolete or inappropriate to ask of participants. The other two questions that could be kept would need revisions to ensure they were using consistent terminology and updated identity labels. I decided to completely revise the demographic questions to something more inclusive, up to date, and applicable to the survey. I worked to rewrite the survey using updated definitions consistent within the queer community and major advocacy organization terminology (as seen in GLADD Media Reference Guide). The original questions and commentary can be seen below.

Process Map

The Original Questions and Commentary

How would you describe yourself? Options: Female, Male, Trans female/Trans woman, Trans male/Trans Man, Gender queer/Gender non-conforming, Different Identity, Decline to State

The reason why this is a problem is multilayered. First, there are mixed terms for sex and gender throughout this category, and there is no specification that this question wants one or the other. Sex is the classification of a person as male, female, or intersex. At birth, infants are assigned a sex, usually based on the appearance of their genitalia. Gender is a person's internal, deeply held sense of their identity. This may or may not correspond with a person’s sex assigned at birth. Gender and sex are related, but not the same.

In this question, “Female” refers to sex, but “Gender queer” refers to gender identity. In addition, “Trans female” or “Trans male” are not identities. Transgender is an umbrella term for people whose gender identity differs from the sex they were assigned at birth. As such, “trans” is an adjective for gender and not sex identification. Placing transwomen in a separate category from women implies that they are not really women but are some deviation.

There are many other problems with this, including how genderqueer and gender nonconforming are put in the same category yet nonbinary people are not on the list at all. There is also no clarification for students who have received a gender change certificate. Should they put their gender assigned at birth or their “new” gender down? In the end I put “Different Identity” since that was the closest option I had to my identity. A good way to counteract this is allowing people to write in their own answers if needed. The survey should also allow people to choose multiple identities if they want to do so.

What sex were you assigned at birth, such on an original birth certificate? Options: Female, Male, Non-Binary, Decline to State

Similar problems here to the previous question. It is unclear why nonbinary is on the “sex” category but not the “gender” category. Nonbinary is not a sex identification that exists legally or medically. The options should be female, male, and intersex. In addition, it is important to note that it is completely inappropriate for a graduate school to ask a student which genitalia they have. Questions about sex should only be asked in specific instance where genitalia and assigned sex at birth are relevant.

Do you consider yourself to be… Options: Bisexual, Gay or Lesbian, Heterosexual or straight, Not listed above, Decline to State

Being gay is not necessarily the same thing as being a lesbian. Some people identify as both but not all. In addition, there are also a lot of identities that could have been included to make this survey more inclusive. Some additional examples include: Queer, Pansexual, Asexual, or allowing multiple labels to be chosen. Because queer is seen as an umbrella term, it is also very common in the LGBTQ+ community to identify as multiple labels at once. For example, I would most likely choose “Queer” and “Gay” as my sexuality labels. A good way to counteract this is allowing people to write in their own answers if needed.

Gender Expression: A person’s appearance, style, dress, or mannerisms (such as the way they walk or talk) may affect the way people think of them. On average, how do you think other people at school would describe your appearance, style dress, or mannerisms? Options: Mostly feminine, Somewhat feminine, Equally feminine and masculine, Somewhat masculine, Mostly masculine, Not listed, Decline to state

There is no good reason to have this question in this survey. The only reason you would need to ask this question is if the study was getting data on clothing options. In addition, it is important to point out that this kind of question can be dysphoria inducing for trans and gender nonconforming (GNC) people. Transgender and GNC folks’ gender expression may change based on safety levels and whether or not they are publically out.

Proposed Survey

What is your gender? You can choose multiple options. Woman, Man, Nonbinary or Genderqueer, Different Identity (please specify), Decline to State
What is your sexual orientation? You can choose multiple options. Options: Straight or Heterosexual, Lesbian, Gay, Bisexual, Pansexual, Asexual, Queer, Different Identity (please specify), Prefer not to say

What Changes Were Made

Questions 2 and 4 (on Sex and Gender Expression) were completely cut.

These were unnecessary and potentially offensive questions. Unless needed for a specific purpose, like gender expression data for clothing donation, then there is no need to ask participants about their genitals or their clothing preferences.

Question 1:

Should be changed to “What is your gender?” The options should be changed to “Woman, Man, Nonbinary or Genderqueer, Different Identity, or Decline to State.” If the person chooses a different identity, there should be an option to write in that identity. The participant should have the option to choose more than one identity if they wish.

Question 3:

Should be changed to “What is your sexual orientation?” The options should be changed to “Straight or Heterosexual, Lesbian, Gay, Bisexual, Pansexual, Asexual, Queer, Different Identity, or Decline to State.” If the person chooses a different identity, there should be an option to write in that identity. The participant should have the option to choose more than one identity if they wish.

Why Were Changes Made

These changes make a clear distinction between sex and gender identity. There is no confusion between the two separate categories, and there is a clear indication of what is being asked. Question 2, which asked about participants’ genitalia, has been removed. Question 1 now acknowledges that transwomen are also women, and are not a separate category. In addition, nonbinary and genderqueer identities are now acknowledged and are put together in a proper category as similar terms. Gender expression has also been removed as a question as it had no specific survey purpose.

These changes are also more inclusive of different sexual orientations and acknowledge current terminology. There has also been an added option to write in a different identity or choose multiple identities if needed.

Results

Although those recommendations were given to the Office of the President, who said they would take them into consideration, it is uncertain whether changes were made. The results of the study have not yet been published, and new surveys with modified questions have not been announced. The proposed changes would be much better for future surveys, but there are some additions and changes that could be made to make the survey more widely applicable. As this proposal was put together without specific preparation beforehand, and I have learned more working in DEI (diversity, equity, and inclusion) and data collection, there are a few things I would do differently.

Conclusions

After further experience in diversity and data work, there are some changes and recommendations I would make to create a fully functional survey template. The first would be to create a transgender question, that allows participants to self identify as transgender if they wish to do so. In addition, I would have definitions available for survey users to reference if they need further clarification. Finally, I would include a use guide to provide recommendations for application in a survey. Here is an updated version of my LGBTQ+ friendly survey, with optional info pop ups and use recommendations.