Accessible and Inclusive UX Research

Mar 11

Let’s talk about conducting design research with participants with disabilities. Where should you start? How should you do it? Is anything different in how to approach a study design with a participant with a disability vs a participant without?

The answer is, yes, some things will be different. And many things will stay the same. I’ve put together this guide after 6 years professionally running UX studies (quantitative and qualitative) with participants with disabilities, and 3 additional years working closely with folks with disabilities on my own personal time. Here is what this short guide will include:

Selecting the right audience, recruitment tips, and compensation.
Building a panel of participants: pros and cons of this approach.
Research method augmentation and tips for various disabilities.
Tech we need to improve these processes.

Let’s dive in

1. Selecting the right audience, recruitment tips, and compensation

I want to ground us in something here: selecting and recruiting people with disabilities to test digital or physical interfaces is NOT a test or selection of people themselves. It is a test of our offer/product/service by people using digital and physical products in a way different from the mainstream.

There are two reasons for this: human and legal. The human reason is that disability occurs differently for everyone; not one person is the same, and we tend to use the medical model rather than the social model when talking about disability.

The legal reason is that we cannot ask a participant to self-identify as having a disability. This is HIPAA-protected information and PII. HIPAA is a U.S. federal law that protects the confidentiality and security of patient health information. Legally, we are not allowed to ask for this type of information when recruiting participants for a study (internally or externally)

PII is personal identifiable information — something we take very seriously in design research and recruitment. We have access to this information, subject to strict rules governing its handling and access. Goes without saying that PII is never to be shared or revealed during or after the study.

Well, Oxana, then how do we recruit participants with disabilities? I’m glad you asked! We recruit them by designing a very carefully constructed screener that does not violate HIPAA or ask for PII. Your screener should ask potential participants about their experience using digital experiences instead.

Ex: “Do you use assistive technology to access websites, mobile applications, or other digital services?” This question does not guarantee that the person answering “yes” has a disability — let’s remember that assistive technology is used by everyone — but ultimately, that’s what you are looking to test: whether anyone using assistive technology can accomplish a task.

Ex: “Do you find yourself losing track of the order of letters or words when reading physical materials or digital interfaces?” Again, the answer “yes” is not a guarantee of a disability, and again , your goal is to test whether anyone experiencing what was described in the question can complete a task successfully.

Remember, we are not testing people themselves. We are testing our offer/product/service’s ability to allow people to accomplish a task.

Once you have your potential participants, you will want to further pre-screen them for participation. If you are recruiting internally, you’re in luck and can get relevant information for your study from your data analytics friends or marketing research friends who can “scrub” your potential participant list to narrow down your criteria. If you are recruiting externally, you can pre-screen them over a phone or video call.

Next, your participants will need to sign consent forms and agree to research activities. If you’re using DocuSign, you have to go in and make sure the form is accessible. It is not accessible from the get-go.

For participants who need a guardian or trusted adult present with them during the study, your consent form needs to reflect their presence. The form language must articulate that the participant is the one giving consent for the study; additional persons present for the study must also consent to the terms of the study. While they are not participating, they will still be able to view and interact with the materials being tested, often ones that are confidential.

Food for thought: using white text on a white background to weed out bots from surveys and screeners also rejects people using screen readers. Screen reader users will relay the question to the person, and they will answer it, making it seem that a bot answered it.

For compensation, always compensate your participants. Remember that their compensation rate should reflect transportation costs or having a guide or trusted person with them. Compensation guide:

–Remote 30 min session: $50+

–Remote 60 min session: $75+

–In-person 30-minute session: $60+

–In-person 60-minute session: $85+

–Workshop 90–120 min: $120+

–Survey compensation: $5-$20

Working with third-party companies to source, recruit, and compensate

If you can’t run recruitment yourself and have a budget to work with vendors, here are some excellent options for you:

Carroll Center for the Blind has a job board (it’s free!) for blind individuals trained to use their screen reader for accessibility testing: they are SRUTT graduates. The center can help you design a usability study, an in-person study, or a testing session.

Knowbility is an organization that can source, recruit, and compensate participants of any and all disabilities for your research. They also have an annual summit and numerous educational resources.

Fable is a company that has its own panel of participants and an Accessible Usability Scale designed by them to measure systems’ accessibility health. It works just like the System Usability Scale, but with an accessibility twist.

Userlytics is another excellent resource with an in-house panel, a myriad of research methods they can help with, and a podcast to boot for additional knowledge and resources.

Another option is to create your own consent forms, recruit from local organizations, social media communities, or job boards, and compensate yourself. Same guidelines apply. I’ve done all 3 methods, and each one has a time and a place (given your budget and time)

2. Building a panel of participants: pros and cons of this approach

Two biggest factors in this decision are budget and research scope. There are tax implications for a person participating in a paid research activity over the course of a year. Usually, if their annual compensation for research participation exceeds a certain amount ($600 in some cases), it becomes “income”. They must report it and file taxes, and so does the entity paying them.

If you build a panel of participants, you can rotate them during your studies and not exceed the annual compensation of $600 for any one person. This works well if you have a large research scope, multiple research activities planned, and multiple phases of research needed. Panels are an excellent way of getting a continuous flow of feedback that is industry or topic-specific.

If you don’t have a continuous need for feedback at this time, building and maintaining a panel may not be the way to go. It may be best to recruit a fresh population each time you need to set up a study.

That’s one con to consider about panels as well: panelists becoming “experts” in the topic or industry. Or becoming too familiar with research, leading to them unintentionally giving industry-coded or topic-coded feedback rather than their personal, candid opinion.

Should you choose to build and maintain a panel, you can do it yourself or work with a third-party vendor. Third-party vendors listed in the section above are excellent choices. They will come at a cost, though, so be prepared for that. In return, they will recruit, screen, schedule, compensate, and rotate your participants for you. Some will help write your study plan and discussion guide. Additionally, they often have access to a larger pool of candidates. It is also a great way to get to know industry leaders in accessibility testing and participants in various parts of the country or even the globe.

Building and maintaining a panel yourself is more involved, but it is also very rewarding. The upside is getting to know your community and having a personal touch with your participants. You are their point of contact every step of the way, which gives you an opportunity to get to know them and make the experience fun and personal for them, too.

3. Research method augmentation and tips for various disabilities

The best research is in-person — you get to see your participants in the context of their innovative ways of accomplishing tasks. It is also a way to ensure that all the assistive technology works as the participant sets it up to work individually for them.

Remote studies work too, but have a few considerations. Let’s go through by the type of participant and their needs.

Participants using screen readers

Every person using a screen reader has their preference for settings and functions. Note key combinations they are using if a keyboard is present, and their navigation preferences when dealing with an app or phone. iOS and Android allow users to navigate by links, headings, etc.

If the session is on-site in a testing room, make sure to ask your participant which keys to have a tactile mark on. They will not be bringing their own keyboards, and most office keyboards have smooth keys. Having a tactile marker on commonly used keys will make it easier for your participant to use the new keyboard.

For in-person sessions, factor in an additional 20–30 minutes to your usual study time. This will account for tech setup and the fact that your participant is listening to a screen reader and then taking action. They may need to set the speed of the screen reader to what they are used to.

When they get stuck, note the friction point and then explain what is causing it once you have the pain point noted. You can let them know that when they try to go from point A to point B, they’re actually navigated to point C, and that is what’s causing the pain point. This lets them know that they are not being tested, the interface is, and they might have a tip on how they usually navigate this challenge.

For remote sessions, you cannot screen share. Zoom, Teams, Google Meet; they all have the same issue: a person using a screen reader will not be able to “read” what is on your shared screen. They cannot interact with or assess the experience that you are sharing. Instead, try to use publicly accessible or internally accessible links.

Most importantly, screen readers “read” interfaces. They can only interact with coded experiences. Preferably, ones that are in production or otherwise publicly accessible. Figma is working on a way to “read” design prototypes, which a screen reader simply views as a picture, but they can be a bit tricky.

Participants using refreshable Braille Display

Let’s say your participant is DeafBlind; make sure you accommodate everyone involved in the session.

Your participant will need an ASL interpreter. Depending on the language spoken by your participant, you will need an interpreter able to speak their silent language version. For studies over 30–45 minutes, your participant will need at least 2 interpreters so that they can take turns. Ask your participant if they will need you to provide them with an interpreter; usually, they will say yes. Then factor in ASL interpreter(s) compensation into your budget.

Account for an additional 30–45 minutes to your regular session time. Refreshable Braille involves several steps of communication, and the interpreters will also need time to communicate to the participant and to you.

Similar to a screen reader, only a coded experience can be tested, and publicly or internally accessible links are best. Screen sharing also does not work.

If the session is remote, make sure that when you record the session, you have multiple cameras. You need to capture the participant and their interpreter(s) as well as the screen.

Ask your participant to turn the sound on when going to the digital experience, that way you will hear what is being tactually communicated back to the participant via Refreshable Braille.

Fun fact: just because it works for a screen reader doesn’t mean it works for a Refreshable Braille display. Braille display needs spaces between paragraphs, for example.

Participants using magnification software

This is when participants use software to zoom in 400–500% Remote, and in-person work is well-suited for these studies. There are no concerns over using a prototype a coded experience or a publicly or internally available website.

The tip I want to share with you is this: if you are running a remote session and you screen share a website, your participant will use their software to zoom in 400–500%. However, what you see on your screen will stay the same. So, have your participants share their screen instead. Especially if you want to record and capture their experience.

Participants with intellectual disabilities and cognitive impairments

Participants with intellectual disabilities, such as Down Syndrome, for example, will have different support needs. Some will need to have a guardian or trusted adult present. In this case, you will need additional language in your consent form to reflect the presence of a second person who is not the participant (refer to my recruitment tips on consent forms for these situations). Some are non-vocal, meaning they cannot communicate verbally and express themselves in other ways.

Your session time will need to be adjusted. Participants with intellectual disabilities may need breaks, or multiple breaks, during a 30-minute session. You can either make your sessions “bite-sized” at around 15 minutes or factor in a break. Anything over 30 minutes can be tricky, even if it’s a fun co-creation activity.

I’ve used different research methods with this population: card sort (in person and remote), survey, interview, and co-creation workshops and activities. Your research materials need to be fun and engaging; color-coding your card sort is acceptable here — it will not influence your participant and instead will keep their attention on the task.

When assessing content, the best approach is to either physically mark the words with highlighters and stickers or do it virtually, depending on your participant’s level of digital affinity. Physical paper, markers, and stickers worked best for me. It also allows you to be fully present with the participant during the task and follow up with questions. Debates are out there on whether digital workshops are the same as in-person ones; I’m here to tell you that, in this particular case, in-person workshop activities are a bit more meaningful.

When a caregiver or trusted adult is present, it’s an excellent opportunity to hear how they explain tasks and communication to the participant. Always address the participant. If the phrasing you’re using is confusing, and you’ve tried to explain it in multiple ways, allow the second person present to help explain your question. Take note of the language, phrases, and metaphors used.

Participants with motor impairment

The only consideration here is to address the participant directly. Depending on the impairment, a caregiver or trusted adult will be present. Your participant may have their speech affected, making them speak slower or differently than you are used to. Don’t be afraid to repeat yourself when asking questions, and to ask them to repeat themselves. Only have their caregiver or trusted adult explain for them when you absolutely cannot understand your participant.

4. Tech we need to improve these processes

For studies with participants with visual impairments

Ways to test Figma/other prototyping tools with a screen reader. Figma is working on a beta version of this.
Ways for screen share to show a zoomed-in version of the website that your participant with low vision is seeing on their end.
Ways to screen share and the screen reader to be able to read the shared page.
Surveys to allow Researchers access to the code to put in alt text for images in multiple-choice.
Ways to weed out bots without weeding out screen reader users as well.

Participants with hearing impairments and who are d/Deaf

Ways for d/Deaf participants to record themselves without needing to make a sound for the system to register that they are recording.
Right now, they have to make a sound to be recorded. This creates an uncomfortable and awkward experience.
Always have an alternative way to answer the question — can’t do a video? That’s fine, write your answer instead in the provided field.

Participants with motor impairments

Accessible venues for research.
If they have to use an elevator, make sure it is big enough to fit them and their companion.
Make sure restrooms in the building are accessible and have an accessible stall.