The company provides content for assessments and accommodations for populations who need it. But what do assessments look like that are built solely for those populations?

1. Understand the research space
2. Two day educator conference
3. Analysis and Lessons Learned

Goals

Create an assessment framework for Math/ELA

Create a set of design principles for ASL assessments & key screens/wireframes

Methodology

My priority was to understand the Deaf and Hard of Hearing population, the way Deaf and Hard of Hearing people communicate, and prior work that has been done in the ASL assessment and learning spaces. Regardless of being a researcher, it is my responsibility to understand and empathize with the people I am working with as much as possible, especially in terms of accessibility and social groups.

1. Understand the research space

Deaf and Hard of Hearing population

• Courses from National Deaf Center
• Information provided by project lead on Deaf populations and communication
• Academic research on population
• YouTube videos of ASL

Current student-centered design principles

• Company documents and research
• Academic research on assessments and student centered design

ASL media

• Games (such as Deafverse)
• Assessments
• Visual language (such as Iconography of ASL)

Using materials provided by the product owner, additional accessibility standards, and a literary research pool, I compiled a loose set of preliminary design principles through card sorting. These activities provided me with a better understanding of the research space, and left me feeling much more prepared for meeting the participants in the next phase in our research, a two day conference with educators and members of the Deaf and Hard of Hearing Community.

The purpose of the conference was to understand how students of the Deaf and Hard of Hearing learn and communicate, with the goal of coming up with two distinct assessment frameworks for ELA and Math, as well as additional information to inform our design and design principles.

2. Two Day Educator Conference

Day 1 - Presentations

• Human/Student Centered Design & Usability
• How ASL students learn in the classroom
• ASL assessments

Day 2 - Break Out Rooms

• Math - Designing assessment task
• ELA - understanding ASL pedagogy

On day one, I gave a presentation about usability, student centered design, and how we apply accessibility to assessments with the intention of aligning those not familiar with usability and accessibility on its definition and how it applies to the discussions we would be having for the rest of the conference.

For the rest of the weekend, I was a sllent observer. I took notes on the presentations, visuals, and discussions held throughout both days, including pictures of presentations and whiteboards and sketches. On day two, we formed breakout rooms for the two subjects, who took vastly different approaches to their time.

For Math, we worked to create a concept sketch of a scenario-based assessment using an existing mathematics framework.

When building scenario-based assessments, items (questions) are centered around a specific topic or situation, such as working in a bakery, or the Brooklyn Bridge, and conclude with a summary of ideas learned throughout the experience. The topics chosen are intended to draw the interest of the student, in order to make the experience more engrossing than an assessment made up of individual items. Since our audience for these assessments were Deaf or Hard of Hearing students, the subject of this scenario based task was Deaf and Hard of Hearing astronomers.

While the subject specialists and conference attendees discussed the details of the content of the task, I focused on how the content would be presented to the student. What components would we need to meet our design standards? What are the differences between our “standard” components and any new components we would have to design? What requirements and caveats need to be noted?

I spent much less time in the ELA break out room, as the conversation focused on the academic framework. ASL and English are two very different and distinct languages and learning patterns, and most of the time was spent discussing ASL and ELA frameworks and how they compare to each other. During my time in the room I was able to identify key design components and requirements.

3. Analysis and Lessons Learned

After the events I completed another round of analysis of notes and information. I used this to check against my original design principles, and made adjustments as needed. The original principles were closely aligned to the conference notes, and I was able to add clarification and new information to the basic ideas.

All in all, I delivered:

• A UX Research and Design project roadmap
• Presentation on Student Centered Design principles
• A set of design principles geared towards assessments for Deaf and Hard of Hearing students
• A comprehensive list of references, resources, and presentations gleaned throughout the project for continuing reference
• A navigatable document containing all of my research notes
• Low-fidelity designs of key screens for assessments that would use ASL for Math and ELA
• Design specifications documentation for standard and new assessment components

Lessons Learned

ASL conference set up

The two day conference meeting room layout was not what I had expected, having a new understanding of accessible spaces for the Deaf and Hard of Hearing. The conference room selected was our executive suite room, so aesthetically, it was the “nicest” room we had to offer. It was, however, very long and thin, which posed two difficulties- being able to see what other people around the room were signing, and movement restrictions for interpreters.

While our participants were able to space themselves around the room in a manner that allowed for full visibility, the paths of movement for interpreters proved to be an issue. Not only did we have to ask each other to pause while interpreters moved around the room to be near the person who was communicating, but the interpreters also were very tired at the end of the first day, contributing to ending our first day’s session a bit early.

Since the second day consisted of breakout rooms, with a smaller amount of participants in each room, we did not face as many difficulties with positioning and movement. Additionally, the main room used was successful in other ways, it was large enough to accomodate for everyone so that even though we had to adjust, we were able to see each other fully. It also had minimal art on the walls and large shades drawn, so there were limited distractions behind signers and interpreters. I do, however, wish the organizers had considered the basic room set up guidelines that had been shared prior to the event a bit more closely.

Abelism / safe spaces

The first thing that came to my mind when I was asked to participate in this study was what right, a hearing person, do I have to be studying something that I will never fully be able to understand or be a part of? I still to this day, do not have an appropriate answer for this question. I understand that any researcher can run into this dilemma when studying any population. However, when it comes to researching marginalized populations and populations with accessibility needs, the question becomes more complicated than when, for example, working with an able bodied group of experienced users to do A/B testing. The questions grow when applying the company or the work sponsor. What right does that company have entering that space? What work has the company done in that space previously? What relationships does it have with those it’s looking to study? And most importantly, what is the intended purpose of this research? Are we at our core, doing this to help those populations?

The sentiments were echoed by our conference participants. This was not the first time that they had been interviewed and invited to discuss the topics at hand. While they appeared eager to share their experiences, knowledge, and insights with us, the framing of able-bodied people working on a project that at that point in time did not include a non-hearing person at all was not overlooked.

While I did my best to educate myself on the population I was working with, to be continuously aware of my position in the room, and how I framed my questions and comments, I can only provide insight into this project from my able-bodied outsider point of view. It is clear that in order to continue this project, the company needs to involve Deaf or Hard of Hearing persons in the entire process, not just as a focus group or a consultant to refer to in stages. The people being represented must be actually represented, not just observed. And when and if the population is observed, it should be done at the guidance of that population itself, not from our assumptions of their view, and with the utmost respect and empathy we can afford.

.