Mixed Methods: Bose Wellness Lab Study to Understand Sleep and Comfort
CHALLENGE
One might experience difficulties falling asleep due to anxiety from racing thoughts, due to noises in the environment from the streets, neighbors, or a snoring partner, and/or other factors. To pursue sleep and wellness-focused features analogous to the noise-masking function we find in the Bose noise-masking sleepbuds™ (currently discontinued), the Wellness Team conducted research to A/B test a concept to see if it makes a difference. (Due to an NDA, I can say no more about the project beyond this point. However, I am still able to share the skills I learned at a high level. Please continue to read if you are still interested.)
TEAM
The UX research of two consisted of a lead research scientist (specialized in neuroscience and experimental psychology) a user research co-op (me). I was in charge of the study setup (hardware and software prototypes), session note-taking, and assisting quantitative data analyses and report writing. During the study on a daily basis, the UX research team collaborated closely with one electrical engineer and one developer to solve any bugs and make sure that the hardware and software prototypes were functioning as expected.
RESEARCH APPROACH
The methodology includes lab experiments and a structured interview. Participants were divided into 2 groups by the type of software they interact with, each group using only the stimuli programmed (between groups).
Recruitment
All participants (50+) were internal participants who met a list of requirements on their physical conditions.
Protocol
Participants would come into the lab our team set up. They would first follow the instructions to connect themselves to a series of gadgets that monitor their behaviors. Then, they’d perform some tasks, during which the researcher checked in at several points to gather subjective data. Finally, the research lead would conduct a structured interview with them regarding their experience and daily behaviors.
Analysis
Both quantitative and qualitative data were collected,
Quantitative data:
Objective data collected by the gadgets on a participant’s performance of the tasks.
Subjective data self-reported by a participant at multiple points during the study
Qualitative data from final structured interview
For quantitative data, we applied both descriptive statistics and inferential statistics. We used histograms, bar charts, and raw data to lay out the trends of data between two groups; we ran data in SPSS to find any significance (with e.g., Welch-James test for a two-way mixed factorial test for non-normal distributions, Kruskal Wallis test for pairwise comparisons, and Dunn’s test, etc.).
For qualitative data, we used affinity diagramming to categorize data into high-level topics. Having these direct quotes from participants during the interviews, we were able to attach them alongside data graphs to tell a more human-centered story.
PERSONAL REFLECTIONS
The project, taking place in an off-site location, inspired me to practice the stage-management skills that I picked up in school as a theatre minor. Let me share my learnings in theatre and stage-management terms.
First, the team came up with ways to communicate between actors and the behind-the-scenes stage management team. There was a “control room” with researchers and a “stage” where a participant was asked to perform certain tasks; the lead researcher had to travel from one room to the other multiple times during the study while the other one (me) monitored everything and sent signals to the other room when needed — it was crucial to remain invisible while sending communicative signals out while necessary (when something needed to be double-checked, when a question was missed, etc.), so that the team of two (or three, including the participant) will not be overwhelmed by noises and movements.
Secondly, when things didn’t work, we’d rehearse and practice the blocking again. For example, when one cord that connects video recording between rooms keeps blocking the door that’s supposed to close tightly to block out noise when the participant performs certain tasks alone in the room, we re-tape the cord to the point where it happens to travel under the door gap with no problem. In another situation, we also had to call the building manager to stop calling the room for any reason because we couldn’t afford any noise during the study (instead, we asked them to call our cell).
I also built relationships with “the crew,” which included not just the UX team, the product team, but also the building management team. Two months working off-campus in that building for this study, I learned that I could work with the front desk so that they’d text me every time a participant arrived while assuring the participant that they were at the right place; I also worked with the team at the front desk so that they could send microwaves to our room (we had to eat quick lunches in the room every day). One time, they even helped me pump air to a medicine ball because the conference chairs in the building were no longer doing the job of supporting my back.
Last but not least, I learned not to “do it tomorrow,” or even “do it after the next show.” After each study, every participant would generate data on four different programs besides the interview notes. If not saved immediately, data would easily be lost or mixed with another participant’s. I learned to stay focused after taking notes for a long interview and wrap up with data documentation before taking a break. After a full day of user sessions, not leaving things “to tomorrow” is also super important, things including cleaning up camera files for filming space, charging device batteries for sessions on the next day, and cleaning up the space.