Energizing UX Methods with Subject Matter Experts for Creative Solutions
Designing innovative language learning tools with collaborative storyboarding and rapid concept testing
THE TASK
ETS (Educational Testing Service) as a company was exploring new areas:
Learning solutions
Young English language learner market
International market
AI (artificial intelligence) technology adoption
THE TEAM
The company’s market research division has identified opportunities for the AI Lab (our team) to support young Chinese learners of the English language. Our team was assembled to work towards solutions for these learners.
Product owner
Project manager
UX Researcher (me)
UX Designer
Learning scientists
Language acquisition researchers
Data scientists & NLP experts (consultants)
My role was to lead the team in UX research tasks and help them get a taste of the UX process and mindset to identify innovative starting points for learner technology design.
We started with data we had…
The market team provided us with 100+ pages of market reports collected by a local agency that laid out the learner landscape and educational policy. To help the team digest this large amount of information, I collaborated with another learning scientist to put together an informational landscape/learner journey map, which we found to be an effective communication tool to help align the team’s understanding.
The learning scientists and language acquisition experts spent time compiling an extensive literature review of different learner groups, and they presented their secondary findings to our team to help us understand these learners’ developmental characteristics, learning trajectories, and pain points.
I worked with the whole team to create research stimuli: (1) pain points and (2) 16 storyboards.
RESEARCH APPROACH
Our next challenge was accessing our target user-learners. Access to these young learners was not established: they resided in another country, and they were minors who needed adult supervision (for research). Market Research also shared that young learners were not “decision-makers” of educational products and tools.
The team was in an explorative state. After brainstorming with stakeholders, we came to the following main questions for the project.
What are parents’ goals, needs, and pain points?
How do parents decide on English-learning products (apps, courses, platforms, etc.) for their kids?
What are some technologies parents are comfortable with their kids using? [criteria for products to trust]
Semi-structured interview with storyboards
To answer questions 1 & 2, I figured that semi-structured interviews would be the most appropriate because we wanted to be able to follow up with participants (parents) to ask them more about past experiences and thought processes.
To answer question 3, I recommended providing visual references for parent participants to provide their feedback upon. The visual reference would come from innovative, technology-immersed ideas. These ideas would be in a high-level, storyboard stage so that the floor was open for participants to share first impressions, personal stories and comments (instead of specific products in final and high-fidelity stages, which is mainly used for test of usability).
[more reference to this storyboard testing method: Rapidly Exploring Application Design through Speed Dating]
RECRUITMENT
Parents from different tiered cities (Tier 1 & Tier 2, and Tier 3+) in China
Each parent has at least one child aged 8-12
Exclude parents who are English-teaching professionals
n=10
PROTOCOL
Each session is 90 minutes long.
Part 1: In-depth interview 💬 (40 min)
Researcher material: interview guide
Data collected: participant’s narratives, feelings, concerns...
Part 2: Evaluate “Pain Point Statements” 📐 (20 min)
Researcher material: statements and rating categories
Data collected: ratings with reasons
Part 3: Storyboard concept testing💡 (20 min)
Researcher material: storyboards (with descriptions)
Data collected: questions, descriptions of concepts, and opinions
To maximize interview efficiency, only when a Pain Point Statement is rated a 5 will its correspondent Storyboard(s) be shown.
I presented Pain Point Statements in a Trello board to help participants visualize their categorization while they think-aloud.
SYNTHESIZE DATA: my individual analysis and team engagement
Part 1: In-depth interview 💬
(what I did) Develop by-participant info/empathy maps and share after interview each session
(what I engaged the team to do) Build personas as a team after deciding on “one differentiating factor”
Part 2: Evaluate “Pain Point Statements” 📐
(what I did) Call out high-rated statements and pair them with quotes
(what I engaged the team to do) Discuss shared understanding
Part 3: Storyboard concept testing💡
(what I did) Document and organize reactions and quotes
(what I engaged the team to do) Find themes for product to focus on with affinity diagramming
After each interview session, I’d quickly put together an information map to document knowledge collected from the semi-structured interview part of the session to share with my team.
it gives me an opportunity to both organize notes while it’s fresh, and also tell a story of each participant. I find it a helpful way for the team digest data bit by bit without feeling anxious while waiting for data.
After the whole round of interviews was concluded, I facilitated a session to find differentiating factors among parents through persona development.
This was a challenging exercise for the team. The motivation of this workshop was to motivate the team to form user profiles together and understand differences in parent groups so that we could find opportunities for design
In another team-wide workshop activity, I asked each team member to spend some time summarizing feedback for one storyboard concept and later present to the whole team
Later, I facilitated affinity diagramming to have the team generate parents’ main needs together, which will inform our ideation process next!
IMPACT
It was the first time for the AI Lab to collaborate together and experience the user-centered, UX process together. The team shared with me that the workshops were engaging, inspiring, and actionable. It provided a solid foundation for the content design team to develop content that’d solve these users’ (parents’) problems.
