Rapid Concept Testing: Solutioning for Young Learners of English in China
THE TASK
ETS (Educational Testing Service) as a company was exploring new areas:
Learning solutions
Young English language learner market
International market
AI (artificial intelligence) technology adoption
THE TEAM
The company’s market research division has identified opportunities for the AI Lab (our team) to support young Chinese learners of the English language. Our team was assembled to work towards solutions for these learners.
Product owner
Project manager
UX Researcher (me)
UX Designer
Learning scientists
Language acquisition researchers
Data scientists & NLP experts (consultants)
My role was to lead the team in UX research tasks, and help them get a taste of the UX process and mindset to identify innovative starting points for learner technology design.
We started with data we had…
The market team provided us with 100+ pages of market reports collected by a local agency that laid out the learner landscape and educational policy. To help the team digest this large amount of information, I collaborated with another learning scientist to put together an informational landscape/learner journey map, which we found to be an effective communication tool to help align the team’s understanding.
The learning scientists and language acquisition experts spent time compiling an extensive literature review of different learner groups, and they presented their secondary findings to our team to help us understand these learners’ developmental characteristics, learning trajectories, and pain points.
Learner characteristics (example)
Culture is test-focused
Parents are anxious about how their children are performing on assessments
Teachers are expected to give daily homework
Learners complain about not understanding what the teacher says
Learner pain points (example)
Parents want more feedback from teachers, esp. regarding test scores
Teachers need more professional development
Learners need adults or teachers present to help regulate their attention
Our next challenge was accessing our target user-learners. Access to these young learners was not established: they resided in another country, and they were minors who needed adult supervision (for research). Market Research also shared that young learners were not “decision-makers” of educational products and tools.
RESEARCH APPROACH
The team was in an explorative state. After brainstorming with stakeholders, we came to the following main questions for the project.
What are parents’ goals, needs, and pain points?
How do parents decide on English-learning products (apps, courses, platforms, etc.) for their kids?
What are some technologies parents are comfortable with their kids using? [criteria for products to trust]
Semi-structured interview with storyboardsTo answer questions 1 & 2, I figured that semi-structured interviews would be the most appropriate because we wanted to be able to follow up with participants (parents) to ask them more about past experiences and thought processes. To answer question 3, I recommended providing visual references for parent participants to provide their feedback upon. The visual reference would come from innovative, technology-immersed ideas. These ideas would be in a high-level, storyboard stage so that the floor was open for participants to share first impressions, instead of specific products in final and high-fidelity stages.
[more reference to this storyboard testing method: Rapidly Exploring Application Design through Speed Dating]
RECRUITMENT
Parents come from different tiered cities (Tier 1 & Tier 2, and Tier 3+) in China
Each parent has at least one child aged 8-12
Exclude parents who are English-teaching professionals
n=10
PROTOCOL
ANALYSIS
INSIGHTS & OUTCOMES
affinity diagramming
Each teammate presented feedback on one storyboard (x16)
Feedback was synthesized through affinity diagramming
Data analysis: session insights summary
By-participant empathy maps (x8)
Persona dimensions: team persona workshop
Personas (x2) with key differentiators
Final deliverables
Persona A
A parent who tutors her son English at home and believes in high-quality teachers are key to child’s English learning gains
Persona B
A parent who hasn’t found in-person classes to be effective and wants to find effective learning tools that provide detailed reports of kid’s performance
Research results - Findings
Main question
What are parents’ goals, needs, and pain points?
Teachers' guidance & supervision
Learner's motivation and engagement
Alignment with school curriculum
Content to be age-appropriate
————————————————————————————-
CHARACTERISTICS (example)
Culture is test-focused
Parents are anxious about how their children are performing on assessments
Teachers are expected to give daily homework
Learners complain about not understanding what the teacher says
PAIN POINTS (example)
Parents want more feedback from teachers, esp. regarding test scores
Teachers need more professional development
Learners need adults or teachers present to help regulate their attention
Brainstorm use cases for AI technology
Data source: “learning tool ideas/concepts” collected from team-wide brainstorming activity
“Listen Together”
“Evening Homework Builder”
“English Magnet”
A systematic approach to developing research stimuli