Enhancing language learning with augmented reality 

Project Overview

This project aims to create an immersive, interactive, and situational learning experience for users that is tailored to their environment using augmented reality and image recognition technology. 

Through real-world contexts, users can improve their retention and recall of new words, ultimately accelerating their language learning progress.

My Role

Solo project I did for my senior capstone UX design studio

Tools

• Figma 

• Adobe Aero

Project Duration

5 months (January to May 2023)

Design Methods

• UXR 
• Comparative Analysis
• Testing Sessions
• Academic and secondary research

Challenges/Problem Statement

Design Opportunity Overview

With the widespread availability of smartphones and smart devices, learners can now access advanced technologies that can supplement their learning to yield more efficient and enjoyable experiences.

Design Process

I structured my project in three milestones.

Milestone #1: Integrate

In this milestone I asked myself, how might we integrate AR seamlessly into the Duolingo application? 

Questions I wanted to answer included: 

1. How does AR fit into the Duolingo brand strategy and philosophy? 
2. What are examples of AR used for education? 
3. How is language learning facilitated? 

To answer my questions I conducted: 

1. Secondary research on Duolingo, AR, and language learning
2. Comparative analysis
3. Sketching and digital low-fidelity prototypes

Milestone #2: Empathize

In this milestone I asked myself, how might we uncover what would be most beneficial for a user’s learning with an AR integration?

Questions I wanted to answer included: 

1. What are the key pain points, wants, and needs of current Duolingo users?
2. What do current users think of my proposed design concepts?

To answer my questions I conducted: 

1. User interviews 
2. Concept testing 
3. Design iterations 

Milestone #3: Refine

In this milestone, I asked myself, how might we ensure a user would enjoy and benefit from our design integration? 

Questions I wanted to answer included: 

1. Are my current designs working? 
2. How can I make my designs better?
   

To answer my questions I conducted: 

1. Supplementary secondary research 
2. Iteratations and testing

Milestone 1: Integrate

To start off my project, I wanted to gain a better understanding of what Duolingo’s brand embodied. This involved doing a deep dive into the start of the Duolingo service to understand what philosophies were at its core. 

The Duolingo research I was conducting led me partially into the pedagogy of language learning but I felt like I needed to know more before moving on. This led me to ask the following question:

How is language learning facilitated and what is AR?

To answer this question, I looked to academic research on languauge learning as well as academic research on augmented reality.

Milestone #2: Empathize

Integration now worked out, it was time to start uncovering user thoughts and opinions through user interviews. This led me to begin exploring the questions I had in the second phase of the project. The goal of conducting user interviews was to understand the uses of the app and identify user wants, needs, and expectations for an AR integration. 

 Notable takeaways:

• 5/5 users feel the app is lacking in the conversational aspect
• ⅘ users say they complete their daily lesson for bed.
• ⅗ users want to simulate real-life conversations
• ⅖ users want AR to detect everyday objects in an effort to help build their vocab with their surroundings

After having a better understanding of how current Duolingo user’s feel about the app experience and how they anticipate AR playing a role in the experience, it was time to test my proposed design activities. 

I tested with Duolingo daily users. I asked users for their first impressions of the activities and recorded responses in an Excel sheet.

After iterating my designs based on concept testing results, I wanted to focus on exploring the capabilities of augmented reality and how users can interact with it. I looked to Adobe Aero to help me high-fi my designs. 

Milestone #3: Refine

The goal of this stage was to high-fi my designs with AR and get a better understanding of the physical interactions a user would have with AR. 

This leads me to my final designs. Click through to see all the onboarding screens!

Concept 1: Identifying objects in the user’s environment (Activity 1: selection)

This concept aims to get users familiar with the objects in their environment. By using their real-world surroundings, the user can better recall and recognize objects in the new language. 

In this activity, the user would start by scanning their environment to give image recognition technology the opportunity to recognize objects. After recognition, Duo would appear on the screen and prompt the user to select specific objects. The user would click and select the object asked of them. After selection, the object would highlight to ensure the user selected the object they wanted. Finally, they will check their answer and continue with the selection process until the activity is over. 

Concept 1: Identifying objects in the user’s environment (Activity 2: drag and drop)

In this activity, the user would start by scanning their environment to give image recognition technology the opportunity to recognize objects. After recognition, a word bank will populate at the bottom of the screen. Then, the user would drag and drop words from the word bank onto the corresponding object in the environment. After all words are on their corresponding objects, the user checks their answers and continues.

Concept 2: Story comprehension 

This concept builds off Duolingo’s gamified and fun experience. Users are able to experience a variety of scenarios that play out right in front of them. Scenarios range from a character playing a game, UFOs, concerts, and snakes, the possibilities are endless. 

In this activity, the AR scenario is a girl playing with a ball. After the scene plays out, the user selects the phrase they think is correct. In this case, the options are: 

1. The girl is reading a book. 
2. The girl is playing with the ball. 

After the user selects their response, they will check their answer and continue the process until the lesson is complete. 

Concept #3 Conversation

This activity builds off of user wants. During interviews, it was discovered that users felt the app was lacking in the conversational aspect. Users wanted to be able to better converse, not just learn "silly voacb words and sentences". This concept provides users that opportunity. 

In this activity, the user would be placed in a story with an AR character. In this case, the two are at a restaurant. The AR character would prompt the user to converse by asking questions. The user would answer the questions by forming sentences from the word bank. In the future, I would like to build this activity out to have a speaking portion.

Made with ♡ by Sarahy

𓆝 𓆟 𓆞 𓆝 

LinkedIn