Braingage

Medium: 

Magic Leap 1 AR Glasses Experience

Role: 

AR UX Co-ideator/Designer/Engineer & Project Lead

Tools: 

For: 

MIT Reality Hack

Year: 

2022

Collaborator(s): 

Gloria Kang, Joanna Dabrowska, Chenwei Li, Ayanna Seals

Process

{{sect1}}

Overview

My hackathon team and I prototyped an AR glasses-based public speaking anxiety intervention that could be accessed and applied in the moments of greatest need.

braingage logo
braingage logo

{{sect2}}

Inspiration

Traditional therapy treats patients during appointments and may have homework or exercises for patients to try in their daily lives, yet there still remains a significant gap of transferring new skills or perspectives learned through therapy to real-world situations that trigger symptoms.

Potential Audience: Young adults starting first job in their professional career want to feel confident to speak up and share their ideas to engage better with teammates and in a workplace.

{{sect3}}

Approach

Inspired by the framework of Just In Time Adaptive Interventions (JITAI), we wanted to explore how perception-first digital mediatory tools such as mixed reality can offer new ways to seamlessly access powerful tools and techniques on the fly.

We built our experience in Unity with the help of Magic Leap's documentation and their Unity SDKs and plugins. Some of us focused more on the storyboarding and UI/UX sketches, and others focused on Unity implementation, including setting up Magic Leap hand-tracking, hand pose gestures, eye-tracking, world reconstruction, fluid UI menus, spatial audio, and mixed reality anxiety-reducing experiences.

{{sect4}}

Outcome

We prototyped a proof of concept of the menu system (featuring hand-tracking and speech-to-text inputs) and one of the interventions (the 4, 7, 8 breathing technique, which consists of a guided breathing meditation synced with expanding and contracting particle system visuals).

We started prototyping the other two interventions:

  • blue dot, which consists of a location-aware ground exercise that places several blue dots on surfaces around the room (with the help of Magic Leap's world reconstruction) and makes them disappear when the participant has stared at them for a couple of seconds (detected by Magic Leap's eye-tracking)
  • energize me, which consists of an empowering confidence visualization and sound effect in which an energy orb powers up the body

The solution will also be integrated with the EDA sensor device called Bitalino, that measures your stress level through sweat detection. If your stress threshold is reached, the application will start by itself. If the stress threshold decreases, the application will shut down automatically.

{{sect5}}

Demo

Prototype Demo:

Recorded presentation:

{{sect6}}

User Flow

user flow diagram
user flow diagram

UI Storyboards:

onboarding screen
onboarding screen

intervention menu
intervention menu

blue dot intervention ui
blue dot intervention ui

exit screen
exit screen

{{sect7}}

Tech Stack

tech stack diagram
tech stack diagram

{{sect8}}

Slides

slide 1: empowering young adults in professional environments
slide 1: empowering young adults in professional environments

slide 2: problem statement
slide 2: problem statement

slide 3: user scenario
slide 3: user scenario

slide 4: persona
slide 4: persona

slide 5: user journey
slide 5: user journey

slide 6: thank you and contact info
slide 6: thank you and contact info

Process

{{sect1}}

Overview

My hackathon team and I prototyped an AR glasses-based public speaking anxiety intervention that could be accessed and applied in the moments of greatest need.

braingage logo
braingage logo

{{sect2}}

Inspiration

Traditional therapy treats patients during appointments and may have homework or exercises for patients to try in their daily lives, yet there still remains a significant gap of transferring new skills or perspectives learned through therapy to real-world situations that trigger symptoms.

Potential Audience: Young adults starting first job in their professional career want to feel confident to speak up and share their ideas to engage better with teammates and in a workplace.

{{sect3}}

Approach

Inspired by the framework of Just In Time Adaptive Interventions (JITAI), we wanted to explore how perception-first digital mediatory tools such as mixed reality can offer new ways to seamlessly access powerful tools and techniques on the fly.

We built our experience in Unity with the help of Magic Leap's documentation and their Unity SDKs and plugins. Some of us focused more on the storyboarding and UI/UX sketches, and others focused on Unity implementation, including setting up Magic Leap hand-tracking, hand pose gestures, eye-tracking, world reconstruction, fluid UI menus, spatial audio, and mixed reality anxiety-reducing experiences.

{{sect4}}

Outcome

We prototyped a proof of concept of the menu system (featuring hand-tracking and speech-to-text inputs) and one of the interventions (the 4, 7, 8 breathing technique, which consists of a guided breathing meditation synced with expanding and contracting particle system visuals).

We started prototyping the other two interventions:

  • blue dot, which consists of a location-aware ground exercise that places several blue dots on surfaces around the room (with the help of Magic Leap's world reconstruction) and makes them disappear when the participant has stared at them for a couple of seconds (detected by Magic Leap's eye-tracking)
  • energize me, which consists of an empowering confidence visualization and sound effect in which an energy orb powers up the body

The solution will also be integrated with the EDA sensor device called Bitalino, that measures your stress level through sweat detection. If your stress threshold is reached, the application will start by itself. If the stress threshold decreases, the application will shut down automatically.

{{sect5}}

Demo

Prototype Demo:

Recorded presentation:

{{sect6}}

User Flow

user flow diagram
user flow diagram

UI Storyboards:

onboarding screen
onboarding screen

intervention menu
intervention menu

blue dot intervention ui
blue dot intervention ui

exit screen
exit screen

{{sect7}}

Tech Stack

tech stack diagram
tech stack diagram

{{sect8}}

Slides

slide 1: empowering young adults in professional environments
slide 1: empowering young adults in professional environments

slide 2: problem statement
slide 2: problem statement

slide 3: user scenario
slide 3: user scenario

slide 4: persona
slide 4: persona

slide 5: user journey
slide 5: user journey

slide 6: thank you and contact info
slide 6: thank you and contact info

Outcome

Other work

Want to create something awesome? Drop me an email.

→ Hi@email.com