NYU Langone Health Projects

Medium: 

Various Projects

Role: 

VR UX Co-ideator/Designer/Engineer, Avatar Designer, Researcher, Pitch Writer/Presenter, & Workshop Facilitator

Tools: 

For: 

NYU Langone Health

Year: 

2021

Collaborator(s): 

Process

{{sect1}}

Overview

Over the course of a summer, I split my time working between NYU Langone Health's Institute for Innovations in Medical Education (IIME) and Healthcare Innovation Bridging Research, Informatics, and Design (HiBRID) Lab as a versatile XR health care consultant.

{{sect2}}

Inspiration

Despite increasing evidence that XR technologies and applications are continuing along an exponential curve of adoption and solution-matching, much of the world / national population is still relatively unaware of the current state of the technology and the range of use cases that have been validated. ​

Training complex physical skills or soft skills, especially remotely (i.e. over Zoom or similar), remains a challenging and cost-heavy task for many organizations. XR training modules offer highly effective, efficient, engaging, and increasingly accessible approaches to training in increasingly complex domains, such as reducing racial bias and teaching therapy modalities.​

{{sect3}}

Approach

I focused on:

1. The research of emerging XR applications in medical services and education

  • Horizon-scanned XR health care landscape
  • Pitched XR landscape and use cases to health care admin teams (CTO, etc.)
  • Led stakeholder roadshow across the hospital system
  • Led XR use case brainstorming workshops & systematic funneling to a priority use case

2. The design of several prototypes for diverse use cases

  • Rapid prototyped a proof of concept for health-related skills training (Bootcamp for New Dads)
  • Automated photogrammetry workflow for anatomy models with Python programming of Tello Drone
  • Created student avatar in Blender for Mozilla Hubs learning modules
  • Prototyped BioDigital Human use cases & radiation therapy immersive educational tool in Looking Glass holographic display

{{sect4}}

Outcome
  • Sparked operational plan to design, develop, & implement a large-scale XR training program
  • Laid the foundations for an XR Community of Practice across a large-scale hospital system
  • Proof-of-concept prototypes for several use cases, from radiation therapy to anatomy education to new family education
  • Organized and presented to senior leadership on key insights into leading industry and research developments in the XR health care sector and beyond

Project Details Below

{{sect5}}

Bootcamp for New Dads

I prototyped a proof of concept for a group VR training for new dads to learn complex physical skills such as holding a baby's head properly and proper swaddling techniques.

Holding Technique Video Demo:

User Flow:

user flow diagram
user flow diagram

Tech Stack:

tech stack diagram
tech stack diagram

{{sect6}}

CDX Presentation

I presented to a group of around 100 clinicians, administrators, and related staff at their weekly Clinician Digital Experience meeting on my research into the potential of XR across departments at NYU Langone Health.

Slides:

{{sect7}}

Use Case Workshop

I led a workshop with 10 team members to brainstorm several dozen use cases (with Miro) and whittle them down to one (Bootcamp for New Dads) to begin prototyping.

{{sect8}}

Mozilla Hubs Student Avatar

I 3D sculpted, rigged, and added a video texture head to a low-poly med student avatar branded with NYU Langone's logo and colors that is optimized to work with Mozilla Hubs. Anatomy students will be able to choose the avatar to represent them in their virtual anatomy classes.

Avatar Demo:

{{sect9}}

Drone Photogrammetry

I wrote some proof-of-concept Python scripts for directing the path of a Tello Drone around plastinated cadaver models. The goal is to increase the efficiency of photogrammetry techniques (which currently require hours of manual labor) via automated 360-degree photo capture.

Drone Flight Path Video Capture Demo:

{{sect10}}

Looking Glass Radiation Therapy Anxiety Reduction Study

I collaborated with Dr. David Byun on early-stage prototyping of a 3D holographic educational tool for his CurieUx study. The study will evaluate the effectiveness of an immersive educational tool built on the Looking Glass Portrait that doctors could use when preparing patients for external beam radiation therapy (EBT). Can prior exposure to intimidating technology (in a realistic and safe virtual encounter) lower patients' anxiety symptoms during in-person therapy appointments?

Menu UI on the Looking Glass
Menu UI on the Looking Glass

3D Viewer of Radiation Therapy Machine on the Looking Glass
3D Viewer of Radiation Therapy Machine on the Looking Glass

Process

{{sect1}}

Overview

Over the course of a summer, I split my time working between NYU Langone Health's Institute for Innovations in Medical Education (IIME) and Healthcare Innovation Bridging Research, Informatics, and Design (HiBRID) Lab as a versatile XR health care consultant.

{{sect2}}

Inspiration

Despite increasing evidence that XR technologies and applications are continuing along an exponential curve of adoption and solution-matching, much of the world / national population is still relatively unaware of the current state of the technology and the range of use cases that have been validated. ​

Training complex physical skills or soft skills, especially remotely (i.e. over Zoom or similar), remains a challenging and cost-heavy task for many organizations. XR training modules offer highly effective, efficient, engaging, and increasingly accessible approaches to training in increasingly complex domains, such as reducing racial bias and teaching therapy modalities.​

{{sect3}}

Approach

I focused on:

1. The research of emerging XR applications in medical services and education

  • Horizon-scanned XR health care landscape
  • Pitched XR landscape and use cases to health care admin teams (CTO, etc.)
  • Led stakeholder roadshow across the hospital system
  • Led XR use case brainstorming workshops & systematic funneling to a priority use case

2. The design of several prototypes for diverse use cases

  • Rapid prototyped a proof of concept for health-related skills training (Bootcamp for New Dads)
  • Automated photogrammetry workflow for anatomy models with Python programming of Tello Drone
  • Created student avatar in Blender for Mozilla Hubs learning modules
  • Prototyped BioDigital Human use cases & radiation therapy immersive educational tool in Looking Glass holographic display

{{sect4}}

Outcome
  • Sparked operational plan to design, develop, & implement a large-scale XR training program
  • Laid the foundations for an XR Community of Practice across a large-scale hospital system
  • Proof-of-concept prototypes for several use cases, from radiation therapy to anatomy education to new family education
  • Organized and presented to senior leadership on key insights into leading industry and research developments in the XR health care sector and beyond

Project Details Below

{{sect5}}

Bootcamp for New Dads

I prototyped a proof of concept for a group VR training for new dads to learn complex physical skills such as holding a baby's head properly and proper swaddling techniques.

Holding Technique Video Demo:

User Flow:

user flow diagram
user flow diagram

Tech Stack:

tech stack diagram
tech stack diagram

{{sect6}}

CDX Presentation

I presented to a group of around 100 clinicians, administrators, and related staff at their weekly Clinician Digital Experience meeting on my research into the potential of XR across departments at NYU Langone Health.

Slides:

{{sect7}}

Use Case Workshop

I led a workshop with 10 team members to brainstorm several dozen use cases (with Miro) and whittle them down to one (Bootcamp for New Dads) to begin prototyping.

{{sect8}}

Mozilla Hubs Student Avatar

I 3D sculpted, rigged, and added a video texture head to a low-poly med student avatar branded with NYU Langone's logo and colors that is optimized to work with Mozilla Hubs. Anatomy students will be able to choose the avatar to represent them in their virtual anatomy classes.

Avatar Demo:

{{sect9}}

Drone Photogrammetry

I wrote some proof-of-concept Python scripts for directing the path of a Tello Drone around plastinated cadaver models. The goal is to increase the efficiency of photogrammetry techniques (which currently require hours of manual labor) via automated 360-degree photo capture.

Drone Flight Path Video Capture Demo:

{{sect10}}

Looking Glass Radiation Therapy Anxiety Reduction Study

I collaborated with Dr. David Byun on early-stage prototyping of a 3D holographic educational tool for his CurieUx study. The study will evaluate the effectiveness of an immersive educational tool built on the Looking Glass Portrait that doctors could use when preparing patients for external beam radiation therapy (EBT). Can prior exposure to intimidating technology (in a realistic and safe virtual encounter) lower patients' anxiety symptoms during in-person therapy appointments?

Menu UI on the Looking Glass
Menu UI on the Looking Glass

3D Viewer of Radiation Therapy Machine on the Looking Glass
3D Viewer of Radiation Therapy Machine on the Looking Glass

Outcome

Other work

Want to create something awesome? Drop me an email.

→ Hi@email.com