just-a-test

Medium:

Role:

Tools:

For:

Year:

Collaborator(s):

Process

‍

Approach

For the initial R&D of the tool, we decided to focus on exploring the design potential of XR technology in improving the quality of existing training by enhancing its experiential learning aspects.

‍

Why XR?

Our motivation for choosing XR as the training medium stemmed from team members’ experience working with the medium and familiarity with its key affordances, including an increased sense of realism and user engagement.

‍

We were encouraged by a study [5] that demonstrated that, compared to e-learning and classroom learning, XR-based learning of soft skills unlocked learners’ significantly greater (1) focus on content; (2) emotional connection to content; and (3) confidence in applying skills.

Through a literature review, our research scientist helped to clarify and validate the psychological mechanisms underlying XR experiences and mapped out the key factors of: (1) presence, or the subjective perception of being a part of an experience [5]; (2) embodiment, or the involvement of one’s body in an experience, including the integration of gestures or even full-body movement [6]; (3) transportation, or the cognitive process of becoming focused on an event occurring in a narrative [7]; and (4) identification, or the emotional and cognitive process of taking on the role of a character in a narrative [8].

Given that soft skills consist of more than cognitive skills, including emotional, social, non-verbal, and gestural skills, we hypothesized that these complex psychological factors would synergize well within XR simulations of role-playing experiences, which are a cornerstone of traditional mental health training [10]. The experience of presence supports the perception that things happening around oneself are actually real and connected to oneself [11, 12]. The experience of embodiment dynamizes this presence by extending one’s awareness to include their full physical being, including interactions with their environment, which can enhance learning by increasing retention and deepening intuitive understandings of knowledge [1, 13, 14]. These affordances provide a foundation for transportation, such as in finding oneself transported into an interaction with a virtual patient, as well as identification, which allows one to play the roles of augmented versions of oneself [15], such as simulating one’s emerging identity as a mental health care provider. Taken together, these affordances can scaffold an authentic sense of social presence [16, 17] and support the potential for emotionally engaging, transformative learning [18].

‍

Through our initial scan of XR soft skill trainings in the market and in research, we found that many consist of:

proposals for future explorations of this form of training [19, 20, 21];
restrictive, pre-structured experiences, which rely on multiple choice prompts or preset dialogue options [22, 23, 24];
computer desktop experiences that lack immersive embodiment [25]; or
setups relying on actors for dialogue flow [26].

‍

With developments in conversational AI, applications are emerging that integrate more fluid interactivity [27]; however, we are not aware of XR training tools that allow both an open-ended expressiveness for trainees to practice conversing in tandem with a subtle curation of virtual patients’ embodied, emotional, and realistic responses.

‍

Inspired by (1) limitations in the field; (2) theories of constructivist and authentic learning [28, 29], which posit that learning is an active, exploratory process and benefits greatly from hands-on activities; as well as (3) studies calling for more sophisticated designs of virtual patients [30, 31, 32]; we were excited to explore creating a highly dynamic and realistic role-playing XR experience for trainees.

‍

Especially in the field of mental health training, in which sensitivity to psychological nuance serves as a foundation of the healing process [33, 34], it may be critical that trainees have expansive opportunities to authentically practice the nuance of soft skills in preparation of meeting with patients.

‍

Why Participatory Design?

Also, given that we were designing for trainees coming from and serving underrepresented communities, we acknowledged the role that cultural sensitivity has in shaping mental health outcomes [35, 36, 37] and committed to a participatory design approach to embed that sensitivity into our design process.

‍

Participatory design involves situating stakeholders, especially end-users, as collaborative partners and inviting them to “play an active role in the design process and the ways problems are defined” [38]. It is a means of ensuring that an intervention carefully addresses the needs and need priorities of the community it aims to serve [39, 40].

Our approach to participatory design involved a mix of: (1) interviews and workshops with subject-matter experts (SMEs) that were designing and facilitating the existing training; (2) participating in an existing training webinar; (3) informal user testing; and (4) a preliminary usability study with potential end-users.

‍

Research Questions

With all of these factors in mind, we synthesized the following research questions to guide our R&D process:

What are the needs and challenges pertaining to XR-based training of community-based workers in conducting mental health screenings of community members?
What interaction design strategies can be used to improve the usability of XR-based mental health training?

‍

Specifying the Curriculum

Our deliberations with the SMEs led to a consensus on the selection of screening skills, specifically the skill of administering the brief PHQ-4 questionnaire, which assesses a patient’s depression and anxiety symptoms [45].

‍

Outcome

Process

‍

Approach

For the initial R&D of the tool, we decided to focus on exploring the design potential of XR technology in improving the quality of existing training by enhancing its experiential learning aspects.

‍

Why XR?

Our motivation for choosing XR as the training medium stemmed from team members’ experience working with the medium and familiarity with its key affordances, including an increased sense of realism and user engagement.

‍

We were encouraged by a study [5] that demonstrated that, compared to e-learning and classroom learning, XR-based learning of soft skills unlocked learners’ significantly greater (1) focus on content; (2) emotional connection to content; and (3) confidence in applying skills.

Through a literature review, our research scientist helped to clarify and validate the psychological mechanisms underlying XR experiences and mapped out the key factors of: (1) presence, or the subjective perception of being a part of an experience [5]; (2) embodiment, or the involvement of one’s body in an experience, including the integration of gestures or even full-body movement [6]; (3) transportation, or the cognitive process of becoming focused on an event occurring in a narrative [7]; and (4) identification, or the emotional and cognitive process of taking on the role of a character in a narrative [8].

Given that soft skills consist of more than cognitive skills, including emotional, social, non-verbal, and gestural skills, we hypothesized that these complex psychological factors would synergize well within XR simulations of role-playing experiences, which are a cornerstone of traditional mental health training [10]. The experience of presence supports the perception that things happening around oneself are actually real and connected to oneself [11, 12]. The experience of embodiment dynamizes this presence by extending one’s awareness to include their full physical being, including interactions with their environment, which can enhance learning by increasing retention and deepening intuitive understandings of knowledge [1, 13, 14]. These affordances provide a foundation for transportation, such as in finding oneself transported into an interaction with a virtual patient, as well as identification, which allows one to play the roles of augmented versions of oneself [15], such as simulating one’s emerging identity as a mental health care provider. Taken together, these affordances can scaffold an authentic sense of social presence [16, 17] and support the potential for emotionally engaging, transformative learning [18].

‍

Through our initial scan of XR soft skill trainings in the market and in research, we found that many consist of:

proposals for future explorations of this form of training [19, 20, 21];
restrictive, pre-structured experiences, which rely on multiple choice prompts or preset dialogue options [22, 23, 24];
computer desktop experiences that lack immersive embodiment [25]; or
setups relying on actors for dialogue flow [26].

‍

With developments in conversational AI, applications are emerging that integrate more fluid interactivity [27]; however, we are not aware of XR training tools that allow both an open-ended expressiveness for trainees to practice conversing in tandem with a subtle curation of virtual patients’ embodied, emotional, and realistic responses.

‍

Inspired by (1) limitations in the field; (2) theories of constructivist and authentic learning [28, 29], which posit that learning is an active, exploratory process and benefits greatly from hands-on activities; as well as (3) studies calling for more sophisticated designs of virtual patients [30, 31, 32]; we were excited to explore creating a highly dynamic and realistic role-playing XR experience for trainees.

‍

Especially in the field of mental health training, in which sensitivity to psychological nuance serves as a foundation of the healing process [33, 34], it may be critical that trainees have expansive opportunities to authentically practice the nuance of soft skills in preparation of meeting with patients.

‍

Why Participatory Design?

Also, given that we were designing for trainees coming from and serving underrepresented communities, we acknowledged the role that cultural sensitivity has in shaping mental health outcomes [35, 36, 37] and committed to a participatory design approach to embed that sensitivity into our design process.

‍

Participatory design involves situating stakeholders, especially end-users, as collaborative partners and inviting them to “play an active role in the design process and the ways problems are defined” [38]. It is a means of ensuring that an intervention carefully addresses the needs and need priorities of the community it aims to serve [39, 40].

Our approach to participatory design involved a mix of: (1) interviews and workshops with subject-matter experts (SMEs) that were designing and facilitating the existing training; (2) participating in an existing training webinar; (3) informal user testing; and (4) a preliminary usability study with potential end-users.

‍

Research Questions

With all of these factors in mind, we synthesized the following research questions to guide our R&D process:

What are the needs and challenges pertaining to XR-based training of community-based workers in conducting mental health screenings of community members?
What interaction design strategies can be used to improve the usability of XR-based mental health training?

‍

Specifying the Curriculum

Our deliberations with the SMEs led to a consensus on the selection of screening skills, specifically the skill of administering the brief PHQ-4 questionnaire, which assesses a patient’s depression and anxiety symptoms [45].

‍

Outcome

Other work

Creature Capsule

Film

Sequence Rings

Virtual Reality

Want to create something awesome? Drop me an email.

→ Hi@email.com