Personalizing Exposure Therapy via Reinforcement Learning

Personalized therapy, in which a therapeutic practice is adapted to an individual patient, can lead to improved health outcomes. Typically, this is accomplished by relying on a therapist's training and intuition along with feedback from a patient. However, this requires the therapist to become an expert on any technological components, such as in the case of Virtual Reality Exposure Therapy (VRET). While there exist approaches to automatically adapt therapeutic content to a patient, they generally rely on hand-authored, pre-defined rules, which may not generalize to all individuals. In this paper, we propose an approach to automatically adapt therapeutic content to patients based on physiological measures. We implement our approach in the context of virtual reality arachnophobia exposure therapy, and rely on experience-driven procedural content generation via reinforcement learning (EDPCGRL) to generate virtual spiders to match an individual patient. Through a human subject study, we demonstrate that our system significantly outperforms a more common rules-based method, highlighting its potential for enhancing personalized therapeutic interventions.

Label-Free Subjective Player Experience Modelling via Let's Play Videos

Player Experience Modelling (PEM) is the study of AI techniques applied to modelling a player's experience within a video game. PEM development can be labour-intensive, requiring expert hand-authoring or specialized data collection. In this work, we propose a novel PEM development approach, approximating player experience from gameplay video. We evaluate this approach predicting affect in the game Angry Birds via a human subject study. We validate that our PEM can strongly correlate with self-reported and sensor measures of affect, demonstrating the potential of this approach.

Spiders Based on Anxiety: How Reinforcement Learning Can Deliver Desired User Experience in Virtual Reality Personalized Arachnophobia Treatment

The need to generate a spider to provoke a desired anxiety response arises in the context of personalized virtual reality exposure therapy (VRET), a treatment approach for arachnophobia. This treatment involves patients observing virtual spiders in order to become desensitized and decrease their phobia, which requires that the spiders elicit specific anxiety responses. However, VRET approaches tend to require therapists to hand-select the appropriate spider for each patient, which is a time-consuming process and takes significant technical knowledge and patient insight. While automated methods exist, they tend to employ rules-based approaches with minimal ability to adapt to specific users. To address these challenges, we present a framework for VRET utilizing procedural content generation (PCG) and reinforcement learning (RL), which automatically adapts a spider to elicit a desired anxiety response. We demonstrate the superior performance of this system compared to a more common rules-based VRET method.

Stress Detection from Photoplethysmography in a Virtual Reality Environment

Personalized virtual reality exposure therapy is a therapeutic practice that can adapt to an individual patient, leading to better health outcomes. Measuring a patient's mental state to adjust the therapy is a critical but difficult task. Most published studies use subjective methods to estimate a patient's mental state, which can be inaccurate. This article proposes a virtual reality exposure therapy (VRET) platform capable of assessing a patient's mental state using non-intrusive and widely available physiological signals such as photoplethysmography (PPG). In a case study, we evaluate how PPG signals can be used to detect two binary classifications: peaceful and stressful states. Sixteen healthy subjects were exposed to the two VR environments (relaxed and stressful). Using LOSO cross-validation, our best classification model could predict the two states with a 70.6% accuracy which outperforms many more complex approaches.

Arachnophobia Exposure Therapy using Experience-driven Procedural Content Generation via Reinforcement Learning (EDPCGRL)

Personalized therapy, in which a therapeutic practice is adapted to an individual patient, leads to better health outcomes. Typically, this is accomplished by relying on a therapist's training and intuition along with feedback from a patient. While there exist approaches to automatically adapt therapeutic content to a patient, they rely on hand-authored, pre-defined rules, which may not generalize to all individuals. In this paper, we propose an approach to automatically adapt therapeutic content to patients based on physiological measures. We implement our approach in the context of arachnophobia exposure therapy, and rely on experience-driven procedural content generation via reinforcement learning (EDPCGRL) to generate virtual spiders to match an individual patient. In this initial implementation, and due to the ongoing pandemic, we make use of virtual or artificial humans implemented based on prior arachnophobia psychology research. Our EDPCGRL method is able to more quickly adapt to these virtual humans with high accuracy in comparison to existing, search-based EDPCG approaches.