Researchers

Pedro Nogueira

External Ph.D. Student at University of Porto, Portugal

Projects

Affective Evaluation of Games & Interactive Media

Developing and Analysing Adaptive, Enjoyable, and Engaging Human-Computer Interfaces

Publications

Year 2016

Games of the Heart and Mind: Affective Ludology and the Development of Emotionally Aware Player Experiences.

@book{perron2016video,
  title={Video games and the mind: essays on cognition, affect and emotion},
  author={Perron, Bernard and Schr{\"o}ter, Felix},
  year={2016},
  publisher={McFarland}
}

Year 2015

Modelling human emotion in interactive environments: Physiological ensemble and grounded approaches for synthetic agents

@inproceedings{nogueira2015modelling,
  title={Modelling human emotion in interactive environments: Physiological ensemble and grounded approaches for synthetic agents},
  author={Nogueira, Pedro A and Rodrigues, Rui and Oliveira, Eug{\'e}nio and Nacke, Lennart E},
  booktitle={Web Intelligence},
  volume={13},
  number={3},
  pages={195--214},
  year={2015},
  organization={IOS Press}
}

With the rising research in emotionally believable agents, several advances in agent technology have been made, ranging from interactive virtual agents to emotional mechanism simulations and emotional agent architectures. However, creating an emotionally believable agent capable of emotional thought is still largely out of reach. It has been proposed that being able to accurately model human emotion would allow agents to mimic human behaviour while these models are studied to create more accurate theoretical models. In light of these challenges, we present a general method for human emotional state modelling in interactive environments. The proposed method employs a three-layered classification process to model the arousal and valence (i.e., hedonic) emotional components, based on four selected psychophysiological metrics. Additionally, we also developed a simplified version of our system for use in real-time systems and low-fidelity applications. The modelled emotional states by both approaches compared favourably with a manual approach following the current best practices reported in the literature while also improving on its predictive ability. The obtained results indicate we are able to accurately predict human emotional states, both in offline and online scenarios with varying levels of granularity; thus, providing a transversal method for modelling and reproducing human emotional profiles.

Vanishing scares: biofeedback modulation of affective player experiences in a procedural horror game

@article{nogueira2016vanishing,
  title={Vanishing scares: biofeedback modulation of affective player experiences in a procedural horror game},
  author={Nogueira, Pedro A and Torres, Vasco and Rodrigues, Rui and Oliveira, Eug{\'e}nio and Nacke, Lennart E},
  journal={Journal on Multimodal User Interfaces},
  volume={10},
  number={1},
  pages={31--62},
  year={2016},
  publisher={Springer}
}

To understand the impact of emotionally driven games on player experience, we developed a procedural horror game (Vanish) capable of run-time level, asset, and event generation. Vanish was augmented to interpret players’ physiological data as a simplified emotional state, mapping it to a set of adaptation rules that modify the player experience. To explore the effects of adaptation mechanisms on player experience, we conducted a mixed-methods study on three different versions of the game, two of which integrated varying biofeedback mechanisms. Players’ affective experiences were objectively measured by analysing physiological data. Additionally, subjective experience was recorded through the use of the Game Experience Questionnaire. Our study confirmed that biofeedback functionality had a statistically significant effect on the ratings of player experience dimensions: immersion, tension, positive affect, and negative affect. Furthermore, participants reported noticeable differences in player experience, favouring the added depth present in the biofeedback-enabled iterations of the game. In the future, these conclusions will help to develop more immersive and engaging player experiences.

Year 2014

Fuzzy Affective Player Models: A Physiology-Based Hierarchical Clustering Method

@inproceedings{nogueira2014fuzzy,
Abstract = {Current approaches to game design improvements rely on time-consuming gameplay testing processes, which rely on highly subjective feedback from a target audience. In this paper, we propose a generalizable approach for building predictive models of players' emotional reactions across different games and game genres, as well as other forms of digital stimuli. Our input agnostic approach relies on the following steps: (a) collecting players' physiologically-inferred emotional states during actual gameplay sessions, (b) extrapolating the causal relations between changes in players' emotional states and recorded game events, and (c) building hierarchical cluster models of players' emotional reactions that can later be used to infer individual player models via fuzzy cluster membership vectors. We expect this work to benefit game designers by accelerating the affective play-testing process through the offline simulation of players' reactions to game design adaptations, as well as to contribute towards individually-tailored affective gaming.},
Address = {Raleigh, NC, United States},
Author = {P. A. Nogueira, R. Aguiar, R. A. Rodrigues, E. C. Oliveira, and L. E. Nacke},
Booktitle = {Proceedings of AIIDE 2014},
Img = {http://hcigames.com/wp-content/uploads/2015/02/Fuzzy-Affective-Player-Models-A-Physiology-Based-Hierarchical-Clustering-Method.png},
Pages = {132-138},
Publisher = {AAAI},
Title = {Fuzzy Affective Player Models: A Physiology-Based Hierarchical Clustering Method},
Url = {https://hcigames.com/download/fuzzy-affective-player-models-a-physiology-based-hierarchical-clustering-method},
Year = {2014},

Current approaches to game design improvements rely on time-consuming gameplay testing processes, which rely on highly subjective feedback from a target audience. In this paper, we propose a generalizable approach for building predictive models of players' emotional reactions across different games and game genres, as well as other forms of digital stimuli. Our input agnostic approach relies on the following steps: (a) collecting players' physiologically-inferred emotional states during actual gameplay sessions, (b) extrapolating the causal relations between changes in players' emotional states and recorded game events, and (c) building hierarchical cluster models of players' emotional reactions that can later be used to infer individual player models via fuzzy cluster membership vectors. We expect this work to benefit game designers by accelerating the affective play-testing process through the offline simulation of players' reactions to game design adaptations, as well as to contribute towards individually-tailored affective gaming.

Year 2013

A Hybrid Approach at Emotional State Detection: Merging Theoretical Models of Emotion with Data-Driven Statistical Classifiers

@inproceedings{nogueira2013hybrid,
Abstract = {With the rising popularity of affective computing techniques, there have been several advances in the field of emotion recognition systems. However, despite the several advances in the field, these systems still face scenario adaptability and practical implementation issues. In light of these issues, we developed a nonspecific method for emotional state classification in interactive environments. The proposed method employs a two-layer classification process to detect Arousal and Valence (the emotion's hedonic component), based on four psychophysiological metrics: Skin Conductance, Heart Rate and Electromyography measured at the corrugator supercilii and zygomaticus major muscles. The first classification layer applies multiple regression models to correctly scale the aforementioned metrics across participants and experimental conditions, while also correlating them to the Arousal or Valence dimensions. The second layer then explores several machine learning techniques to merge the regression outputs into one final rating. The obtained results indicate we are able to classify Arousal and Valence independently from participant and experimental conditions with satisfactory accuracy (97\% for Arousal and 91\% for Valence).},
Address = {Atlanta, GA, United States},
Author = {P. A. Nogueira, R. A. Rodrigues, E. Oliveira, and L. E. Nacke},
Booktitle = {Proceedings of WI-IAT 2013},
Doi = {10.1109/WI-IAT.2013.117},
Img = {http://hcigames.com/wp-content/uploads/2015/02/A-Hybrid-Approach-at-Emotional-State-Detection-Merging-Theoretical-Models-of-Emotion-with-Data-Driven-Statistical-Classifiers.png},
Organization = {IEEE},
Pages = {253-260},
Publisher = {IEEE},
Title = {A Hybrid Approach at Emotional State Detection: Merging Theoretical Models of Emotion with Data-Driven Statistical Classifiers},
Url = {https://hcigames.com/download/a-hybrid-approach-at-emotional-state-detection-merging-theoretical-models-of-emotion-with-data-driven-statistical-classifiers},
Volume = {2},
Year = {2013},

With the rising popularity of affective computing techniques, there have been several advances in the field of emotion recognition systems. However, despite the several advances in the field, these systems still face scenario adaptability and practical implementation issues. In light of these issues, we developed a nonspecific method for emotional state classification in interactive environments. The proposed method employs a two-layer classification process to detect Arousal and Valence (the emotion's hedonic component), based on four psychophysiological metrics: Skin Conductance, Heart Rate and Electromyography measured at the corrugator supercilii and zygomaticus major muscles. The first classification layer applies multiple regression models to correctly scale the aforementioned metrics across participants and experimental conditions, while also correlating them to the Arousal or Valence dimensions. The second layer then explores several machine learning techniques to merge the regression outputs into one final rating. The obtained results indicate we are able to classify Arousal and Valence independently from participant and experimental conditions with satisfactory accuracy (97\% for Arousal and 91\% for Valence).

A Regression-Based Method for Lightweight Emotional State Detection in Interactive Environments

@article{nogueira2013regression,
Abstract = {With the popularity increase in affective computing techniques the number of emotion detection and recognition systems has risen considerably. However, despite their steady accuracy improvement, they are yet faced with application domain transferability and practical implementation issues. In this paper, we present a novel methodology for modelling individuals' emotional states in multimedia interactive environments, while addressing the aforemen- tioned transferability and practical implementation issues. Our method relies on a two-layer classification process to classify Arousal and Valence based on four distinct physiological sensor inputs. The first classification layer uses several regression models to normalize each of the sensor inputs across participants and experimental conditions, while also correlating each input to either Arousal or Valence. The second classification layer then employs decision trees to merge the various regression outputs into one optimal Arousal/Valence classification. The presented method not only exhibits convincing accuracy ratings -- 89\% for Arousal and 84\% for Valence - but also presents an adaptable and practical ap- proach at emotional state detection in interactive environment experiences.},
Address = {Angra do Heroísmo, Açores, Portugal},
Author = {P. A. Nogueira, R. A. Rodrigues, E. Oliveira, and L. E. Nacke},
Img = {http://hcigames.com/wp-content/uploads/2015/05/A-Regression-Based-Method-for-Lightweight-Emotional-State-Detection-in-Interactive-Environments.png},
Journal = {Proceedings of epia 2013},
Publisher = {Springer-Verlag Berlin Heidelberg},
Title = {A Regression-Based Method for Lightweight Emotional State Detection in Interactive Environments},
Url = {https://hcigames.com/download/a-regression-based-method-for-lightweight-emotional-state-detection-in-interactive-environments},
Year = {2013},

With the popularity increase in affective computing techniques the number of emotion detection and recognition systems has risen considerably. However, despite their steady accuracy improvement, they are yet faced with application domain transferability and practical implementation issues. In this paper, we present a novel methodology for modelling individuals' emotional states in multimedia interactive environments, while addressing the aforemen- tioned transferability and practical implementation issues. Our method relies on a two-layer classification process to classify Arousal and Valence based on four distinct physiological sensor inputs. The first classification layer uses several regression models to normalize each of the sensor inputs across participants and experimental conditions, while also correlating each input to either Arousal or Valence. The second classification layer then employs decision trees to merge the various regression outputs into one optimal Arousal/Valence classification. The presented method not only exhibits convincing accuracy ratings -- 89\% for Arousal and 84\% for Valence - but also presents an adaptable and practical ap- proach at emotional state detection in interactive environment experiences.

Guided Emotional State Regulation: Understanding and Shaping Players' Affective Experiences in Digital Games

@inproceedings{nogueira2013guided,
Abstract = {Designing adaptive games for individual emotional experi- ences is a tricky task, especially when detecting a player's emotional state in real time requires physiological sensing hardware and signal processing software. There is currently a lack of software that can identify and learn how emotional states in games are triggered. To address this problem, we developed a system capable of understanding the fundamen- tal relations between emotional responses and their eliciting events. We propose time-evolving Affective Reaction Mod- els (ARM), which learn new affective reactions and manage conflicting ones. These models are then meant to provide in- formation on how a set of predetermined game parameters (e.g., enemy and item spawning, music and lighting effects) should be adapted, to modulate the player's emotional state. In this paper, we propose and describe a framework for modulating player emotions and the main components in- volved in regulating players' affective experience. We ex- pect our technique will allow game designers to focus on defining high-level rules for generating gameplay experi- ences instead of having to create and test different content for each player type.},
Address = {Palo Alto, CA, United States},
Author = {P. A. Nogueira, R. A. Rodrigues, E. C. Oliveira, and L. E. Nacke},
Booktitle = {Proceedings of aiide 2009},
Img = {http://hcigames.com/wp-content/uploads/2015/02/Guided-Emotional-State-Regulation-Understanding-and-Shaping-Players-Affective-Experiences-in-Digital-Games.png},
Pages = {51-57},
Title = {Guided Emotional State Regulation: Understanding and Shaping Players' Affective Experiences in Digital Games},
Url = {https://hcigames.com/download/guided-emotional-state-regulation-understanding-and-shaping-players'-affective-experiences-in-digital-games},
Year = {2013},

Designing adaptive games for individual emotional experi- ences is a tricky task, especially when detecting a player's emotional state in real time requires physiological sensing hardware and signal processing software. There is currently a lack of software that can identify and learn how emotional states in games are triggered. To address this problem, we developed a system capable of understanding the fundamen- tal relations between emotional responses and their eliciting events. We propose time-evolving Affective Reaction Mod- els (ARM), which learn new affective reactions and manage conflicting ones. These models are then meant to provide in- formation on how a set of predetermined game parameters (e.g., enemy and item spawning, music and lighting effects) should be adapted, to modulate the player's emotional state. In this paper, we propose and describe a framework for modulating player emotions and the main components in- volved in regulating players' affective experience. We ex- pect our technique will allow game designers to focus on defining high-level rules for generating gameplay experi- ences instead of having to create and test different content for each player type.