Physiological Computing
Physiological Computing (PC) is a term used to describe any computing system that uses real-time quantitative physiological data as an input stream to control a user interface. The most basic sort of PC is one that records a biosignal and displays it to the user via a screen. Other systems, such as Brain Control Interfaces (BCI), take a stream of physiological data and convert it into input control at the interface level. PC also includes computer systems that simply monitor physiology in order to assess psychological states, which are used to trigger real-time adaptation. For example, if the system detects high blood pressure, it may assume the user is experiencing high frustration and offer emotionally-supportive help cues.
People
Lead
Participants
João CostaPh.D. Student, Novel Game Interaction Technologies and Immersive Game Design 
Rina WehbePh.D. Student, Errors, Perception of AI, Health Games
Publications
Year 2014
João Costa,
James Robb, and
Lennart E. Nacke. 2014. Physiological Acrophobia Evaluation Through In Vivo Exposure in a VR CAVE. In
Proceedings of IEEE GEM 2014. Toronto, ON, Canada. IEEE, 1-4.
doi:10.1109/GEM.2014.7047969 @inproceedings{costa2014physiological,
Abstract = {Acrophobia (i.e., the fear of heights) is commonly treated using Virtual Reality (VR) applications. Patients that suffer from this clinical condition can experience extreme levels of anxiety, stress, and discomfort, even at relatively low heights. VR computer-assisted virtual environments (CAVEs) have been found to be highly immersive and successful in the treatment of acrophobia. The general method of evaluating therapy progress is through self-reported questionnaire measures. However, these are subject to participant bias. Physiological measures, on the other hand, could provide a more objective way of assessing acrophobia. To our knowledge, psychophysiological measures are not commonly used in the evaluation of acrophobes and their therapy progress within CAVEs. Thus, we present a CAVE application for acrophobia treatment, which includes a physiological feedback mechanism to assess patient progress. It also permits patient movement to facilitate increased presence and immersion. In this application, players sequentially gain access to increasing heights as they successfully progress through lesser heights, as assessed through the evaluation of their physiological responses to VR stimuli.},
Address = {Toronto, ON, Canada},
Author = {J. P. Costa, J. Robb, and L. E. Nacke},
Booktitle = {Proceedings of IEEE GEM 2014},
Doi = {10.1109/GEM.2014.7047969},
Img = {http://hcigames.com/wp-content/uploads/2015/05/Physiological-acrophobia-evaluation-through-in-vivo-exposure-in-a-VR-CAVE.png},
Keywords = {Acrophobia,Biofeedback,Buildings,CAVE,Electroencephalography,Games,In vivo,Medical treatment,Physiological Measures,Physiology,Virtual Reality,Virtual reality},
MendeleyTags = {Buildings,Electroencephalography,Games,In vivo,Medical treatment,Physiology,Virtual reality},
Pages = {1-4},
Publisher = {IEEE},
Title = {Physiological Acrophobia Evaluation Through In Vivo Exposure in a VR CAVE},
Url = {https://hcigames.com/download/physiological-acrophobia-evaluation-through-in-vivo-exposure-in-a-vr-cave},
Year = {2014},Acrophobia (i.e., the fear of heights) is commonly treated using Virtual Reality (VR) applications. Patients that suffer from this clinical condition can experience extreme levels of anxiety, stress, and discomfort, even at relatively low heights. VR computer-assisted virtual environments (CAVEs) have been found to be highly immersive and successful in the treatment of acrophobia. The general method of evaluating therapy progress is through self-reported questionnaire measures. However, these are subject to participant bias. Physiological measures, on the other hand, could provide a more objective way of assessing acrophobia. To our knowledge, psychophysiological measures are not commonly used in the evaluation of acrophobes and their therapy progress within CAVEs. Thus, we present a CAVE application for acrophobia treatment, which includes a physiological feedback mechanism to assess patient progress. It also permits patient movement to facilitate increased presence and immersion. In this application, players sequentially gain access to increasing heights as they successfully progress through lesser heights, as assessed through the evaluation of their physiological responses to VR stimuli.
Year 2011
Lennart E. Nacke, Michael Kalyn, Calvin Lough, and Regan L. Mandryk. 2011. Biofeedback Game Design: Using Direct and Indirect Physiological Control to Enhance Game Interaction. In
Proceedings of CHI 2011. Vancouver, BC, Canada. ACM, 103-112.
doi:10.1145/1978942.1978958 @inproceedings{Nacke2011a,
Abstract = {Prior work on physiological game interaction has focused on dynamically adapting games using physiological sensors. In this paper, we propose a classification of direct and indirect physiological sensor input to augment traditional game control. To find out which sensors work best for which game mechanics, we conducted a mixed-methods study using different sensor mappings. Our results show participants have a preference for direct physiological control in games. This has two major design implications for physiologically controlled games: (1) Direct physiological sensors should be mapped intuitively to reflect an action in the virtual world; (2) Indirect physiological input is best used as a dramatic device in games to influence features altering the game world.},
Address = {Vancouver, BC, Canada},
Author = {L. E. Nacke, M. Kalyn, C. Lough, and R. L. Mandryk},
Booktitle = {Proceedings of chi 2011},
Doi = {10.1145/1978942.1978958},
File = {::},
Img = {http://hcigames.com/wp-content/uploads/2015/02/Biofeedback-Game-Design-Using-Direct-and-Indirect-Physiological-Control-to-Enhance-Game-Interaction.png},
Keywords = {affective computing,affective gaming,biofeedback,entertainment,games,physiological input,psychophysiology},
MendeleyTags = {affective computing,affective gaming,biofeedback,entertainment,games,physiological input,psychophysiology},
Pages = {103-112},
Publisher = {ACM},
Title = {Biofeedback Game Design: Using Direct and Indirect Physiological Control to Enhance Game Interaction},
Url = {https://hcigames.com/download/biofeedback-game-design-using-direct-and-indirect-physiological-control-to-enhance-game-interaction},
Year = {2011},
BdskUrl1 = {https://hcigames.com/wp-content/uploads/2015/01/Biofeedback-Game-Design-Using-Direct-and-Indirect-Physiological-Control-to-Enhance-Game-Interaction.pdf},
BdskUrl2 = {http://dx.doi.org/10.1145/1978942.1978958}}Prior work on physiological game interaction has focused on dynamically adapting games using physiological sensors. In this paper, we propose a classification of direct and indirect physiological sensor input to augment traditional game control. To find out which sensors work best for which game mechanics, we conducted a mixed-methods study using different sensor mappings. Our results show participants have a preference for direct physiological control in games. This has two major design implications for physiologically controlled games: (1) Direct physiological sensors should be mapped intuitively to reflect an action in the virtual world; (2) Indirect physiological input is best used as a dramatic device in games to influence features altering the game world.
Stephen H. Fairclough, Kiel Gilleade,
Lennart E. Nacke, and Regan L. Mandryk. 2011. Brain and Body Interfaces: Designing for Meaningful Interaction. In
Proceedings of CHI EA 2011. Vancouver, BC, Canada. ACM, 65-68.
doi:10.1145/1979742.1979591@inproceedings{fairclough2011brain,
Abstract = {The brain and body provide a wealth of information about the physiological, cognitive and emotional state of the user. There is increased opportunity to use these data in computerised systems as forms of input control. As entry level physiological sensors become more widespread, physiological interfaces are liable to become more pervasive in our society (e.g., through mobile phones). While these signals offer new and exciting mechanisms for the control of interactive systems, the issue of whether these physiological interfaces are appropriate for application and offer the user a meaningful level of interaction remains relatively unexplored. This workshop sets out to bring together researchers working in the field of psychophysiological interaction to discuss the issue of how to design physiological interactions that are meaningful for users.},
Address = {Vancouver, BC, Canada},
Author = {S. H. Fairclough, K. Gilleade, L. E. Nacke, and R. L. Mandryk},
Booktitle = {Proceedings of chi ea 2011},
Doi = {10.1145/1979742.1979591},
Img = {http://hcigames.com/wp-content/uploads/2015/05/Brain-and-Body-Interfaces-Designing-for-Meaningful-Interaction.png},
Organization = {ACM},
Pages = {65-68},
Publisher = {ACM},
Title = {Brain and Body Interfaces: Designing for Meaningful Interaction},
Url = {https://hcigames.com/download/brain-and-body-interfaces-designing-for-meaningful-interaction},
Year = {2011},
BdskUrl1 = {https://hcigames.com/wp-content/uploads/2015/01/Brain-and-Body-Interfaces-Designing-for-Meaningful-Interaction.pdf},
BdskUrl2 = {http://dx.doi.org/10.1145/1979742.1979591}}The brain and body provide a wealth of information about the physiological, cognitive and emotional state of the user. There is increased opportunity to use these data in computerised systems as forms of input control. As entry level physiological sensors become more widespread, physiological interfaces are liable to become more pervasive in our society (e.g., through mobile phones). While these signals offer new and exciting mechanisms for the control of interactive systems, the issue of whether these physiological interfaces are appropriate for application and offer the user a meaningful level of interaction remains relatively unexplored. This workshop sets out to bring together researchers working in the field of psychophysiological interaction to discuss the issue of how to design physiological interactions that are meaningful for users.
Lennart E. Nacke. 2011. Directions in Physiological Game Evaluation and Interaction. In
Proceedings of the BBI Workshop at CHI 2011. Vancouver, BC, Canada. .
@inproceedings{nacke2011directions,
Abstract = {Physiological sensors are becoming cheaper and more available to game players. This has led to their increased usage in game research and the game industry, where applications range from biofeedback games to design evaluation tools supporting game user researchers in creating more engaging gameplay experiences. This paper gives a brief overview of these current directions of game industry and research threads.},
Address = {Vancouver, BC, Canada},
Author = {L. E. Nacke},
Booktitle = {Proceedings of the bbi workshop at chi 2011},
Img = {http://hcigames.com/wp-content/uploads/2015/02/Directions-in-Physiological-Game-Evaluation-and-Interaction.png},
Title = {Directions in Physiological Game Evaluation and Interaction},
Url = {https://hcigames.com/download/directions-in-physiological-game-evaluation-and-interaction},
Year = {2011},
BdskUrl1 = {https://hcigames.com/wp-content/uploads/2015/01/Directions-in-Physiological-Game-Evaluation-and-Interaction.pdf}}Physiological sensors are becoming cheaper and more available to game players. This has led to their increased usage in game research and the game industry, where applications range from biofeedback games to design evaluation tools supporting game user researchers in creating more engaging gameplay experiences. This paper gives a brief overview of these current directions of game industry and research threads.
Year 2010
Kai Kuikkaniemi, Marko Trupeinen, Hannu Korhonen, Niklas Ravaja, Guillaume Chanel, and
Lennart E. Nacke. 2010. BioS-Play: Workshop on Multiuser and Social Biosignal Adaptive Games and Playful Applications. In
Proceedings of a Workshop at Fun and Games 2010. Leuven, Belgium. ACM.
@article{kuikkaniemi2010bios,
Abstract = {BioS-Play is a workshop targeted to explore the focused domain of biosignal adaptive games and playful application in a multiuser or social context. Using biosignal instrumentation is an established process in medical domains and experimental psychology. During recent years there have been many efforts in industry and research to develop applications, games and various kinds of interfaces which use biosignal analysis in real time. However, most of these applications are single user setups. According to our earlier work we have realized that there is great potential in developing biosignal adaptive applications for multiuser and social scenarios. The applications we have been developing have been either games or playful applications. Games have unique characteristics, which work well for biosignal augmentation. In addition, games can be used in a laboratory setup for exploring the social biosignal interaction design elements overall in a structured and efficient way. Ultimately, we believe that combination of biosignal adaptation, social and multiuser context, and playful interaction is useful for various kinds of situations and can elicit never before- seen experiences. The workshop organizers have several years of experience in researching games, psychophysiology, multiuser application development, and combination of them all. The workshop will be organized with a help of the Presemo presentation environment, which is also a biosignal, adaptive, social, and playful system. The workshop aims at creating new collaborations by facilitating networking of interested researchers and discussing research future research ventures in this domain, distributing knowledge among participants and developing a roadmap related to the future development in this field. Workshop participants are expected to show interest in developing biosignal prototypes or running experiments with experimental psychophysiological systems. In addition to game researchers and individuals with experience in analyzing and working with biosignals, we hope that also people with experience or strong interest in multiuser applications and social interaction will join the session.},
Address = {Leuven, Belgium},
Author = {K. Kuikkaniemi, M. Turpeinen, H. J. Korhonen, N. Ravaja, G. Chanel, and L. E. Nacke},
Journal = {Proceedings of a workshop at fun and games 2010},
Publisher = {ACM},
Title = {BioS-Play: Workshop on Multiuser and Social Biosignal Adaptive Games and Playful Applications},
Url = {https://hcigames.com/download/bioS-play-workshop-on-multiuser-and-social-biosignal-adaptive-games-and-playful-applications},
Year = {2010},
BdskUrl1 = {https://hcigames.com/wp-content/uploads/2015/01/BioS-Play-Workshop-on-Multiuser-and-Social-Biosignal-Adaptive-Games-and-Playful-Applications.pdf}}BioS-Play is a workshop targeted to explore the focused domain of biosignal adaptive games and playful application in a multiuser or social context. Using biosignal instrumentation is an established process in medical domains and experimental psychology. During recent years there have been many efforts in industry and research to develop applications, games and various kinds of interfaces which use biosignal analysis in real time. However, most of these applications are single user setups. According to our earlier work we have realized that there is great potential in developing biosignal adaptive applications for multiuser and social scenarios. The applications we have been developing have been either games or playful applications. Games have unique characteristics, which work well for biosignal augmentation. In addition, games can be used in a laboratory setup for exploring the social biosignal interaction design elements overall in a structured and efficient way. Ultimately, we believe that combination of biosignal adaptation, social and multiuser context, and playful interaction is useful for various kinds of situations and can elicit never before- seen experiences. The workshop organizers have several years of experience in researching games, psychophysiology, multiuser application development, and combination of them all. The workshop will be organized with a help of the Presemo presentation environment, which is also a biosignal, adaptive, social, and playful system. The workshop aims at creating new collaborations by facilitating networking of interested researchers and discussing research future research ventures in this domain, distributing knowledge among participants and developing a roadmap related to the future development in this field. Workshop participants are expected to show interest in developing biosignal prototypes or running experiments with experimental psychophysiological systems. In addition to game researchers and individuals with experience in analyzing and working with biosignals, we hope that also people with experience or strong interest in multiuser applications and social interaction will join the session.
Audrey Girouard, Erin T. Solovey, Regan L. Mandryk, Desney Tan,
Lennart E. Nacke, and Robert J.K. Jacob. 2010. Brain, Body and Bytes: Psychophysiological User Interaction. In
Proceedings of CHI EA 2010. Atlanta, GA, United States. ACM, 4433-4436.
doi:10.1145/1753846.1754167@inproceedings{GSMTNJ_CHI2010,
Abstract = {The human brain and body are prolific signal generators. Recent technologies and computing techniques allow us to measure, process and interpret these signals. We can now infer such things as cognitive and emotional states to create adaptive interactive systems and to gain an understanding of user experience. This workshop brings together researchers from the formerly separated communities of physiological computing (PC), and brain-computer interfaces (BCI) to discuss psychophysiological computing. We set out to identify key research challenges, potential global synergies, and emerging technological contributions.},
Address = {Atlanta, GA, United States},
Author = {A. Girouard, E. T. Solovey, R. L. Mandryk, D. Tan, L. E. Nacke, and R. J. K. Jacob},
Booktitle = {Proceedings of CHI EA 2010},
Doi = {10.1145/1753846.1754167},
File = {::},
Isbn = {978-1-60558-930-5},
Keywords = {affective computing,body,brain,brain-computer interfaces,bytes,hci,interaction,physiological computing,psychophysiological signals,ui},
MendeleyTags = {affective computing,body,brain,bytes,hci,interaction,physiological computing,ui},
Pages = {4433-4436},
Publisher = {ACM},
Title = {Brain, Body and Bytes: Psychophysiological User Interaction},
Type = {Conference proceedings (article)},
Url = {https://hcigames.com/download/brain-body-and-bytes-psychophysiological-user-interaction},
Year = {2010},
BdskUrl1 = {https://hcigames.com/wp-content/uploads/2015/01/Brain-Body-and-Bytes-Psychophysiological-User-Interaction.pdf},
BdskUrl2 = {http://dx.doi.org/10.1145/1753846.1754167}}The human brain and body are prolific signal generators. Recent technologies and computing techniques allow us to measure, process and interpret these signals. We can now infer such things as cognitive and emotional states to create adaptive interactive systems and to gain an understanding of user experience. This workshop brings together researchers from the formerly separated communities of physiological computing (PC), and brain-computer interfaces (BCI) to discuss psychophysiological computing. We set out to identify key research challenges, potential global synergies, and emerging technological contributions.
