SENSING PHYSIOLOGICAL AROUSAL AND VISUAL ATTENTION DURING USER INTERACTION

UoM administered thesis: Phd

  • Authors:
  • Oludamilare Matthews

Abstract

Arousal is a psychophysiological state that is characterised by increased attention and alertness. Arousal detection is paramount during user interaction because arousal influences perception, cognition and performance, which all have significant impacts on user experience (UX). Self-reported means of measuring arousal are manual and prone to bias. Behavioural modes of sensing arousal, such as the analysis of voice prosody, keystroke dynamics and body gestures yield inconsistent results when applied in different applications. Physiological sensors for detecting arousal such as electroencephalograms and galvanic skin response are sensitive to confounding factors like motion and temperature. Recent studies have leveraged multimodal arousal detection to improve detection accuracy. However, due to the cost of purchasing additional sensors, skills to set them up and the availability of all the sensors, multimodal arousal detection has limited potential for widespread use. These modes of arousal detection also provide limited visual context about users' measure of arousal. We use eye trackers to collect pupillary response and gaze behaviour data. The analysis of pupillary response is used to sense changes in arousal while gaze detection reveals the visual context, i.e., the user's focal attention during moments of increased arousal. To improve generalisability, our approach was developed and evaluated using multiple eye-tracking datasets containing known causes of arousal. Despite the limitation of our approach (i.e., sensitivity to light changes), results suggest that our approach can be used to sense arousal of several forms (cognitive load, emotional and frustration). Furthermore, our unimodal approach detects users' focal attention during moments of increased arousal. As web cameras with eye-tracking abilities become more accessible, there is increased potential for the widespread use of our technique in the wild. Unobtrusive arousal sensing opens up opportunities for UX researchers, UI designers and software developers in adaptive computing, affective gaming, intelligent tutoring systems, user modelling and recommender systems.

Details

Original languageEnglish
Awarding Institution
Supervisors/Advisors
Award date31 Dec 2019