A paper on the Multimodal Stress Detection (MSD) project titled “Further Investigating Pen Gesture Features Sensitive to Cognitive Load” has recently been accepted to an IUI 2013 workshop on Interacting with Smart Objects (ISO). The paper is a continuation of the project’s first workshop paper on MSD (presented at the ICMI 2011 MMCogEmS workshop), which examined pen-input features (which we termed “gesture dynamics”) that would be reliable detectors of cognitive load. Along with collaborators at NICTA in Australia, I have been continuing to look at ways to use natural input behaviors to detect changes in cognitive states, specifically, stress. The new paper, first-authored by NICTA student intern Ling Luo, finds several new interesting pen-gesture features to show detectable changes in the presence of cognitive stress. In the long-term, this project may yield useful insights for adaptive accessible systems, and I plan to look at similar research questions for children using such systems. For more information, see the camera-ready version of the paper; or check out the abstract below:
A person’s cognitive state and capacity at a given moment strongly impact decision making and user experience, but are still very difficult to evaluate objectively, unobtrusively, and in real-time. Focusing on smart pen or stylus input, this paper explores features capable of detecting high cognitive load in a practical set-up. A user experiment was conducted in which participants were instructed to perform a vigilance-oriented, continuous attention, visual search task, controlled by handwriting single characters on an interactive tablet. Task difficulty was manipulated through the amount and pace of both target events and distractors being displayed. Statistical analysis results indicate that both the gesture length and width over height ratio decreased significantly during the high load periods of the task. Another feature, the symmetry of the letter ‘m’, shows that participants tend to oversize the second arch under higher mental loads. Such features can be computed very efficiently, so these early results are encouraging towards the possibility of building smart pens or styluses that will be able to assess cognitive load unobtrusively and in real-time.
Lisa Anthony 