Monthly Archives: February 2013

IUI 2013 workshop paper accepted on pen gesture features and cognitive load!

A paper on the Multimodal Stress Detection (MSD) project titled “Further Investigating Pen Gesture Features Sensitive to Cognitive Load” has recently been accepted to an IUI 2013 workshop on Interacting with Smart Objects (ISO). The paper is a continuation of the project’s first workshop paper on MSD (presented at the ICMI 2011 MMCogEmS workshop), which examined pen-input features (which we termed “gesture dynamics”) that would be reliable detectors of cognitive load. Along with collaborators at NICTA in Australia, I have been continuing to look at ways to use natural input behaviors to detect changes in cognitive states, specifically, stress. The new paper, first-authored by NICTA student intern Ling Luo, finds several new interesting pen-gesture features to show detectable changes in the presence of cognitive stress. In the long-term, this project may yield useful insights for adaptive accessible systems, and I plan to look at similar research questions for children using such systems. For more information, see the camera-ready version of the paper; or check out the abstract below:

A person’s cognitive state and capacity at a given moment strongly impact decision making and user experience, but are still very difficult to evaluate objectively, unobtrusively, and in real-time. Focusing on smart pen or stylus input, this paper explores features capable of detecting high cognitive load in a practical set-up. A user experiment was conducted in which participants were instructed to perform a vigilance-oriented, continuous attention, visual search task, controlled by handwriting single characters on an interactive tablet. Task difficulty was manipulated through the amount and pace of both target events and distractors being displayed. Statistical analysis results indicate that both the gesture length and  width over height ratio decreased significantly during the high load periods of the task. Another feature, the symmetry of the letter ‘m’, shows that participants tend to oversize the second arch under higher mental loads. Such features can be computed very efficiently, so these early results are encouraging towards the possibility of building smart pens or styluses that will be able to assess cognitive load unobtrusively and in real-time.

Leave a comment

Filed under Publication

C# implementation of $P recognizer available, online demo in JavaScript!

We have recently made available a reference implementation of our $P recognizer in C#, which you can find on the $P project page. This version augments our original online demo and implementation in JavaScript, both still available as well. When you download the C# .zip file, you receive (1) a DLL of just the recognizer which you can use in your C# applications, (2) a canvas drawing and recognizing demo in C# equivalent to the online JavaScript demo, and (3) a “How To” document explaining how to incorporate these versions in your own projects. Try it out and let us know how it goes!

A reminder: if you implement $P in a new language or in a new way, feel free to let us know and we will link to it from our page as well! Don’t forget to cite us!

Leave a comment

Filed under Software / Data