Last week at the ICMI 2012 conference, I presented a new gesture recognizer in the $-family, called $P. $P is highly accurate, needing few training examples or templates, and is able to handle gestures made with any number of strokes in any order or direction, but uses simple concepts that make it accessible to those other than experts in machine learning or pattern matching. Find my presentation slides here.
Tag Archives: multistrokes
Co-authors Radu-Daniel Vatavu and Jacob O. Wobbrock and I have had a paper accepted to ICMI 2012, titled “Gestures as Point Clouds: A $P Recognizer for User Interface Prototypes,” in which we introduce $P, the latest member of the $-family of gesture recognizers. $P can handle multistroke and unistroke gestures alike with high accuracy, and remedies the main limitations of $N in terms of cost to store and match against all possible multistroke permutations.
Here is the abstract:
Rapid prototyping of gesture interaction for emerging touch platforms requires that developers have access to fast, simple, and accurate gesture recognition approaches. The $-family of recognizers ($1, $N) addresses this need, but the current most advanced of these, $N-Protractor, has significant memory and execution costs due to its combinatoric gesture representation approach. We present $P, a new member of the $-family, that remedies this limitation by considering gestures as clouds of points. $P performs similarly to $1 on unistrokes and is superior to $N on multistrokes. Specifically, $P delivers >99% accuracy in user-dependent testing with 5+ training samples per gesture type and stays above 99% for user-independent tests when using data from 10 participants. We provide a pseudocode listing of $P to assist developers in porting it to their specific platform and a “cheat sheet” to aid developers in selecting the best member of the $-family for their specific application needs.
I am pleased to announce that the Mixed Multistroke Gestures (MMG) dataset from our GI 2012 paper is now publicly available for download! It contains samples from 20 people who entered each of 16 gesture types 10 times, using either their finger or a stylus on a Tablet PC, at three different speeds (slow, medium, fast), for a total of 9600 samples. The samples are stored in the $N Recognizer‘s data format, and each person’s samples are separated into user-speed sub-folders. See more details on the gestures, the users who entered them, and $N’s accuracy in recognizing them in our GI 2012 paper. You may download the dataset here. If you use it in your work, please cite us!
Last week I presented our work on enhancing our $N multistroke recognizer by integrating Yang Li‘s Protractor matching method at GI 2012. The presentation was more algorithm-focused than most of the HCI talks in the conference (and most of my other work), but still generated a few interesting questions. The slides for the talk are posted here. Stay tuned for future enhancements to the $-family of recognizers!