The challenge has two subtasks which aim at promoting studies in the detection and interpretation of the relevant gesturing in the context of conversational interactions. The first subtask is to recognize the interlocutor’s gestures in general so as to distinguish those that have a communicative function (e.g. giving feedback) from those that are other type of gestures (e.g. scratching an itchy arm). The second subtask is to classify the communicative gestures further, and to use relevant features for the recognition of feedback giving features. The data used for this challenge concerns first encounter dialogues in Finnish and Swedish languages, and is collected within the NOMCO project [1,2].

Evaluation metric

In order to evaluate the performance of the methods targeting this task, the confusion matrix and the f1-score should be provided. In the detection of a gesture, a frame-based counting will be applied. The recognition method should output one of the annotated classes or “no class” for each frame. This will be compared to the ground truth in order to build a confusion matrix.


Please see [2].


[1] C. Navarretta, E. Ahlsén, J. Allwood, K. Jokinen and P. Paggio (2012). Feedback in Nordic First-Encounters: a Comparative Study. Proceedings of the Language Resources and Evaluation Conference (LREC-2012). Istanbul, Turkey.
[2] The NOMCO project.