Abstract

Skeleton-based action/gesture recognition has already witnessed excellent progress on processing large-scale, laboratory-based datasets with pre-defined skeleton joint topology. However, it’s still an unsolved task when it comes to real-world scenarios with practical limitations such as small-scaled dataset sizes, few-labeled samples, and various skeleton topologies. In this paper, we work on the recognition of micro-gestures, which are subtle body gestures collected in real-world scenarios. Specifically, we utilize contrastive learning to heritage the knowledge from known large-scale datasets for enhancing the learning on fewer samples of micro-gestures. To overcome the gap caused by various domain distributions and structure topologies between the datasets, we compute skeleton representations from augmented sequences via momentum-based efficient and scalable encoders as additional inductive priors. Importantly, we propose an effective dense-graph based unsupervised architecture that resorts to a queue-based dictionary to store positive and negative keys for better contrast with queries to learn substantially efficient and discriminant patterns in the feature space. Together with cross-dataset experimental results show that our model significantly improves the accuracies on two micro-gesture datasets, SMG by 7.4% and iMiGUE by 18.41% advocating its superiority.