A Unified Framework for Multi-Target Tracking and Collective Activity Recognition
People
Abstract
We present a coherent, discriminative framework for simultaneously tracking multiple people and estimating their collective activities. Instead of treating two problems separately, our model is grounded in the intuition that a strong correlation exists between a person's motion, their activity, and the motion and activities of other nearby people. Instead of directly linking the solutions to these two problems, we introduce a hierarchy of activity types that create a natural progression that leads from a specific person’s motion to the activity of the group as a whole. Our model is capable of jointly tracking multiple people, recognizing individual activities (atomic activities), the interactions between pairs of people (interaction activities), and finally the behavior of groups of people (collective activities). We also propose an algorithm for solving this otherwise intractable joint inference problem by combining belief propagation with a version of the branch and bound algorithm equipped with integer programming. Experimental results on challenging video datasets demonstrate our theoretical claims and indicate that our model achieves the best collective activity classification results to date.
Paper
A Unified Framework for Multi-Target Tracking and Collective Activity Recognition
ECCV, 2012, accepted as an Oral Presentation
W. Choi and S. Savarese
[paper]
[supplementary material]
[bibtex]
[pptx (Office 2010)]
[videolectures.net]
Video
[video]
Data
Collective Activity Dataset:
[README]
[images]
[additional annotations]
New dataset:
[README]
[images]
[annotations]
[visualization tool]
Related Works
Learning Context for Collective Activity Recognition
CVPR, 2011
W. Choi, K. Shahid and S. Savarese
[pdf]
[bibtex]
What are they doing? : Collective Activity Classification Using Spatio-Temporal Relationship Among People
9th International Workshop on Visual Surveillance (VSWS09) in conjuction with ICCV 09
W. Choi, K. Shahid, S. Savarese
[pdf]
[project]
[bibtex]