Home | Publications | BCL+22

Multi-Object Tracking and Segmentation via Neural Message Passing

MCML Authors

Guillem Brasó

* Former Member

→ Group Daniel Cremers
Computer Vision & Artificial Intelligence

Laura Leal-Taixé

Prof. Dr.

Principal Investigator

* Former Principal Investigator

Abstract

Graphs offer a natural way to formulate Multiple Object Tracking (MOT) and Multiple Object Tracking and Segmentation (MOTS) within the tracking-by-detection paradigm. However, they also introduce a major challenge for learning methods, as defining a model that can operate on such structured domain is not trivial. In this work, we exploit the classical network flow formulation of MOT to define a fully differentiable framework based on Message Passing Networks. By operating directly on the graph domain, our method can reason globally over an entire set of detections and exploit contextual features. It then jointly predicts both final solutions for the data association problem and segmentation masks for all objects in the scene while exploiting synergies between the two tasks. We achieve state-of-the-art results for both tracking and segmentation in several publicly available datasets.

article