+********************************************************************** * * * Einladung * * * * Informatik-Oberseminar * * * +********************************************************************** Zeit: Dienstag, 5. März 2024, 10.00 Uhr Ort: Raum 025, Mies-van-der-Rohe Str. 15 (UMIC Gebäude) Referent: Ali Athar, M.Sc. Lehrstuhl Informatik 13 Thema: Segmenting and Tracking Objects in Video Abstract: Research to develop methods that can accurately localize and track objects in video has been ongoing for decades. Approaches capable of accomplishing this are highly sought after for a variety of applications including autonomous robots, self-driving vehicles, sports analytics, video editing, etc. Despite significant progress in recent times, the task is far from solved, in particular for challenging scenarios involving occlusions, motion blur, and camera ego-motion. In this thesis, we present a series of works that advance the state of research in this domain in various ways, as outlined below. Our first work, STEm-Seg, is an end-to-end trainable method for instance segmentation that models the input video as a single 3D space-time volume and relies on clustering per-pixel embeddings to segment and track objects. This differs from existing approaches, which largely follow the tracking-by-detection paradigm. Our novel formulation for these embeddings enables us to cluster the embeddings in an efficient and end-to-end learned fashion. The second work, called HODOR, is aimed at mitigating the need for densely annotated data for training video tracking methods. Specifically, it tackles the task of Video Object Segmentation (VOS) in a weakly supervised manner where it can be trained using static images or sparsely annotated video. To this end, we adopt a novel approach that encodes objects into concise descriptors. This is in contrast to existing approaches that predominantly learn space-time correspondences, which makes it challenging to train them in such a setting. Whereas the two aforementioned works propose network architectures, our third project proposes a dataset and benchmark called BURST that aims to unify the current, fragmented landscape of datasets in video segmentation research. BURST includes a benchmark suite that evaluates multiple tasks related to object segmentation in video with shared data and consistent evaluation metrics. The idea behind this is to facilitate knowledge exchange between the research sub-communities tackling these tasks and also to encourage the development of methods with multi-task capability. Finally, our fourth work, TarViS, can be seen as a logical continuation of the above in that it is a method that can tackle multiple video segmentation tasks. To achieve this, we decouple the task definition from the core network architecture and use a set of dynamic query inputs to specify the task-specific segmentation targets. This formulation enables us to train a single model jointly on a collection of datasets spanning multiple tasks (Video Instance/Object/Panoptic Segmentation). During inference, the model can switch between tasks by simply hot-swapping the input queries accordingly. Es laden ein: die Dozentinnen und Dozenten der Informatik