PIF2024 - CONECTANDO EL PASADO, EL PRESENTE Y EL FUTURO: ADQUIRIR Y APROVECHAR CONOCIMIENTO APRIORI PARA REVELAR LA ESTRUCTURA TEMPORAL DE VIDEOS SIN RECORTAR - PID2023-151351NB-I00

The goal of this project is to lay the methodological foundations for endowing temporal sequence encoding, the task of representing temporal sequences, with an high level of temporal abstraction so that prior knowledge about the temporal unfolding of these sequences can be acquired, properly represented and deployed at inference time to better understand the present and anticipate the future. The methods developed will be validated in the challenging computer vision domain of untrimmed video understanding, to address the interrelated tasks of temporal action segmentation, localization and anticipation under a common representation framework. Since Language, together with Vision, is a fundamental modality through which human beings acquire knowledge about the world, we will use it to acquire prior knowledge about the unfolding of temporal sequences. The main tasks to be performed can be organized around three main objectives: 1) Developing an unsupervised hierarchical representation of videos suited for downstream tasks requiring high levels of temporal abstraction, 2) Acquiring and representing prior knowledge about the unfolding of events, 3) Deploying prior knowledge at inference time in the tasks of temporal action localization, segmentation and anticipation.