PhD defense of Hortensia BARRAL

Hortensia BARRAL

Self-supervised learning for infrared video enhancement

Abstract

This thesis contributes to the problem of infrared video denoising, in particular fixed pattern noise (FPN) denoising, addressing the problem at different stages and with different methodologies. Firstly, we propose new algorithms for the FPN calibration stage, based on a realistic model of the FPN that takes into account integration time and can model non-linear temperature sensor response. Then we focus on FPN denoising after calibration. We propose a generalization of the FPN removal problem in which several synchronized videos are acquired with the same camera. This has practical relevance for applications in video surveillance that use a rotating camera. We propose a novel algorithm based on the minimization of an energy to remove it. We then turn our attention to the application of neural networks (NNs) to FPN denoising. Infrared sensors pose several challenges that prevent the application of NNs to practical cases. The absence of ground truth data prevents supervised training on real data. Plus, infrared images suffer from spatially and temporally correlated noise which prevents the application of self-supervised techniques, which rely on assumptions of spatial or temporal noise independence. We propose new network architectures as well as new self-supervised learning methods, enabling model training without ground truth.

Key Words

infrared, denoising, Self-supervised, learning, video restoration

Supervision

• Pablo ARIAS

Jury

• Said LADJAL, Professeur, Télécom Paris, Rapporteur
• Antoni BUADES, Professeur, Universitat de les Illes Balears, Rapporteur
• Charles KERVRANN, Directeur de recherche, Université de Rennes, Examinateur
• Jean-François AUJOL, Full professor, Université de Bordeaux, Examinateur
• Giuseppe VALENZISE, Chargé de recherche, CentraleSupélec, Examinateur