From
Timetable to
Place Salle 1G58, ENS Paris-Saclay,
Yanhao LI
Noise, Compression and Resampling in Image and Video Forensic
Abstract
The invention of digital cameras and the development of editing software have made it incredibly easy to manipulate images and videos. The emergence of deep generative models has further facilitated the manipulations with higher realism. To counteract malicious manipulations, we begin our study by analyzing the noise characteristics in images and videos, which serve as passive signatures. Specifically, we investigate the impact of JPEG compression on image prior noise, and exploit temporal redundancy in videos for accurate noise estimation.
However, when it comes to detecting fake videos, most manipulated videos undergo strong compression that distorts the underlying noise characteristics, making noise-based analysis challenging. Fortunately, video compression itself leaves distinct traces, and manipulated videos are often subjected to multiple compressions. Motivated by this, we propose a method for detecting video double compression, allowing us to verify video authenticity by analyzing the spatial consistency of compression traces to authenticate a video.
The year 2022 marked a turning point with the emergence of Stable Diffusion for generating realistic fake images, making the detection of synthetic images more crucial than ever. In response, we propose a method for detecting synthetic images by revealing artifacts in the Fourier domain to achieve effective, generalizable and explainable detection. As synthetic images are usually compressed by JPEG which degrades the detection performance, we further explore the possibility of improving the detection by suppressing JPEG artifacts prior to the detection.
Finally, we observe that resampling is a common operation in both image generation and manipulation, leaving distinctive artifacts that can be analyzed in the Fourier domain. We thus conduct a theory to describe the spectral correlations introduced by resampling, and propose an unsupervised detection method that is effective across various resampling scenarios.
Through this comprehensive study, we bridge the gap between compression, noise modeling, artifact analysis, and forgery detection, contributing to the advancement of image and video forensics.
Key words
image forensics, video forensics, resampling detection, synthetic image detection, noise estimation, a contrario detection
Supervision
- Rafael GROMPONE VON GIOI, Chercheur, Centre Borelli, ENS Paris-Saclay (directeur de thèse)
- Miguel COLOM, Chercheur, Centre Borelli, ENS Paris-Saclay (co-directeur de thèse)
- Jean-Michel MOREL, Professeur, City University of Hong Kong (co-directeur de thèse)
Jury
- Patrick BAS, Directeur de recherche, CNRS, Université de Lille (Rapporteur & Examinateur)
- Agnès DESOLNEUX, Directrice de recherche, CNRS, ENS Paris-Saclay (Examinatrice)
- Vicky KALOGEITON, Professeure adjointe, École Polytechnique, IP Paris (Examinatrice)
- Luisa VERDOLIVA, Professeure, University Federico II of Naples (Rapporteuse & Examinatrice)