Are discrete diffusion models with few data just optimised copy-paste generative methods ?

Diffusion models achieve strong generative performance in many applications, but remain difficult to interpret. This issue is particularly important in medical imaging, where datasets are often small and the novelty of generated images must be carefully assessed. Motivated by the generation of breast anatomy images from limited training data, we study what continuoustime discrete diffusion models learn and generate when trained on few images. Focusing on convolutional architectures, we model their inductive biases by constraining the score function to be local and translation-equivariant, and derive the optimal score under the empirical training distribution for a uniform corruption process. We prove that the corresponding reverse diffusion process can generate only images that satisfy the patch mosaic rule of local consistency. Our analysis builds on the seminal work of Kamb and Ganguli for continuous state-space Gaussian diffusion models and establishes a discrete state-space counterpart under a uniform corruption process. We further establish a connection between the resulting reverse diffusion procedure and classical exemplar-based image synthesis methods. In particular, we show that Tweedie τ -leaping with the optimal constrained score and a modified version of the optimization-based texture synthesis algorithm of Kwatra et al. rely on closely related patch-copying operations. Experiments on 2D breast anatomy images show that practical convolutional score networks produce samples close to those generated by the theoretical optimal score when using identical noise seeds. Finally, we quantify the sample novelty of these convolutional discrete diffusion models, across several training settings.