Improving Thematic Balance in Press Release Generation with GMMs and Dimensionality Reduction

Large Language Models (LLMs) are widely used to generate high-quality textual content, such as news articles and business reports. However, their performance is highly dependent on the quality and balance of the training data. In real-world editorial settings, corpora are often thematically imbalanced, leading to generation biases and poor coverage of minority topics. To address this challenge, we introduce a targeted data augmentation framework that combines dimensionality reduction with Gaussian Mixture Models (GMMs) to identify and enrich underrepresented regions of the semantic space. We tune UMAP to preserve the original data structure by maximizing the V-measure between cluster assignments in the reduced space and a reference clustering in the original embedding space, in accordance with UMAP's goal of topology preservation. GMMs are then fitted in the reduced space to detect sparse clusters, where synthetic samples are generated via LLM prompting. The augmented corpus is used to fine-tune LLMs for text generation. Experimental results demonstrate improvements in lexical overlap (ROUGE), precision (BLEU), and semantic similarity (BERTScore), showcasing the effectiveness of targeted augmentation in enhancing LLM-based generation.