GoldiMask changes how diffusion language models use training context

Researchers introduced GoldiMask, a fine-tuning method for discrete diffusion language models, in a paper submitted to arXiv on 29 Sep 2026. The work focuses on how supervised fine-tuning masks response tokens: the model learns to restore hidden tokens using the ones left visible. The researchers say that uniform random masking does not explicitly account for how the choice of visible context and the choice of prediction targets affect one another.

GoldiMask first chooses which tokens to expose as context, using model signals in an objective that approximately maximizes a submodular function. The objective balances the benefit of revealing a token with its value as a target for prediction. The method then assigns weights to the remaining targets, taking into account how much the chosen context helps with each one and its remaining potential for learning. In this way, context selection and target weighting are both part of the fine-tuning procedure.

The paper evaluates GoldiMask with three model backbones and three training datasets. It reports that the method achieves the best average accuracy in most of the tested settings, with improvements on reasoning and code-generation tasks. The researchers also report ablation results, which indicate that both selecting context and weighting targets contribute to the gains. The abstract does not specify individual accuracy scores or describe how large the improvements were.

The study also reports an effect on inference: GoldiMask reduced the number of decoding iterations on GSM8K and MATH-500 when using confidence-threshold parallel decoding. Accuracy remained comparable when decoding used higher confidence thresholds, according to the paper. These results concern the listed tasks and evaluation setup; the source does not claim that the method improves decoding for all models or workloads.

For AI builders, the paper presents fine-tuning masks as more than a way to hide tokens: their design can shape both the information available during prediction and which predictions receive emphasis. GoldiMask’s reported results suggest that coordinating those choices can matter for reasoning and code generation, while the ablations point to contributions from both components. The findings are research results across the reported backbones and datasets, rather than evidence of a general benefit in every deployment setting.