When is augmentation helpful?

Lens adapted to this topic: Limits and competing explanations

A serious skeptical reading argues that augmentation is often credited too broadly. Some gains may come from altered optimization, extra fine-tuning, or regularization rather than from faithful new data. Augmentation can amplify generator mismatch, add label noise, or fail to improve performance when the original dataset already captures the relevant variation. Comparisons should therefore use strong training baselines, realistic held-out tests, and cost-aware evaluation.

• Synthetic augmentation is not universally beneficial: in locally symmetric settings it may fail to improve learning rates or may worsen performance through generator mismatch.
• Some textual augmentation gains disappear when models receive additional fine-tuning without augmentation, suggesting training procedure—not new information—may explain them.
• LLM-generated text does not consistently outperform established augmentation methods, and its cost-benefit advantage may be limited.
• Augmentations that alter the training distribution can help through implicit regularization, but this does not guarantee that the generated examples resemble valid real-world data.
• Common augmentation methods may require very many generated points before producing a substantial positive margin, limiting practical robustness claims.

6 sourcesView sources