Investigated
What is distillation-based copying?
Short answer
Image: maxim-blog.ghost.io
Distillation-based copying trains a student model on a teacher model’s outputs—sometimes probability distributions or logits, sometimes sampled responses—to reproduce capabilities more cheaply or in a smaller system. It can be a legitimate compression technique, but API-based extraction may be described as copying when outputs are collected to build a substitute model; the label alone does not determine whether infringement occurred.
The views on this question
Two lenses on the same evidence. Source weight and the primary source ratio show what each rests on.
Alternative
Source weight 45%Lens adapted to this topic: Replication, provenance, and disputed boundaries
A broader and more critical usage treats distillation-based copying as capability extraction from an accessible model, especially through large-scale API queries. This perspective emphasizes that students can reproduce distinctive behaviors or teacher-derived sentences, and that the boundary between ordinary learning, model replication, and unauthorized appropriation depends on access practices, provenance, and legal rights.
- Black-box distillation can use API outputs—and, where exposed, partial logits—to construct a functional substitute model under query constraints.
- Research summarized in the sources reports that distilled students may reuse teacher sentences in new scenarios, making provenance and memorization important questions.
- Large-scale targeted querying to extract capabilities is described by critics as a distillation attack, with concerns about originality, safety, and ecosystem monoculture.
- Calling distillation “IP theft” is not technically or legally conclusive; the relevant question is which protected right, if any, was infringed and by what conduct.
- Outputs can transfer selected capabilities without automatically transferring safety behavior or censorship, because the student learns from whatever the training transcript demonstrates.
Follow the white rabbit
Deeper threads worth pulling on next.
- How does knowledge distillation work?Different angle, same storyExplains key techniques like soft labels, temperature scaling, and loss functions used in teacher→student training.
- Can API extraction be legal copying?Different angle, same storyExamines legal distinctions around using API outputs to train models, including contracts and copyright.
- What are model stealing attacks?