Hacker News

points by butlike 3 days ago | 0 comments

> the second guy can train their model by using the first model to generate the training data

Isn't this somewhat akin to a model learning from its own inputs? Doesn't that behavior lead to model collapse? Either the model is trained on pristine data, in which case it stagnates, or it's distilled from another model, which leads to model collapse.

[–] sfblah 3 days ago

I think the better analogy is kick-starting. Yes, you'd get model collapse eventually, but you can kick-start a new model by cloning an old one and then you start training it like a normal model from then on out.