Is the actor trained on world model's mistakes as well? If so, it should learn to force hallucinations that force it into an easier state.