(Some random internet user pasted me things that were clearly from my ChatGPT session, he identified me because it had links to my github). So it wasn't a hallucination, it appeared to be some sort of KV cache leakage, though I obviously don't know the true root cause of the bug.