Hacker News

points by thehamkercat 3 days ago | 0 comments

Other providers probably don't even know/care that this is happening

Last year it happened to me more than once that dictating something in chatgpt would become a completely different message after processing, not in a misheard type of way but just a separate conversation more easily explained by the same type of KV cache issue here.

[–] kami23 3 days ago

I think in the last year I've had it happen once where I seemingly got someone else's response, it was very odd. I rewound and got what I expected. It was correlated to Anthropic's roll out of new models at the time which always seems to cause instability in their apis.

[–] kolanos 3 days ago

Other providers might have better isolated KV stores?

[–] jmalicki 3 days ago

I've had this happen with ChatGPT too.

[–] jmalicki 2 days ago

(Some random internet user pasted me things that were clearly from my ChatGPT session, he identified me because it had links to my github). So it wasn't a hallucination, it appeared to be some sort of KV cache leakage, though I obviously don't know the true root cause of the bug.

[–] Orygin 3 days ago

I wonder if the chatgpt shirtless guy is from the same kind of issue