points by jrmg 2 days ago | 0 comments
I assumed that what they’re ’collecting’ there is at most stats on how much you like the product. Especially because, in my experience, if you say ‘good’ the next question is one that explicitly asks whether you permit Anthropic to analyze the session.
[–] j45 2 days ago
[–] jrockway 2 days ago
I have no idea if they're doing this, but you can always run the last turn through another LLM and ask it "should I request feedback based on this turn?"
I believe this is how the auto-mode classifier works. It sees a little bit of the transcript (with tool output removed, to avoid prompt injection) and answers "yes, allow the action" or "no, deny the action".
[–] j45 2 days ago
Interesting idea - I'm not sure if there's enough reliability that another LLM would reliably answer the same way over time since it's non-deterministic.
A determination of it has to be made some way beyond looking for frustrated or angry keywords.
If there was direct transparency regarding this it would be great vs digging into it on our own.