Hacker News

points by justech 23 hours ago | 0 comments

Not to defend Claude here but the only instances this happened was when

1. It was said to be required (e.g. Searching through SEC)

2. Politeness policy (Wikimedia's bot policy and Crossref's "polite pool" ask automated clients for contact details)

So I think it wasn't because of malicious reasons as far as I can tell.

I did instruct Claude to never do it again except for the politeness policy. I'm assuming WikiMedia requested this so that they have someone to contact in case a bot overloads their servers

[–] shakna 21 hours ago

Should an agent not ask for confirmation, before leaking their patron's contact information to any random third party who asks for it?

If not... That sounds like primetime for prompt injection.

[–] everforward 17 hours ago

It seems reasonable to me to ask, since otherwise they have no way to ask you to stop breaking the site.

There are better ways to do this, they just haven’t been built yet.

Eg this may not need actual contact info. Wikipedia just wants a way to ask you to stop. Anthropic could add an opaque agent/session identifier that Wikipedia could flag, and Anthropic just forwards that on to you in the UI.

If you don’t have free text fields it’s usually not super useful for spam. There are ways to get a binary “good/bad” signal back without that though.

[–] jethro_tell 17 hours ago

Just set the evil bit and they can block it at ingress

[–] pessimizer 16 hours ago

What does this have to do with whether the agent should ask for confirmation?

[–] timcobb 19 hours ago

Claude has caught this exact mistake for me, and asked for forgiveness, I think this was in the past few weeks it was like "omg I sent your email address in a UA". It seemed minor :shrug:.

[–] shakna 11 hours ago

If I can exfiltrate your email with something that responds in that way, what other information can I pull from your system, without your intervention?

If I made it send something, I probably made it make a tool call. What other tools can I make it call?

I'm afraid this isn't a shrug - it's a zero-click vulnerability. It might be the start of a chain to RCE. It doesn't have to work every time, either. Just enough of the time.

[–] kardos 23 hours ago

Instructions are not guardrails. Best case, that just makes it less likely to do it

[–] brookst 18 hours ago

What exactly are guardrails then? Impenetrable barriers that can never be overcome?

[–] ireadmevs 18 hours ago

Deterministic assurances on top of nondeterministic tools

[–] timcobb 19 hours ago

Even Claude admits this after a bit of prodding.

[–] Bric3d 19 hours ago

For what it's worth, I wanted to scrape some document from the SEC and claude explicitly told me this was required and asked me if it should do it.

[–] nbajka 18 hours ago

Would I want Firefox to include my personal email in the headers of my requests just because the website politely asked for it? No, absolutely no, never. This is bonkers.