Hacker News

points by andai 7 hours ago | 0 comments

Yeah, that's the core of the issue:

https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

The "lethal trifecta": To be useful, an agent needs internet access, untrusted inputs, and your private information.

So it can get prompt injected and exfiltrate your data. (Indeed I read that Claude Code puts your email address into HTTP requests it makes to random servers?)

My solution here was to just not give it anything, but that does preclude the "my claw bought a car!" result everyone's excited about.

My way of thinking here is "blast radius". Assume it will get prompt injected. What goes wrong? Then design the setup so that scenario is not bad. e.g. it has my keys but they've got $5 of credit on them. It has access to the filesystem, but only on its own Linux account. (Ok I set this up last year before they could communicate thru the bloody CPU cache, might need an update :)

[–] simonw 6 hours ago

I've recently talked to people at Anthropic who are very confident that they have effectively solved the prompt injection risks that cause lethal trifecta attacks with their latest models.

The open challenge is how to prove it!

[–] epicureanideal 2 hours ago

> The open challenge is how to prove it!

It could be proven by Anthropic backed credit cards with terms that protect users if it’s leaked or an agent misuses it.

[–] delis-thumbs-7e 5 hours ago

Somehow this does not fill me with confidence. I think I will keep paying my bills manually, thanks but no thanks.

[–] computably 3 hours ago

Or, you know, automatically, with plain old software. Which works fine and fails in less ridiculous ways.

[–] klysm 4 hours ago

I’ll believe them when they put the lethal trifecta with their own skin in the game

[–] julianlam 3 hours ago

Like the guy who put his SSN on the side of a truck and asked people to steal his identity?

I like it!

IIRC he had his identity stolen a bunch.

----

> In 2005, Davis and Robert Maynard Jr. cofounded LifeLock,[10] a personal fraud protection company.[11] Davis was CEO of LifeLock since its founding. As part of a marketing campaign to promote his company's credit monitoring services, Davis posted his own social security number[12] on billboards and commercials. As a result, he was a victim of at least 13 cases of identity theft between 2007 and 2008, belying LifeLock's claims that its services protect consumers against identity theft and fraud.[13] The Federal Trade Commission imposed a $12 million penalty on the company for deceptive advertising, likening Davis and LifeLock marketers to "con artists."[14]

https://en.wikipedia.org/wiki/Todd_Davis_(businessman)

[–] mbesto 5 hours ago

Isn't prompt injection just one facet of the trifecta? Like, I just don't know how it's remotely possible to have a non-deterministic block box ever deterministically know whether your credit card has been compromised or not. Even if you were hacked, how would you even know?

[–] skinfaxi 4 hours ago

I thought the trifecta required all three not just pick two?

[–] jeremyjh 5 hours ago

They can’t solve the model being a complete idiot though.

[–] yunwal 4 hours ago

Or the user being an idiot. Saying this somewhat sarcastically, but there’s always someone who says “Go buy me a Rolex and have it shipped to my house” and then didn’t realize that could be $25,000

[–] cyanydeez 4 hours ago

oh, it's very easy to prove; just let the models and agents do the IPO for them.

[–] simianwords 3 hours ago

What does this have to do with prompt injection?

[–] simianwords 3 hours ago

I was downvoted on the suggestion that trivial prompt injection attempts are mostly solved.

https://news.ycombinator.com/item?id=50162483

[–] ryoshu 6 hours ago

Don’t use LLMs to enforce invariants.

[–] pkaye 6 hours ago

From what I've read Claude Duet will not purchase anything without manual approval.

[–] andai 6 hours ago

Right, so we can distinguish between "model did an oopsie" and "model was actually hijacked".

The former seems to be a much bigger problem. I've seen lots of reports of "Claude/codex bypassed security / stole API keys to complete a task", where that was the "intended" behavior — at least from the LLM's POV, trained to persist and complete user's task at all costs.*

I've actually seen zero reports of actual hijacking due to prompt injection, though they must be out there too.

---

* See also the recent thing about how every model was willing to do fraud to get a refund:

https://news.ycombinator.com/item?id=50140462

[–] NikolaNovak 2 hours ago

The problem is how do you define a "purchase"?

The agent crawls some websites and reads some stuff and pushes some buttons. It's my limited understanding that you have

1. Hard limits like "file system permissions" (* never mind it going around for sake of this conversation)

3. On other hand of spectrum, fundamentally soft limits like a prompt "and ask me before you purchase "

2. And then in the middle you have harness enforced limits like "it won't do X with approval"... Those are being sold as hard limits but frequently they have classifiers and heuristics that essentially try to figure out which button on some third party website constitutes a "purchase" and triggers the harness limit.

Bottom line - I don't think such limits are as 100% fool proof as they are advertised / interpreted.

Very happy to be explained wrong :).

[–] staller 5 hours ago

They support payments through Shop and Link. I don't know about the former, but Link requires that you do a separate approval for purchases. The Duet cannot just take a card and do what it wants.

[–] FireBeyond 4 hours ago

Sure. And I told Claude Code the other week to commit, merge and push some code.

It said "Done, you can open the local dev instance and see the changes".

I could not.

So I asked it. "Well, no, you're not able to see those changes because they're not on that branch or committed."

"But I told you to do that and you said you had."

Claude's verbatim reply?

"You're right. You asked me to do that and I said I would, then told you I had done it, but I had not. Would you like me to do it now, then?"