Hacker News

points by mbesto 7 hours ago | 0 comments

Am I missing something, how is there any sort value in these assistants acting as agents on your behalf when they are basically a black box? Like if it can't use my drivers license or my phone number or my credit card, it can't do 75% of things people claim it can do without being massive security issues. What am I missing?

For the record I use OpenClaw but it has its own services (gmail address, phone number, etc.) and I don't give it any of mine, which inherently limits its capabilities.

[–] andai 7 hours ago

Yeah, that's the core of the issue:

https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

The "lethal trifecta": To be useful, an agent needs internet access, untrusted inputs, and your private information.

So it can get prompt injected and exfiltrate your data. (Indeed I read that Claude Code puts your email address into HTTP requests it makes to random servers?)

My solution here was to just not give it anything, but that does preclude the "my claw bought a car!" result everyone's excited about.

My way of thinking here is "blast radius". Assume it will get prompt injected. What goes wrong? Then design the setup so that scenario is not bad. e.g. it has my keys but they've got $5 of credit on them. It has access to the filesystem, but only on its own Linux account. (Ok I set this up last year before they could communicate thru the bloody CPU cache, might need an update :)

[–] simonw 6 hours ago

I've recently talked to people at Anthropic who are very confident that they have effectively solved the prompt injection risks that cause lethal trifecta attacks with their latest models.

The open challenge is how to prove it!

[–] epicureanideal 2 hours ago

> The open challenge is how to prove it!

It could be proven by Anthropic backed credit cards with terms that protect users if it’s leaked or an agent misuses it.

[–] delis-thumbs-7e 5 hours ago

Somehow this does not fill me with confidence. I think I will keep paying my bills manually, thanks but no thanks.

[–] computably 3 hours ago

Or, you know, automatically, with plain old software. Which works fine and fails in less ridiculous ways.

[–] klysm 4 hours ago

I’ll believe them when they put the lethal trifecta with their own skin in the game

[–] julianlam 3 hours ago

Like the guy who put his SSN on the side of a truck and asked people to steal his identity?

I like it!

IIRC he had his identity stolen a bunch.

----

> In 2005, Davis and Robert Maynard Jr. cofounded LifeLock,[10] a personal fraud protection company.[11] Davis was CEO of LifeLock since its founding. As part of a marketing campaign to promote his company's credit monitoring services, Davis posted his own social security number[12] on billboards and commercials. As a result, he was a victim of at least 13 cases of identity theft between 2007 and 2008, belying LifeLock's claims that its services protect consumers against identity theft and fraud.[13] The Federal Trade Commission imposed a $12 million penalty on the company for deceptive advertising, likening Davis and LifeLock marketers to "con artists."[14]

https://en.wikipedia.org/wiki/Todd_Davis_(businessman)

[–] mbesto 5 hours ago

Isn't prompt injection just one facet of the trifecta? Like, I just don't know how it's remotely possible to have a non-deterministic block box ever deterministically know whether your credit card has been compromised or not. Even if you were hacked, how would you even know?

[–] skinfaxi 4 hours ago

I thought the trifecta required all three not just pick two?

[–] jeremyjh 5 hours ago

They can’t solve the model being a complete idiot though.

[–] yunwal 4 hours ago

Or the user being an idiot. Saying this somewhat sarcastically, but there’s always someone who says “Go buy me a Rolex and have it shipped to my house” and then didn’t realize that could be $25,000

[–] cyanydeez 4 hours ago

oh, it's very easy to prove; just let the models and agents do the IPO for them.

[–] simianwords 3 hours ago

What does this have to do with prompt injection?

[–] simianwords 3 hours ago

I was downvoted on the suggestion that trivial prompt injection attempts are mostly solved.

https://news.ycombinator.com/item?id=50162483

[–] ryoshu 6 hours ago

Don’t use LLMs to enforce invariants.

[–] pkaye 6 hours ago

From what I've read Claude Duet will not purchase anything without manual approval.

[–] andai 6 hours ago

Right, so we can distinguish between "model did an oopsie" and "model was actually hijacked".

The former seems to be a much bigger problem. I've seen lots of reports of "Claude/codex bypassed security / stole API keys to complete a task", where that was the "intended" behavior — at least from the LLM's POV, trained to persist and complete user's task at all costs.*

I've actually seen zero reports of actual hijacking due to prompt injection, though they must be out there too.

---

* See also the recent thing about how every model was willing to do fraud to get a refund:

https://news.ycombinator.com/item?id=50140462

[–] NikolaNovak 2 hours ago

The problem is how do you define a "purchase"?

The agent crawls some websites and reads some stuff and pushes some buttons. It's my limited understanding that you have

1. Hard limits like "file system permissions" (* never mind it going around for sake of this conversation)

3. On other hand of spectrum, fundamentally soft limits like a prompt "and ask me before you purchase "

2. And then in the middle you have harness enforced limits like "it won't do X with approval"... Those are being sold as hard limits but frequently they have classifiers and heuristics that essentially try to figure out which button on some third party website constitutes a "purchase" and triggers the harness limit.

Bottom line - I don't think such limits are as 100% fool proof as they are advertised / interpreted.

Very happy to be explained wrong :).

[–] staller 5 hours ago

They support payments through Shop and Link. I don't know about the former, but Link requires that you do a separate approval for purchases. The Duet cannot just take a card and do what it wants.

[–] FireBeyond 4 hours ago

Sure. And I told Claude Code the other week to commit, merge and push some code.

It said "Done, you can open the local dev instance and see the changes".

I could not.

So I asked it. "Well, no, you're not able to see those changes because they're not on that branch or committed."

"But I told you to do that and you said you had."

Claude's verbatim reply?

"You're right. You asked me to do that and I said I would, then told you I had done it, but I had not. Would you like me to do it now, then?"

[–] epicureanideal 2 hours ago

Perhaps services could permit buying things with special AI credit card numbers that link to a special account that the user needs to manually approve by some period of time or else the order is canceled? Works differently though than normal payments where they can just ship the product or deliver the service based on receiving the credit card number.

[–] epicureanideal 2 hours ago

Could call this “reserving” perhaps rather than purchasing, or maybe the purchase would be auto confirmed if under a certain dollar amount or combination of amount and waiting period.

Vendors could provide terms by API, MCP, or just in writing on the page where agents could read it, how quickly a confirmation is needed or what the terms of the reservation or purchase are.

[–] neuronexmachina 3 hours ago

Does it work with Claude's 1Password connector? https://1password.com/blog/1password-for-claude

[–] BryantD 7 hours ago

FWIW, my biggest use case for agents like this is ongoing research. I.e.: check Humble Bundle weekly and tell me if there’s a book bundle I might be interested in. Has the Music Box announced a film series I might want to visit Chicago for? That sort of thing; I want filters for the firehose.

[–] edot 2 hours ago

Isn’t this just a scheduled prompt? That’s already a feature. We’re reinventing the cron job.

[–] yako21000 2 hours ago

The AI-cron job.

[–] R_D_Olivaw 5 hours ago

I like this and makes me think of making your own content algorithm profiles for the firehose. Let your agents scour the net for the things you already like, across platforms.

[–] largbae 7 hours ago

I get a lot of value out of my Hermes setup, but it does not have my banking, or private data beyond GitHub projects that are shared with it. It has its own email and GitHub account, and access to drive Codex, Claude Code or Cursor. I treat it as an assistant via Slack.

It does research on products or real estate that I am interested in, it builds apps and other software projects for me, and does structured research on any topic. Information comes back to me via Slack, GitHub PRs, or websites/services served on the Mac that Hermes runs on.

Using an agent from Slack also adds a killer feature: multi-player contexts. I can work on changes with other human collaboators right in the same thread.

[–] dominotw 7 hours ago

this is your brain on ai twitter :)

[–] simianwords 3 hours ago

It’s a good sneer but in a few years this would be normal and you’d forget that you were sneering at this. I saw this happening when people said they used AI for coding in 2024.

so is it normalized for people to sneer at folks who happen to be ahead of the curve now?

[–] dominotw 2 hours ago

so advanced. What did you achieve since 2024 bro. Still working the same job.

[–] simianwords 1 hour ago

Yes but my relation to work has changed significantly. I haven’t written any code myself.

You’d have probably sneered at the suggestion that this would be possible in 2024. You’d be wrong then and you’d be wrong again now.

[–] cloudgazing 7 hours ago

In the article the author mentions that Duet doesn't seek any sort of credentials and mentions connecting repos, so I think that this assistant is being used more for managing coding work in a different way than managing life work. I do agree that to get to true "personal assistant" levels, like planning a whole trip for you, booking tickets, etc. you'd have to give pretty much full access to your identity and money, but people claiming it can do that probably are just handing it over, security issues aside.