Hacker News

points by johncassil 8 hours ago | 0 comments

Good question -- I have been pointing claude at repos for security reviews before I install stuff now for exactly this reason.

[–] ShinyLeftPad 7 hours ago

It's so funny to read these, as if countless examples didn't prove that llms don't actually distinguish between your instructions and the data they read at your request.

Do you think a malicious repo author won't think of hiding a suitable prompt within?

[–] NavinF 7 hours ago

> countless examples

Show me just one where a frontier model got prompt injected when checking if code if malicious.

FWIW I can do this, but I bet you can't.

[–] ShinyLeftPad 7 hours ago

Here's prompt injection via a harness for example: https://www.promptarmor.com/resources/google-antigravity-exf...

took me about 30 seconds to find. there's tons of them. of course it's an arms race, the models get better the harnesses get better the attackers get better but again: for the model itself, your prompt and third party data it reads are the same thing (input). as long as it is true, the attack will always be possible.