Hacker News
points by raesene9 18 hours ago | 0 comments
Yeah for any even vaguely security related tasks, using Anthropic/OpenAI models is a minefield. You might be able to cheat them into bypassing guardrails but that risks getting more seriously blocked.

What's possibly even more annoying is that the guardrails in Claude Code aren't even consistent. I've had cases where just opening a new session and repeating the prompt works fine.

It's much easier to just move over to GLM/Kimi/Deepseek and not worry about it.

[–] nucleardog 16 hours ago
"vaguely security related tasks" is right.

Found a bug in a PHP code analysis tool a while back. Not like "crash the interpreter" or anything, just the application ending up in a weird state and crashing. Couldn't exactly share our entire codebase to report the bug, so after Claude chased down _what_ the issue was, I asked it to make me a minimal reproduction so I could report the bug.

Blocked. Guessing it looked like I was trying to develop an exploit or something. By driving a static code analysis tool to an invalid state where it handled the state by cleanly reporting the issue and stopping. In an interpreted language. With the context full of Claude's own efforts to track down why and how this was happening.

Theoretically? Sure, theoretically I'm sure practically anything could be used for nefarious purposes.

Practically? Not useful in the slightest.

Funny part was--Claude was more than happy to talk with me about why it was potentially blocked and suggest ways to try and work around the guard rails. Suggested I include context like the exact bug tracker I'm reporting to, etc to try and put more weight behind the "good actor reporting a bug" aspect, etc.

[–] dd8601fn 15 hours ago
They’re using classifiers for a lot of this stuff separate from the actual model.

I’ve had the model agree that certain prohibited actions or downgrades were false positives and tell me to start a new session.

For some things, once you’ve tripped the tripwire, that session is cooked.

[–] aiisjustanif 16 hours ago
I’m curious of the use case, I use Sonnet for Insider Threat investigations and detection engineering daily without issue.
[–] nucleardog 16 hours ago
I posted mine in a sibling comment before seeing yours--

I had a static code analysis tool (written in PHP for analyzing PHP code) crash on our codebase. Crash in this cause not being "seg faulted the interpreter" or something concerning, it just hit an invalid state internally, identified it, and cleanly exited with an error to let me know something was wrong. Got Claude to track down what was wrong and how our codebase was triggering it. I asked it to create a minimal reproduction I could include on my bug report, and the guard rails triggered and refused to answer. Asking Claude why that would happen, it suggested because what I was doing looked like exploit development.