Hacker News
points by 3form 13 hours ago | 0 comments
"Cyber benchmarks, while achieving an average refusal rate of 96.6% on malicious cyber requests, the highest we measure among open models."

I feel like stating the obvious here and hence like I'm missing something. Where is the statistic about not refusing the non malicious requests? I can achieve the same performance with refusing when random(1000) < 967.

[–] kouteiheika 11 hours ago
These statistics and safeguards are pretty much meaningless, because it's completely trivial to get around them. You can prefill and edit the model's responses/thinking and force it to agree to virtually anything even with unmodified weights, and if you don't want to bother with jailbreaking and can host it yourself it's essentially a one-man weekend project to uncensor the model and have it agree with anything you want by default.

Guardrails on open-weight models is security theater purely to shield whoever released them from a potential PR nightmare.

[–] prophesi 10 hours ago
It's unfortunate, because the security theater performed here detracts from the model's overall cybersecurity performance. And uncensoring has another small hit to performance.
[–] pixl97 13 hours ago
Yea, the problem with maliciousness is intent.

Much like having a hacking tool doesn't mean you're doing anything illegal, attacking a machine doesn't necessarily mean you're committing a crime. I can hack the living shit out of my local network if I want to.

[–] nicce 6 hours ago
Somehow that statement just screams that our model is not useful for security work
[–] ex-aws-dude 8 hours ago
It is kind of comical how things that previously would have been obviously true are now only statistically true

"Our product won't hack you 96.6% of the time"

[–] nikanj 13 hours ago
Nobody cares about that number, because refusals don't lead to company-ending news headlines.