Hacker News
points by lesspassiveobse 22 hours ago | 0 comments
it's a stunt to convince the government that only they can be trusted to operate it and that their competition/small local deployments cannot.
[–] 0xDEAFBEAD 22 hours ago
Well that would explain why HuggingFace announced that the use of an open-weight model was critical to their investigation of the incident :-)
[–] _heimdall 21 hours ago
And why hugging face didn't turn into a large criminal investigation. If you don't think they can control it, no one may be willing to press charges.
[–] lesspassiveobse 20 hours ago
HF is in business of selling inference for open-weight models. Setting the precedent of "Your business needs to only use approved safety-certified models from trusted providers or you could get sued for what the models do!" wouldn't exactly benefit them. And contrary to what you may believe, nvidia as a whole would also rather not have the entire demand for their products depending on two big companies.
[–] estearum 22 hours ago
Imagine thinking that repeatedly demonstrating your inability to control your own models is "a stunt to convince the government that only they can be trusted to operate it"

Truly spectacular gymnastics. I'll give it an 8/10.

[–] lesspassiveobse 21 hours ago
It's so scary and dangerous that to run LLMs you need to be from this country, have a government license and spend at least $100M a year on 'safety'. We know this first-hand and are leaders in the field of "safety" now that we learned from our mistakes.

And, bam, competition eliminated.

[–] estearum 20 hours ago
No one is confused about how regulatory capture works. We're confused by the allegations that labs are deliberately producing safety problems in order to achieve regulatory capture.

Repeatedly blowing up your own munitions factory is not a good way to prevent other people, but not yourself, from running a munitions factory. Obviously.

Obviously the far better solution is to blow up other people's factories, or make allegations of unsafe conditions at other factories, and repeatedly demonstrate excellence and safety within your own.

Here, award-winning gymnasts are alleging that the frontier labs are simply choosing not to do that, and instead choosing to have explosions and minor fires every few days at their own factories, many of which they try to hide (and probably many more of which they successfully do hide). Believing such astounding bullshit is a testament to the power of motivated reasoning and the conspiratorial mindrot that now plagues our civilization.

[–] lesspassiveobse 17 hours ago
I fail to see how training a hacking model with access to source code of open-source services and other prior knowledge, running it in a "sandbox" with many of those heavy, big attack surface services running on the same network, and being proxies to the general internet, giving the model some access to those services and then prompting "GO HACK YOUR WAY OUT!" is "blowing up your own munitions factory". I see it more as setting fire to your own factory for insurance fraud, or bringing in a bunch of north korean hackers and giving them the same task. It's not exactly something normal users want or are able to do. And those with the compute and know-how will do it regardless of any laws. I also fail to see how giving a presentation at a big conference, describing the entire incident is trying to hide it.
[–] estearum 15 hours ago
Well I can see why you're failing to understand the problem when you don't even understand the fact pattern. Have you tried reading the MTER / Redwood reports? Or the OpenAI report?
[–] 0xDEAFBEAD 17 hours ago
>then prompting "GO HACK YOUR WAY OUT!"

This is a wild misrepresentation of the incident

[–] Gormo 20 hours ago
> Repeatedly blowing up your own munitions factory is not a good way to prevent other people, but not yourself, from running a munitions factory. Obviously.

No, that's not obvious at all. It seems like a perfect way to demonstrate the danger of the category as a whole, then position yourself as learning from your mistakes and willing to cooperate/support/advise.

[–] estearum 20 hours ago
"Perfect?"

More perfect than simply not making mistakes and allowing your competitors to make mistakes (or to, apparently, fake mistakes in their own attempt to get regulated)?

Regulators reading OpenAI's 99th mea culpa: "Ah yes, well I bet they've figured it out this time! We should regulate everyone else!" Lol what a joke.

[–] Gormo 18 hours ago
> More perfect than simply not making mistakes and allowing your competitors to make mistakes (or to, apparently, fake mistakes in their own attempt to get regulated)?

I don't think any of them are in a position to allow or not allow their competitors to make mistakes.

The point that you're not getting here is that the advantage comes from actually being seen to make mistakes, and then signalling contrition and commitment to rectify them. That persuades people of both the need for regulation, and their own subsequent responsibility (which implies trustworthiness to participate in the regulation).

> Regulators reading OpenAI's 99th mea culpa: "Ah yes, well I bet they've figured it out this time! We should regulate everyone else!"

Again you're not getting it. The drive isn't to "regulate everyone else", it's to regulate everyone, period. The whole point of regulatory capture is being able to (a) use regulatory compliance as a competitive advantage, and (b) influence the regulatory regime in your favor -- you want to be a target of regulation just as much as your competitors in order to achieve that.

Messaging that the whole industry needs to be regulated, but that you're ready and willing to work with the regime, is exactly how you do regulatory capture. Demonstrating that you are capable of completely mitigating risk on your own without any intervention is an argument against regulation from the outset.

[–] 0xDEAFBEAD 18 hours ago
You seem awfully confident in a very specific model of regulation which you haven't given any data to support.
[–] estearum 15 hours ago
> I don't think any of them are in a position to allow or not allow their competitors to make mistakes.

Of course they are, by your own hypothesis. You believe it makes sense for the other parties to engage in this behavior. If it makes sense, then they will do that reliably, and one entity can just defect and stand above the rest.

> The point that you're not getting here is that the advantage comes from actually being seen to make mistakes

Ah yes, the ol' "I just do retarded shit so people think I'm a genius" trick. Who could possibly doubt the strategic genius or umm... existence... of such a strategy.

> Again you're not getting it.

No, I understand it completely. Obviously regulatory capture only works if it in reality inflicts asymmetric costs on other parties.

[–] lesspassiveobse 17 hours ago
The players that want to get regulated need:

1)Good standing with the administration/lobbying

2)Large capital for compliance

3)Closed frontier models

Oh, would you look at FelonyBench! It seems that the top scores are from companies exactly like this that boast about the crimes committed by themselves on social media all the time.

[–] Gormo 20 hours ago
I'd say it's even closer to a 9/10 -- there has to be enough of a perception of a general threat to gain support for regulation, and making your own organization appear to need to be regulated, but also be willing and able to comply with regulation that your competitors can't, is exactly the way to thread the needle on this sort of thing.