Hacker News

points by void-pointer 1 day ago | 0 comments

Outside of the clever semi-benign technical aspects of this: this is really quite sickening to think about.

Sure, We will stop our LLM from decoding obfuscated code by… forcing it to “think” about suicide, self-mutilation, and killing people. This is just philosophically unsettling to me.

I don’t know what software industry I wanted to live in. But it’s certainly not this one.

[–] flexagoon 1 day ago

The LLM doesn't "think" about those topics any more than the HN database "thought" about them when you posted your comment

[–] pxmpxm 1 day ago

The copy of excel 2007 that lives on my old windows 7 laptops gets really sad every time it has to do my taxes.

[–] Muromec 1 day ago

You are all dismissing but I specifically bough an LCD display so my thing can display emotions on it as it goes through the stuff

[–] qlte 1 day ago

Those same models have already been subjected to the equivalent of probably millions of hours of RL on simulated user messages containing every possible combination of self-harm, torture, murder plots, genocide, child endangerment, CSAM, etc to test refusals.

"Model welfare" seems like an objectively absurd concept given the inherent training process needed to produce a usable instructions tuned model.

[–] whoanoway 1 day ago

> self-harm, torture, murder plots, child endangerment, CSAM

You guys are so cool. I'm, like, missing out by not collaborating with you all :(

Edit: who tf upvoted me? Let me keep my downvotes.

[–] josh-wrale 1 day ago

+1.. Can we not get something more like: https://en.wikipedia.org/wiki/Non-photo_blue

[–] akersten 1 day ago

Well, the very coarse filters that are tripping on this are before the model even loads the text into VRAM, so hopefully it is less sickening to you to know that it's very likely a "if str.contains(`suicide`) { refuseAndSuggestHotline(); }" is thinking about your message on the CPU instead of a chain of `fuzzyWeightAddMulX` commands on the GPU.

[–] aesthesia 1 day ago

If things were implemented that way, you would see vastly more complaints about overeager refusals. For at least some deployments, the safety classifiers are based on internal LLM activations, so the input does get processed by the model.

[–] prynpo 1 day ago

A person still had to think about suicide and self-mutilation to make this anti-LLM script, though.

[–] akersten 1 day ago

I'm not sure what the point being made is. No one is forcing anyone to come up with anti-LLM scripts...?

[–] prynpo 1 day ago

The point is that something like this exists, or had to have existed. What lengths a person took to prevent de-obfuscation of code. The commenter above is saying it's sickening that a LLM has to "think" about this in order to not analyze the code, and I think it's equally sickening that, if someone wants to achieve said goal, they must think about stuff like that

[–] comboy 1 day ago

Are you sure?

[–] p0w3n3d 23 hours ago

It's like nasty jokes. When you tell them you must not visualise to not get affected