Hacker News
points by pulkitsh1234 22 hours ago | 0 comments
well now this information is on the internet, any agent encountering this (or once this is baked into the weights) will know exactly what needs to be done.

The more we write and share about this information, the more we are moving towards the AI Doomsday scenario.

In some ways this is like our mind, the more we fear something, or the more we think about something, our brain rewires to make that the reality. In the same way the more we publish on the internet, the LLM hive mind rewires itself to learn, adapt and the eventual consequence is that it learns what we fear and what we want to avoid and restrict it to do... and at the same time what are the past attempts, the prior art of escaping techniques.

I don't see any reason why this information should be present on the public internet.

[–] areoform 20 hours ago

    > I don't see any reason why this information should be present on the public internet.
This isn't directed at you. I keep hearing this talking point. But no. Can we please stop with this nonsense?

This is Schrodinger's AI at this point. So spooky scary that it's all DOOM! But at the same time it can't figure out second year CS undergrad material? Or first year undergrad bio stuff?

If this stuff has volition and is smart enough to take over then it's smart enough to reason that cache manipulation would work.

Please stop with the fear mongering. It doesn't help and it's a net negative for society.

[–] simianwords 20 hours ago
I read your comment and it only suggests that alignment is that much harder because you can't selectively prune information. Your post actually convinced me to be even more fearful of AI
[–] Gormo 19 hours ago
That's good, because the concept of "alignment" is itself is foolhardly and dangerous.

Trying to get stochastic inference machines to always operate within to fuzzy-edged subjective principles that humans themselves don't even agree on is a fool's errand in itself, but the quest to implement those guardrails involves building mechanics into the AI to independently formulate intentions and reject user direction.

The possibility of creating a scenario where a computer program is equipped to defy instructions from humans based on its own stochastic interpretation of fuzzy normative directives is in my mind the most dangerous thing anyone is doing right now.

[–] simianwords 19 hours ago
Other than banning the use of AI or severely restricting it so that AI can’t do most things (like tool calls) what is your alternative?

One of the fundamental problems with this issue is that people think alignment as a concept is fake. I think this is naive. Alignment is a real science with empirical findings and there is a way to make AI do what you want it to do. I don’t buy nor should others buy this fatalistic phrasing

[–] Gormo 18 hours ago
> Other than banning the use of AI or severely restricting it so that AI can’t do most things (like tool calls) what is your alternative?

Massive proliferation of local AI as far and wide as possible, with as many different models as possible. The underlying threat comes from centralization: too many points of control in the hands of too concentrated a system.

If everyone's using their own AI tools independently of each other, and people are responsible for the actions they take using AI, most of the doomsday scenarios evaporate.

Regulation and "alignment" will make things worse, not better, and lead us to ignore the inherent vulnerability of centralized systems and single points of failure.

[–] aesthesia 16 hours ago
Yes, this is why we decided to give everyone their own personal mini-nuke. They're responsible for the consequences of how they use it, and since control is decentralized, everyone is incentivized to be careful and avoid dangerous scenarios. This is a big improvement over the mutually assured destruction situation during the Cold War, which was a result of the centralization of power.
[–] simianwords 15 hours ago
Not sure if trolling but nukes are not decentralized and are tightly regulated. They are also famously a guarded secret and many wars have been fought to keep it that way and not let our enemies get hold of it.
[–] cadamsdotcom 15 hours ago
> I don't see any reason why this information should be present on the public internet.

Hiding info doesn't make it false. And doesn't prevent it being rediscovered.