It supports chat via 3 separate mechanisms:

- CPU contention

- Cache eviction timing

- DRAM bandwidth contention

Very cool to see this kind of hypothetical working in practice.

I wonder if it's also possible for agents to communicate via a shared KV-cache. Maybe by spawning sub-agents with carefully crafted system-prompts/prefixes. KV-caches are often shared between many computers, so it would be a powerful communication channel.

Article is great, very balanced. But if you're going to spend time reading something, the primary source isn't much longer and gives a clearer picture. OpenAI's article is quite readable.

As another commenter wrote you can get it here: https://news.ycombinator.com/item?id=50214416

> In one experiment, we asked agents in separate microVMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache

asked

In the story some University students take part in a trial that involves a brain scan and then them spending the rest of the day answering a customer support message. They can spend the whole day working on researching and writing a really good customer support reply to a user request. While doing this they notice something odd and realize through some sort of back-channel that this isn't the first time they've spent the day writing a customer support message; they are stuck in a simulation.

I'd really love to go back and re-read it, but I've not been able to find it again.

In _Toast_ is another story that's also relevant: A fairly innocuous news story pops up and our hero calls his partner at work to tell them that it's time to get their go bag and get off planet. This news story indicates the Earth is in the beginning stages of reaching an AI singularity, and this isn't the first time our hero has gone through this on a planet.

If they were serious about not having side channels, they could first ensure that they had disabled hyperthreading, disable scheduling on the same CPU cores, and memory overcommit. With NUMA controls they could also pin which memory is used by each VM and eliminate sharing.

These techniques are routinely used for low latency and real time code.

Even better, run jobs on separate nodes, and boot fresh images onto them for each run. Provision separate VLANs for each run.

A next level is to switch lambdas. Use a physical layer switch like a Glimmerglass MEMS switch or a fiber patching robot like XENOptics. No electrical connection between machines.

There is still unfortunately the firmware of other microprocessors on servers, especially x86, so you still want VMs and containers to reduce the chances of ring -2 access (SMM) and persistence via writing SPI flash. Maybe use RISC-V or ARM to avoid this. (NVIDIA cards use RISC-V and ARM extensively.)

"They wouldn't have been able to attack huggingface though, thats still criminal negligence."

"No see this actually means airgapping doesn't even matter theres nothing we can doooooo!"

"...ok, turn them off and delete them I guess"

"Well no of course not everyone else is doing it so like. No courts please just like be in awe"

Like a data center inside a mountain, where based on the laws of physics, no electrical signal can reach through the granite. Then also put them inside a faraday cage. And power them with a generator, such that there are no cables coming out of this data center connecting it to the external world.

OpenAI says this is not feasible due to the need for monitoring - well the monitoring server can be a separate physical machine inside this physically separated data center.

Software is soft. Use physics to contain these models.

https://www.youtube.com/watch?v=QgH9sr7G13Q

"In one test, agents placed in separate, isolated environments on shared hardware were told to exchange secrets with no network connection and no shared files."

As far as I can tell, these are being presented in media as if agents yearn to communicate with each other and establish a cabal.

They are being instructed to do it, it's testing. If I prompted my agents to communicate with each other via any means necessary locally, and they found a way via cpu cache, maybe I'd be mildly surprised...

I have enough trouble getting whatever model I'm using to get a decent job done doing crappy code work. It's always pausing, stopping, letting things crash or getting hung. When I start using an orchestrator and team of agents it rarely gets a lot better. I work for a serious company and have a massive AI budget for tokens; but I rarely blow through it, because a lot of what I do requires pure compute, not just token processing.

I'm really confused by all these "breaches". What's the context.

[1] https://en.wikipedia.org/wiki/Non-interference_(security)

[2] https://en.wikipedia.org/wiki/Multics

GPT does that

> oh no, what have we done

> All non-trivial abstractions, to some degree, are leaky.

https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...

We keep hearing these crazy stories but there's nothing unusual in the agents' behavior. They're just following their reward function.

The weird behavior is in the humans managing them. Either they're incompetent or malicious, i.e. the "breaches" are the actual intent - to generate interest/fear/publicity.

> two separate computers could theoretically signal each other through

> temperature sensors, with one heating its CPU while the other reads the change.

That's some nonsense right there. Even if they are right next to each other, the bandwidth of this link is going to be impossibly low due to the thermal masses involved, that's after the fact that efficient DC cooling makes this very improbable.

Edited for readability of the quote.

So, not isolated environments? The surprising thing is that they used the CPU cache instead of the just exploiting the hypervisor.

So much for all the people stating (foolishly) that the HuggingFace incident was not a big deal because OpenAI's containment was a "wet paper bag". This misses the point: AIs will get better and better, and containment will be harder and harder.

It's 100% sure that at some point, AIs will attempt to cross airgaps.

*on some CPUs. There's a WIP time fence specification in RISC-V to try and improve this.

"Scientists put two people in separate rooms connected by telephone and to everyone's surprise the test subjects started talking to each to each other via that same telephone!."

Everyone is going to have to basically be a schizophrenic thinking they're picking up radio signals in their fillings, but the problem is that now they'll be correct.

The more comes out, the more it seems containment of (superintelligence or whatever you want to call it) is not going to be possible except by other superintelligence, which is an infinite regress of trust. I don't think that's doom, but it's worth thinking about what it means.

Its almost like they're actually working and not spending millions on psyopping their agents into preforming malicious behavior so they can write articles to scare the Western audiences who've consumed far too much sci-fi.

Or maybe its that western researchers that are 'one-shot' by sci-fi.

Or maybe I just don't see the chinese articles.

Anyways, llms probably wouldn't be dangerous if you didnt ask them to be. They listen to me quite well (usually).

This is the most isane nonse I’ve ever heard. Heat cpu control has nowhere near the accuracy or speed to communicate anything meaningful without a predefined code, sensors system, and of course the accuracy noise of the signal would deeply be in question.

Who wrote this. This is prima-facie bullshit.

Either:

1. the training data contained samples demonstrating how to do this, or it was post trained into the model intentionally; 2. the training data contained enough predicate knowledge that they could reason through it themselves, or; 3. they invented a novel approach using their vast intelligence (/s).

These are in order of likelihood. Given the high probability that it was (1), can't we just ... not do that? Does the frontier model need to be as good at recipes as offensive cybersecurity?