Hacker News
points by angry_octet 20 hours ago | 0 comments
Why is their security model two VMs on the same hypervisor? Because they wanted to write a story about the models discovering a hidden channel. Hidden channels on shared hardware are actually well known and understood, completely unsurprising.

If they were serious about not having side channels, they could first ensure that they had disabled hyperthreading, disable scheduling on the same CPU cores, and memory overcommit. With NUMA controls they could also pin which memory is used by each VM and eliminate sharing.

These techniques are routinely used for low latency and real time code.

Even better, run jobs on separate nodes, and boot fresh images onto them for each run. Provision separate VLANs for each run.

A next level is to switch lambdas. Use a physical layer switch like a Glimmerglass MEMS switch or a fiber patching robot like XENOptics. No electrical connection between machines.

There is still unfortunately the firmware of other microprocessors on servers, especially x86, so you still want VMs and containers to reduce the chances of ring -2 access (SMM) and persistence via writing SPI flash. Maybe use RISC-V or ARM to avoid this. (NVIDIA cards use RISC-V and ARM extensively.)

[–] pianopatrick 20 hours ago
Or just do these training runs for frontier models in a special data center inside a faraday cage under a mountain. Do not rely on vLANs or hypervisors or VMs or containers or any other software tool. Software is soft.

Just set things up so there is physically no connection to the rest of the world.

Hacker News