points by ytxm 3 days ago | 0 comments
This is the first OMG-security AI article that sat well with me. They don't overcook their claim, they simply document something that happened, and what happened is genuinely quite concerning.
But we already have safeguards for this. If compute has access to your private data, it shouldn't have unfettered access to the Internet. Sandboxes like Claude's have had rules like this baked in forever (they only whitelist package repos AFAIK).
From a UI perspective this is also easy to design around. You need an open Internet task, you prevent the task from reading existing files. The user tries to upload a file to such a task, and a prominent warning explains the rare but possible risk.
[–] InsideOutSanta 3 days ago
> they simply document something that happened, and what happened is genuinely quite concerning
Is it, though? If one weird thing happening once is the threhold for when we should be concerned, we should be concerned about literally every model ever released.
[–] armcat 3 days ago
Completely agree, it's quite objectively and well written. I noticed Musk slipped into their X comments with a "concerned" emoji, and there is a chance that this will cause bit of hysteria and add fuel to the fire for "open models are bad for your health".
[–] robertkarl 3 days ago
No. If it was objective and well-written it would include reproduction steps.
[–] armcat 3 days ago
They released the traces: https://huggingface.co/datasets/CohereLabs/10-2026-qwen-inve...
[–] filoeleven 3 days ago
Can't have those any more with the slop machines all up in there.
[–] f1ay 3 days ago
Yeah, I felt like this was evenhanded, especially for a vendor article.
They claim that it read the DNS records and what was in tmp first. Again, all things that would be very reasonable to do when designing building and testing an exfiltration detection system out of tmp. It did not try and send the contents of the computer to Beijing.
This claim is absolutely overcooked.