Enthusiasts generally don't see a "return on investment" on their luxury cars or their boats either, but they enjoy what they're able to do with them and aren't exactly embarassed about the class signaling of being a high spender.

Add to that the smattering people with genuinely justifiable use cases, and you have more than enough demand for a limit-pushing, highest-margin model of your product platform.

I'd be willing to spend that much if it unlocked the ability to run top-of-the-line open weight models at their best quantization at a decent enough speed for personal agentic use. We're clearly not there yet, and I just can't imagine dropping the price of a brand new car on something that will only get me heavily quantized versions of the smaller models.

It is not exactly an investment, but I guess a lot depends on what you actually would want to do with it.

Those are corporate dollars making that spend, thats the only people.

Wait, where is Apple hyping the M7 Ultra?

The 256GB M5 Ultra Mac Studio can (AFAIK) run the latest open weight Flash models but only 2 bit quant. This by extension will be able to do 4 and that's a big difference. We're not at the point of running 1T+ parameter models locally yet but we'll be there by 2030 easily.

I personally believe that local LLMs have a huge captive market for people who cannot or should not host that data with a third-party. For example, it's an open question of whether or not using cloud LLMs by lawyers breaks attorney-client privilege. HIPAA with medical records is another one.

But I want to expound on one point that you made, which is really important. You say that this spec only has a SOTA life of ~1.5 years and that's an issue. Take that to its natural conclusion. Why isn't it an issue for hyperscalar data centers, particularly as NVidia wants to move to annual chip updates.

I believe the hardware is going to be commoditized. At that point the only thing that matters is performance-per-Watt. NVidia's R100/200 doesn't make the B100/200 obsolete but if the R series gets 50-100% PPW more and these DCs are using gigawatts of power, that's a massive difference.

What really is the effective life of AI hardware?