I personally believe that local LLMs have a huge captive market for people who cannot or should not host that data with a third-party. For example, it's an open question of whether or not using cloud LLMs by lawyers breaks attorney-client privilege. HIPAA with medical records is another one.
But I want to expound on one point that you made, which is really important. You say that this spec only has a SOTA life of ~1.5 years and that's an issue. Take that to its natural conclusion. Why isn't it an issue for hyperscalar data centers, particularly as NVidia wants to move to annual chip updates.
I believe the hardware is going to be commoditized. At that point the only thing that matters is performance-per-Watt. NVidia's R100/200 doesn't make the B100/200 obsolete but if the R series gets 50-100% PPW more and these DCs are using gigawatts of power, that's a massive difference.
What really is the effective life of AI hardware?
What on earth is a "natural conclusion" in this context? I'm going to assume that part of your comment was written by AI.
> Why isn't it an issue for hyperscalar data centers, particularly as NVidia wants to move to annual chip updates.
Because he's not a hyperscaler? He's not operating data centers where compute is fungible, he's buying an individual machine that he wants to use for years. He can't just keep buying a new machine and fob off the old one to someone else's less demanding workload.