Hacker News

points by throwaway2037 3 days ago | 0 comments

That is wild. What if there is a huge break-through in LLM models where they can reduce compute by 90%? All of those data centers would suddenly crater (many customers cancel) and go bankrupt. I feel it could go either way.

[–] Silhouette 3 days ago

Surely the thing the AI giants fear the most in this respect must be the possibility that competitive open models and locally available hardware reach the point of viability for most people. If that happens then where is the moat for the giants?

If progress with frontier models doesn't continue at the same pace then the investments and funny money deals might start to look very fragile.

If progress does continue then that progress might be towards smaller models more than quantum steps up in quality or capability that would increase the risk of the scenario above.

All of this is set against a backdrop of public hostility and political problems that I can only imagine increasing in the near term as well.

This feels like an unusually high stakes game of poker. I don't think I'd want to bet on Big AI right now if I had a few billion lying around. Then again - maybe that's why I don't have a few billion lying around!

[–] BlobberSnobber 3 days ago

Well, they are ensuring that hardware prices stay extremely high by buying all the flash memory at a premium, so at least they've got that aspect kinda under control?

[–] Silhouette 2 days ago

I've made that same point myself in the past. It's more of an accidental moat though - and not one they can defend for very long if they're no longer getting value out of those components themselves.

I saw an interesting observation the other day that Jensen Huang has played this game the best because his company is the one actually selling real products and making real money in return. If all the AI giants are buying up huge amounts of hardware speculatively because they assume they will need those data centres in the future and then the bottom falls out of the market then it's the AI companies left holding the baby. And if we do get to a point where local models are competitive then the OpenAIs and Anthropics of the industry might face real difficulties but it would just be a different market for Nvidia to also be selling real hardware and still in exchange for real money.

[–] mike_hearn 3 days ago

That's already happened. It's one reason prices have dropped so much lately (n.b. in the quarter that's not in their S-1, I think!) They've been able to improve model efficiency a lot through better algorithms and training models to be less verbose.

[–] vlovich123 2 days ago

Training costs haven’t plummeted like they have for inference afaik and the efficiencies are just translated into running larger and longer training. Basically Jevon’s paradox in action just like on the inference side.

[–] edmundsauto 1 day ago

I am projecting but I think these companies are looking at their output (“intelligence”) as electricity. There’s never too much electricity, really - it just goes into lower margin uses.

For intelligence, I bet if inference got 90% more efficient they would just increase the “thinking” by 90%.

[–] acchow 2 days ago

We’ve already seen 90% reduction in intelligence costs multiple times over the last few years. Each time, token consumption rises 20-100x

[–] pierreb-aiva 2 days ago

Jevon’s paradox would apply. If I can use order(s) of magnitude more AI for the same price, I would absolutely do that. It would also unlock many use cases that are currently gated by model pricing, which is largely dictated by the price of compute + frontier intelligence markup.

And thats just the inference side of things. For training, the big players like Anthropic would still be able to benefit from cheaper compute, by generating more RL rollouts, running more RL environments, and thus still providing frontier intelligence that would justify the markup.

Many customers cancelling would only be a risk if the compute expenditure was more decentralized. But its not