Hacker News
points by cactusplant7374 4 days ago | 0 comments
The moat is high TPS. No one has rolled it to average consumers though.
[–] vikramkr 4 days ago
Tokens per second? That only really matters for things like chat and sessions where you are monitoring a session as it runs which is going to be an increasingly small percentage of usage. That's bottlenecked by human attention. Agents in the background, long running tasks, unattended tasks, all that stuff is what is even capable of driving significant usage because you can just increase consumption massively by parallelizing without being the bottleneck. I wouldn't be shocked if already the vast majority of token usage was from unattended sessions. TPS is not that important there
[–] imtringued 3 days ago
You're ignoring the fact that a waiting human costs money which is actually the primary reason why AI hasn't resulted in significant productivity increases, while overly focusing on the human being the bottleneck. The point isn't to monitor the session, the point is to verify the output with human inspection. If the agent is fast enough, attending to the agent can make financial sense.

Human memory has a finite capacity and decays over time. If you do many inspection tasks in parallel you are splitting that finite memory. If you do the same tasks sequentially, the human can give instant feedback to the model without costly context switching. This in turn increases iteration speed and reduces total labor costs. This is no different from developers preferring JavaScript or Python for immediate feedback and fast iterations.

[–] cactusplant7374 4 days ago
It is impossible to make sophisticated software products at a high level with a low TPS. Try making a modern emulator or an award winning game. Both require a lot of iteration and not everything works well with multi agent.
[–] gmerc 4 days ago
DeepSeek is serving more TPS than anyone, about as much as OpenRouter does in a week in a day, at 38$/H800/Ascend so about 900k/~90% profit over 3 years (or ~80s% profit including training expenses).

Meanwhile, Oracle is handing Tencent 100.000 GPUs at 1.60$ hour, dangerously close to breakeven...

What does this tell you?