points by djelu 4 days ago | 0 comments
Considering the disclosed numbers for DeepInfer and the usefulness/popularity of DeepSeek 4.1 Flash, I expect it is inevitable that more companies are going to consider the self-hosting option seriously, and perhaps play with the idea of offering inference as a vendor.
If the vendor price is calibrated to a ten-month hardware payback, then at equal utilization there is no arbitrage in self-hosting: the crossover is roughly whether your sustained API spend exceeds the hardware plus that ten months. DeepSeek's own disclosed numbers give the benchmark to beat: 226.75 H800 nodes averaged over a day (~1,814 GPUs), 608B input tokens/day of which 56.3% hit the KV cache, 168B output tokens/day, ~73.7k tok/s input per node while prefilling and ~14.8k tok/s output while decoding.
That utilization is the hard part, and it is roughly what the IBM CEO is pointing at with "where are my kernel engineers". Self-hosting open weights on two or three racks is not the problem; amortizing them at a frontier lab's duty cycle is.