Hacker News
points by mattlondon 4 days ago | 0 comments
I think, despite the desperate attempt at IBM to remain relevant (and in business), their CEO has a point here that do you really need SOTA models for everything?

I had a really nice time with GLM5.2 for a month or two - it was, in my experience, approx Opus 4.5ish level of capability when I think Opus 4.8 was the new hotness

Even today, I use Gemini 3.8 Flash as my daily driver for most coding. Its fast and good, even if it is not the absolute SOTA at coding benchmarks it really is very good. Yes I have tried Opus 5.5 and 4 Argon etc and yes the are better, but do I need that?

I think that we are going to be at the stage soon-enough where models are "good enough" for 99% of people's needs. The SOTA will keep on marching ever-onwards I am sure, but I would not be surprised if people stick with previous-gen/previous-previous-gen/etc models because why upgrade?

If this does play out like this, I do wonder how that works out for the frontier labs (specifically OpenAI and Anthropic, less so Meta & Google) - if open models or even just cheaper Sonnet-level models from a few generations ago are good enough for basically everyone's daily work, where is the revenue? how can you justify pumping billions and billions into new generation models and datacenters etc when there is no driving need to use them?

[–] roncesvalles 4 days ago
>do you really need SOTA models for everything?

For work in which you are in the loop, absolutely. Do people not value their own time or something? Why in the world would you use less than the best LLM available to you? You're not going to run out of the $20 plan even if you use Opus 5.5 at xhigh and code 8 hours/day (assuming you're doing it piecewise using a Claude product) so what exactly are you conserving by using something inferior?

For large continuous volumes of programmatically submitted LLM tasks, sure.

[–] hhh 4 days ago
I have single sentence prompts in one of my legacy codebases that it takes Opus 5.5 over $1000 in API spend to return an answer correctly on. I am not getting by with a $20 plan.
[–] roncesvalles 4 days ago
That's ~41 million tokens. Your codebase better be the size of the entire MySQL codebase.
[–] hhh 3 days ago
Similar size.
[–] mattlondon 4 days ago
That's my point - do you need the best available to you? What if tomorrow your employer said no more 5.5, or Anthropic IPOs and suddenly investors want to see a profit and that 20USD a month for Opus 5.5 becomes 20USD per-day or even per-hour? We're already seeing corps restrict their employees token usage.

Could your survive on say Opus 4.8? Or GLM 5.3?

I bet you could if you had to.

I think it's the same argument about cars. Why wouldn't you want a Ferrari? Do you not value your time? Why not go as fast as physically possible?! You need the absolute fastest, most-capable, best-handling car for every single journey you take! Or... Back in reality not every journey you take is trying to lap the track as quickly as possible and just any reasonable car will get you where you need to go.

This is my point. Will Anthropic and OpenAI continue to exist selling Ferraris when actually people realise they can get by with a Toyota Corolla 99% of the time? Ferrari exists in the car world so I guess there is an argument for "yes". I am not sure myself - Ferraris are after all rare and exotic cars ... I suspect that even if we are not there now, we are already seeing current SOTA models that are mostly equivalent/better than the average engineer - another generation or two they'll be waaaaay better than the average engineer. How much more car do you need, and how much are you prepared to pay for it?

At least for me, I don't like using Opus or Fable because I don't like the way it talks to me (talking down to me, refusals, moral high-grounding, overly pomposity etc). 3.8 Flash is currently my favourite model due to how it communicates and works with me and the speed is much better suited for rapid iteration. So put me in the comfortable, reliable, use-it-every-day-to-get-shit-done-with-no-fuss Toyota Corolla camp. Its good enough for me and I don't get any benefits in my day to day life by using Opus 5.5 and it's sanctimonious personality.

[–] ecshafer 4 days ago
Currently the difference in my work between the best models and okay models is pretty night and day to the point its not worth using the older models. Now I haven't used the newest top open source models. But if I give the same code base to say Qwen 3.6 and GPT 5.6, Qwen will analyze the source of the problems wrong, provide actually wrong solutions, write code that takes 100x as long to run. Its not even worth it outside of querying for snippets of code. GPT 5.6 will help actually root cause the problems.
[–] mattlondon 2 days ago
Ah but if the price is 10x - as the IBM CEO suggests - your employer might just ask you to use that, and then expect you to make up the gap between the cheap and expensive models using your own brain instead of just pissing money up the wall on high token costs AND your salary. You know, like the old days when we actually had to do the work ourselves.

I think this is a very high likelihood scenario.

[–] EasyMark 3 days ago
I think the point is that you are not everyone. Not everyone has your requirements, sounds like paying the premium is worth it for you, but others simply don't need it.
[–] jaccola 4 days ago
I love 3.8 Flash, if only it weren’t unusably flaky. Sometimes simple responses take minutes or I get rate limited after sending a single message (I guess it’s a shared pool rate limit). Have you not experienced this (would love to know if there’s a way to actually use it reliably!)
[–] jhrmnn 4 days ago
I've personally experienced three big jumps in LLM capability: o3, opus 4.6, and opus 5.5. I could go back to opus 4.6, but it did require way more involvement from my end.
[–] spwa4 4 days ago
What subscription do you use for Gemini 3.8 flash and how much usage do you get?