points by enraged_camel 3 days ago | 0 comments
People keep asking OpenAI for a mode that's as good as Opus 5.5, and they release this instead. Interesting strategy. Let's see how it works out for them.
[–] cloverich 3 days ago
[–] laurels-marts 3 days ago
There’s also a chance they surrender the dev market altogether to Ant and just focus on the “normie” segment with dots, computer use, excel etc. In which case they won’t have anything competitive to opus-5.5 anytime soon for SWE.
[–] user43928 3 days ago
This release is interesting for enterprise at API rates.
At least unless the Terminal Bench 4.0 score is more indicative than the other ones, because that one shows it clearly worse than 6.1 Sol for both performance and cost.
[–] rockdoe 3 days ago
Luna and Sol-6.1 are crazy cost effective and all flows here have switched to them, so I'd say really well so far.
There's very few stuff that needs Astra/Opus level intelligence.
Luna 6.1 would be amazing.
[–] babelfish 3 days ago
Taking the benchmarks in the article at face value, this is ~on par with Opus 5.5 for ~1/4 the price!
[–] mwigdahl 3 days ago
Which is why I don't take the benchmarks at face value...
[–] babelfish 3 days ago
So you have decided the model is bad without having used it yet?
[–] cbg0 3 days ago
If they had a model on par with Opus 5.5 there would be no need to undercut it by that much, that's probably the logic.
[–] rfgplk 3 days ago
Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x.
[–] mwigdahl 3 days ago
No, I have decided I don't trust benchmarks without using the model first.
[–] entrope 3 days ago
Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark
[–] enraged_camel 3 days ago
...no it's not? At max effort, it scores 50% on terminal bench vs. Opus 5.5's 70%.
[–] babelfish 3 days ago
ALE (top benchmark in post) is 60% for Terra vs 63% for Opus - I guess we'll have to wait for AA Index!
[–] saretup 3 days ago
48 for GPT-6.1 Terra Max vs 58 for Opus 5.5 Max on AA Index
[–] hansonkd 3 days ago
I hate Opus 5.5, constant overreach and over-engineering for any project
[–] Silagi 3 days ago
That's wild to say when comparing to the OpenAI family of models. I've never seen so many tests that validate tests that validate tests.
[–] tamimio 3 days ago
Right?! I think sol has some obsessive tendencies, i literally caught it the other day spawning a sub sub agent doing a final fourth test for a piece of code that isn’t even critical say in embedded or similar, now I have to instruct it to only test once!
[–] cesarvarela 3 days ago
Medium or High? Depending on the task, you might want to switch between them.
[–] petesergeant 3 days ago
You’re holding it wrong.
My expectation is Fable 5.5 is next overall improvement, and my bet is OpenAI's "Bel" will be the next competitive one (or, their lack of safety culture will bite them/us in the process).