Hacker News

points by babelfish 3 days ago | 0 comments

Taking the benchmarks in the article at face value, this is ~on par with Opus 5.5 for ~1/4 the price!

[–] mwigdahl 3 days ago

Which is why I don't take the benchmarks at face value...

[–] babelfish 3 days ago

So you have decided the model is bad without having used it yet?

[–] cbg0 3 days ago

If they had a model on par with Opus 5.5 there would be no need to undercut it by that much, that's probably the logic.

[–] rfgplk 3 days ago

Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x.

[–] mwigdahl 3 days ago

No, I have decided I don't trust benchmarks without using the model first.

[–] entrope 3 days ago

Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark

[–] enraged_camel 3 days ago

...no it's not? At max effort, it scores 50% on terminal bench vs. Opus 5.5's 70%.

[–] babelfish 3 days ago

ALE (top benchmark in post) is 60% for Terra vs 63% for Opus - I guess we'll have to wait for AA Index!

[–] saretup 3 days ago

48 for GPT-6.1 Terra Max vs 58 for Opus 5.5 Max on AA Index