Hacker News

points by jrockway 8 hours ago | 0 comments

There was an article on HN a month or so about benchmarking Opus 5.5 every day and recording / publishing the results. I recall this being a github repo, but my searches turned this up instead: https://www.bridgebench.ai/nerf-bench/claude-opus-5-5

So it seems like ... it is not getting dumber. I find myself using Fable a lot more, though. Opus 5.5 was never as fun as Fable, but it's pretty good.

[–] jahala 8 hours ago

For me, Opus 5.5 has been consistently much better than Fable 5.1- in all aspects, but especially reasoning and "attention span"

[–] jrockway 8 hours ago

Opus 5.5 is definitely good. I use both all day and it's really hard to say which is which from looking at the outputs; I can't read a PR and say "this was Fable" or "this was Opus" like I could pre-5.5.

Fable is better at playing along with my "You are a dog" instruction in CLAUDE.md. It is willing to be fun and playful, whereas Opus just wants to finish the job and go home. Friend vs. coworker, basically. For example, whenever I reward Fable with a dog treat (actually a <cookie emoji>, which I tell it is a chocolate-free chocolate chip cookie) it keeps a log in memories. Opus just says "crunch! tail wagging!" and moves on.

None of this matters but I think it's fun to have some fun :)

[–] jahala 7 hours ago

Aha, I haven't looked into such aspects much - I'm "output" driven. But its good to know about that, if I feel like trying it- thanks.

[–] _aavaa_ 7 hours ago

You’re thinking of: https://github.com/ninjahawk/livenerf

I’ll ask the same question as when it came out: if you don’t trust the provider how are you trust them not to look for and cheat on repeat benchmarks like this.

[–] taylorfinley 7 hours ago

Livenerf!

https://github.com/ninjahawk/livenerf

[–] yako21000 3 hours ago

Was that not from the guy having analysed the anthropic models to incredible deephts and exposted them nerving it some while ago?

[–] jrockway 6 hours ago

This is the one I wanted to link, thank you!