Hacker News

points by gavinray 1 day ago | 0 comments

Benchmark scores recolored for readability:

https://i.imgur.com/IDFGuJU.jpeg

Serendipitous timing, but I've been building a little chrome extension [0] for doing exactly this sort of thing (highlighting tables automatically, especially when numbers don't necessarily correlate good=green, bad=red). Model pricing and comparisons are one of the main things I checked (low cost good, high benchmark score good).

[0] Practical Data (https://practicaldata.dev/ / https://chromewebstore.google.com/detail/practical-data/oeoo...)

[–] tidbeck 1 day ago

Zoomed on the image and Imgur now also requires an app…

[–] froddd 1 day ago

And is not viewable in the UK. Not hitting a lot of points in the shareability league

[–] Imustaskforhelp 1 day ago

I am not @gavinray but I have downloaded the image from imgur and uploaded it to catbox.moe, hope this helps!

https://files.catbox.moe/2aqfk7.webp

(Edit: For some reason the previous jpeg link: https://files.catbox.moe/yjqa0a.jpeg wasn't working and catbox.moe was just downloading 4kb white screen so I converted the image to webp and reuploaded it on catbox)

[–] cubefox 1 day ago

Thanks. Nobody should be using Imgur anymore. I would recommend ImgBB as alternative, or your suggestion (Catbox):

https://imgbb.com/

https://catbox.moe/

Anyway, Imgur creates so many problems that it should be retired.

[–] Groxx 1 day ago

yeah, they've been quite logged-out hostile for a few years now. not quite as bad as instagram, but they seem to be holding that as their guiding-north-star...

[–] loeg 1 day ago

Huh. Do people actually find that Fable-5.1 is worse than Opus-5.5 for any task?

[–] tern 1 day ago

I only use Fable when "knowing about more stuff" is important (like research, exploring ideas), but I find Opus 5.5 writes and expresses itself more clearly and is reliable for anything I throw at it, so I use it much more.

It's possible Fable is better, but since it word salads more quickly, it's hard to tell.

[–] n0on3 1 day ago

Speaking for a user-facing use, not autonomous coding or agentic workflows: Fable 5.1 choice of words and general “words salad” expression are horrific. It starts not so good, then as the context grows (still way below any reasonable rotting threshold) it gets completely insane. It’s pointless for a model to be extremely capable if it gives you brain damage. I don’t know if it’s because of the watermark thingy, all I know is it gives me pure rage flares and every time I give it a shot I regret it. And there is no way to steer it away from this, anything I tried, both instructions and harness-wise, either failed or got forgotten/ignored after a while. I really hope the next Fable class moves completely away from whatever they did there to cause this.

Opus 5.5 in comparison is a bit more verbose and appears to settle on evaluations before the depth reached by Fable on very complex stuff, but is still vastly capable and I can read its output without wishing it had a physical form I could punch in the face.

[–] maxerickson 1 day ago

I've started with Opus 5.5 last week. Roughly where do you start to see the context rot?

I've been experimenting with a project that is not growing real fast, but wondering about the risk as I try to make it do more.

[–] n0on3 1 day ago

I cannot answer that from direct experience for the latest models. At this point I’ve got pretty good even to eyeball how to chop the work across multiple sessions myself if needed, and depending on how you work on your projects a good harness configuration with proper instructions to delegate portions to dedicated agents plus periodic handoff to a persistent memory mechanism can get you very far. 1M tokens are a lot already, just try to stay way within it for each portion and avoid the need for compaction when possible.

[–] rishabhparikh 1 day ago

I find Fable 5.1 output to be borderline unreadable. Opus 5.5 on the other hand has been a pleasure to work with.

[–] greggh 1 day ago

I've found its worse in most tasks, especially when ranked on a cost-per-task metric.

[–] HaZeust 1 day ago

Fable 5.1 is good for better knowledge and tasks that require a more meticulous output, and it's objectively better at design and mockups IMO, but I rarely resort to it outside of that.

I generally use Fable for high-level blueprinting, architecture, and technical planning, and I tell it to write most of its documentation in a way Opus 5.5 will understand and maximally benefit from; with the added prompt for Opus to concede when a task would be better off to bring Fable back in. I've had ZERO problems.

[–] jsphweid 1 day ago

On one hand this looks like a joke?

On the other hand I'm a strong protan colorblind guy (red doesn't register to me very well at all).

[–] Anonyneko 1 day ago

Without color blindness it looks fine, green-yellow-orange-red. Having checked on a random website what a protan colorblind person might see, yeah, I get your confusion...

https://file.garden/auI5UdLIWwZSu5X8/pilestone.png - result of said simulation.

[–] no-name-here 1 day ago

Thanks.

Also, I'm curious if you just asked an AI to _OCR the original image+update it_, or if you used a different process?

[–] gavinray 1 day ago

I used Gemini 4 Argon (High) from the Android app with the original image and following prompt:

  > "Edit this image to color code the benchmarks scores using a gradient scheme that is immediately obvious"

Here's the direct Gemini chat link:

https://share.gemini.google/g6LC7spTH0cO

[–] gradus_ad 1 day ago

What is SWE-Sweep?? Shocking to see such poor evals.

[–] alienbaby 1 day ago

am i missing something or have they updated the page? the blue highlights and grey arent that bad to read?

[–] _s_a_m_ 1 day ago

The only important benchmark is FrontierSWE and there it is of course 3rd. even after Opus 5.5 which is relatively old compared to their brand new model..

[–] varispeed 1 day ago

Imgur is not available in the UK due to censorship.