points by torben-friis 3 days ago | 0 comments
I think the main problem is surfacing two topics:
- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)
- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?
If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?
[–] andai 3 days ago
>effort / performance curves by model
Oh, easy:
Artificial Analysis has their own curves which are pretty similar.
[–] irthomasthomas 3 days ago
That's good, but they haven't updated it for a month. It is missing all of the current frontier models.
[–] aftbit 3 days ago
Seems to be missing all of the 6.0 and 6.1 models?
[–] dist-epoch 3 days ago
> Is a high effort terra above a low effort Sol
You can literally see this in the charts from the article
[–] tedd4u 3 days ago
[–] someguyiguess 3 days ago