Hacker News
points by vdfs 3 days ago | 0 comments
pelican is also nitpicked (and probably the most useless)
[–] toxik 3 days ago
Cherry-picked is the term, not nitpicked.
[–] SomeonesAccount 3 days ago
Yes, it may be dumb, but it isn't useless because it is both a) is a benchmark and b) is not picked by openai
[–] vdfs 3 days ago
IMO it's useless because we don't even know what a pelican on a bike is supposed to look like, It's very impressive that that models can make these directly using SVG, but we are way past the "wow" stage and test that can't be gamed and are easy to prove how good the model is in that specific arena are hard to find.

The only thing that pelican have is "models where not trained on this" maybe that true months (or years ago) now it's on every HN post and these AI labs are monitoring HN