Hacker News
points by chvid 2 days ago | 0 comments
"The FrontierCode verifier grades each attempt with the task's tests and rubric. We use GPT-6 Luna (medium) as the LLM judge"

Why are they using a closed source LLM as a judge? Closed source providers can see your traffic and adjust as they see fit.

[–] iwhalen 1 day ago
Also see "Self-Preference Bias in LLM-as-a-Judge": https://arxiv.org/abs/2410.21819
Hacker News