People vote on pairs of model responses and they aggregate everyone's votes into a leaderboard.

One thing to be aware of that the distribution of prompts on arena.ai can be pretty different than real-world usage. The people who spend time on that website casting votes tend to be more interested in entertainment than professional productivity. And turn-based voting can punish useful behavior like asking clarifying questions instead of guessing at assumptions.