Hacker News
points by agairola 1 day ago | 0 comments
on moderation, we run an open guardrail benchmark at raxIT Labs and the hosted d1 models do well. d1.1 is #1 of 30 at checking model output for harmful content (94.4 balanced accuracy), d1-ultra is #1 on harmful user requests (88.2). we haven't run the new 2B open weights yet so no number for those. results and method: https://dm-as-guardrails.raxitlabs.com
[–] CMay 1 day ago
This tracks with my experience with their models. Nice work. It would be interesting to see what they could do if they optimized even more for moderation like this without going so far as to hurt the model performance overall.
Hacker News