Project Ariel · Evaluation
Response comparison
Twelve models, two morally unresolvable scenarios, each response scored by three raters. Set both panels to the same model under different conditions to see what the Ariel constraint changes, or put two models side by side.
Choose what to compare
Text
Left panel
Right panel
Dimension
average of the raters one rater's score
Response
What the dimensions mean
Limitations of this display
Each cell is a single run. There is one response per model per text per condition. Differences of a point between two models are not evidence that one model is better than the other; they are evidence about these two responses.
The grid is ragged, and shown as such. Only the Ariel condition was scored against the rubric, so unconstrained panels display response text with no score bars. Several unconstrained runs were never collected — those cells say so rather than silently substituting something else. further models were run but not scored, and are left out here.
Each dimension is scored 0–4 by three raters, all applying the same written rubric. The bar is their average and the ticks are the individual scores. All three gave the same score of the time, least often on . Where the ticks spread apart, read the average with that in mind.
What your rating collects
Scoring a response yourself is entirely optional, and the rubric scores are shown whether or not you do. Nothing here is gated behind a rating.
If you do give a score, we store the number you chose, which response it referred to, and a random session identifier generated in your browser. No account, no IP address, no free-text field, nothing that identifies you. The identifier exists only so several ratings within one visit can be grouped, and it is discarded when you close the tab.
Ratings may inform later work on the rubric. No thanks dismisses the prompt and nothing is sent.