i

Generated screens, measured

commissioned · 0 rows

A generator will make a competent screen in a minute. This room is for finding out what "competent" survives contact with a number.

Why it is empty

Nothing here was generated by a pipeline, and nothing here may be. A drawn approximation of what a generator produces would be a claim about generators wearing the clothes of evidence about them — which is the failure this estate has already made once, at a cost of six hundred invented records.

It is also the one dataset here that gets worse with age. A row is a fact about one tool on one day, not a fact about generators. The models change quarterly. So the capture belongs close to the teaching, with whatever is actually in people's hands, and every row carries the date it was made. The plan is to recapture annually and throw the old rows away rather than average them in.

The schema, the capture script and the rules are written and working — pipeline/capture_generated.mjs and its CAPTURE.md. What is missing is the only part that cannot be automated: somebody running the prompts.

What a row will hold

The prompt, verbatim
Including any system preamble. A paraphrase is not the prompt, and a variance claim rests on the prompt being identical across variants.
The tool and its version
"An AI" is not a source. The model or build string where the tool states one.
The date it was generated
The field that decides whether the row is worth anything. These go stale faster than anything else in the building.
Three variants per prompt
One prompt run three times, all three kept. What varies when nothing in the request did is the only way to show that the output is a sample rather than an answer.
Every contrast pair, computed
Text on background off the rendered output, with the ratio rounded down and the threshold named — the same arithmetic as the pair table.
Target sizes and type sizes
Measured, then compared against the declared spec: how many targets fall under 44px, which type sizes are off the stated scale.
What it does well and badly
A judgement, written by a person, null until they write it.

Until then

  • Every pair, with its numberThe check this dataset automates, done by hand on five colours. Working now.
  • The buttonWhat measuring a control against a specification looks like when the rows exist: 63 primary buttons, 42 of them under the height WCAG 2.5.5 names.
  • What each tool computesA hundred and twenty measuring instruments, and the specification each one actually implements.