Review toolkit

Vitrinka reads your runner's output

The review toolkit consumes what your test harness already leaves behind: screenshots, plus an optional text map beside each one. Vitrinka never starts a browser, never drives a device and never runs OCR. Everything a rule needs is a file your Playwright, Cypress, Selenium, Appium or Maestro suite can write in a few lines, and every rule is published here so a team can implement it inside its own suite instead.

The review brief

The rules a review applies are the team's data on the server, not a prompt someone retypes. Each rule is one markdown document: frontmatter id, family, check (mechanical, partial or judgment), severity, an optional platform and enabled, then the rule sentence and a ## Verify list. Rules resolve by id in three layers — the built-in rubric, distilled from real review annotations, then the workspace's rules, then the project's overrides.

• In the browser — Settings → Workspace → Review and a project's Review tab list every resolved rule with the layer it comes from, an enable switch, an editor and a reset to the layer below. Changing a rule is an admin's job; it is linted on save, versioned and audited.
• From a terminal — vitrinka review rules list, get, lint and put read and publish rule files; put lints first and every put is a new version.
• Into a review — vitrinka review brief (MCP review_brief) renders the enabled rules as one compact prompt block, with an optional --persona seat line. The judge below applies it, and /vitrinka:review hands it to the agent reviewing a board.

Review runs

A review over many boards should not read every screenshot every time. /vitrinka:review starts from a server-prepared review plan: one call over the boards decides what must be judged, and opens a review run that keeps a record of what was actually seen.

• Scope — each journey's latest capture pass by default. Boards holding several capture passes (or many screens) answer with every option's cost — screens, screens to judge, sheets — and the plan waits until the scope is chosen: latest, all, or named sections.
• Only what differs — blank, skeleton and undecodable shots, and a runner's failed shots, are capture gaps: never a finding, never a pass. Byte-identical screens are judged once. A near-duplicate of the same device and theme is judged by a crop of only the pixels that differ from its group's representative. A screen judged in an earlier completed run under the same rubric is reused; editing any rule re-judges everything.
• Proof of sight — the run's first sheet prints a short code in its pixels, never as text. The reviewer echoes it before it may read sheet 2 or file a finding, so a host that does not receive images stops after one image instead of reporting "no findings".
• A receipt for every card — every screen in scope ends as inspected, represented (by a crop or an identical twin), reused, a capture gap, or blocked. Findings already open on the same screen version are listed on its tile, and filing one again is refused as a duplicate.

vitrinka review plan <board> <board> --scope latest
vitrinka review plan --run <id> --cards
vitrinka config reviewer codex   # this machine's reviews judge on the Codex sidekick

The plan reports its image cost beside a computed baseline — every screen in scope on 12-screen survey sheets — so the saving is a number. Over MCP the same door is review_plan, and the run's sheets are read with get_card_image {run, sheet}.

The text map: boxes.json

For each image, write a sibling named <image filename>.boxes.json, including the image extension: login.png.boxes.json. The file is a JSON array; each entry carries text, x, y, w, h and an optional role. Coordinates are source-image pixels with a top-left origin. Width and height must be positive and every coordinate finite. Negative origins are allowed so a lint can report overflow.

[
  {"text":"Continue","x":24,"y":320,"w":144,"h":40,"role":"button"},
  {"text":"Email","x":24,"y":200,"w":60,"h":20,"role":"label"}
]

Capture text exactly as shown, in capture order, duplicates included. Limits are 2 MiB and 20,000 entries; unknown fields and missing required fields are rejected.

The map travels with the screenshot everywhere a screenshot travels: vitrinka qa run publish sends sidecars beside each reporter's original screenshot path, vitrinka board capture --boxes <path> accepts an explicit map (and adopts the sibling automatically), and a set push includes sidecars beside its images. The server snapshots each map into the card revision, so republishing a set never moves an earlier revision's anchors.

Writing the map from your runner

Web screenshots are captured at the device pixel ratio, so multiply CSS rectangles by devicePixelRatio; native tools already report screen pixels. Each snippet collects visible text nodes and writes the sidecar next to the screenshot it describes.

Playwright

const boxes = await page.evaluate(() => {
  const dpr = window.devicePixelRatio || 1;
  const walker = document.createTreeWalker(document.body, NodeFilter.SHOW_TEXT);
  const out: { text: string; x: number; y: number; w: number; h: number; role?: string }[] = [];
  for (let n = walker.nextNode(); n; n = walker.nextNode()) {
    const text = n.textContent?.trim();
    if (!text) continue;
    const range = document.createRange();
    range.selectNodeContents(n);
    const r = range.getBoundingClientRect();
    if (r.width <= 0 || r.height <= 0) continue;
    const role = (n.parentElement?.closest("[role]") as HTMLElement | null)?.getAttribute("role") ?? undefined;
    out.push({ text, x: r.x * dpr, y: r.y * dpr, w: r.width * dpr, h: r.height * dpr, role });
  }
  return out;
});
await page.screenshot({ path: "login.png" });
fs.writeFileSync("login.png.boxes.json", JSON.stringify(boxes));

Cypress

cy.screenshot names the file; write the map through a task.

cy.screenshot("login");
cy.document().then((doc) => {
  const dpr = doc.defaultView.devicePixelRatio || 1;
  const boxes = [...doc.querySelectorAll("button, a, label, h1, h2, h3, p, span, input")]
    .map((el) => ({ el, r: el.getBoundingClientRect(), text: (el.innerText || el.value || "").trim() }))
    .filter(({ r, text }) => text && r.width > 0 && r.height > 0)
    .map(({ el, r, text }) => ({ text, x: r.x * dpr, y: r.y * dpr, w: r.width * dpr, h: r.height * dpr, role: el.getAttribute("role") || undefined }));
  cy.task("writeFile", { path: "cypress/screenshots/login.png.boxes.json", body: JSON.stringify(boxes) });
});

Selenium

The same DOM walk as Playwright, run through execute_script.

boxes = driver.execute_script("""
  const dpr = window.devicePixelRatio || 1, out = [];
  const walker = document.createTreeWalker(document.body, NodeFilter.SHOW_TEXT);
  for (let n = walker.nextNode(); n; n = walker.nextNode()) {
    const text = (n.textContent || '').trim(); if (!text) continue;
    const range = document.createRange(); range.selectNodeContents(n);
    const r = range.getBoundingClientRect(); if (r.width <= 0 || r.height <= 0) continue;
    out.push({text, x: r.x * dpr, y: r.y * dpr, w: r.width * dpr, h: r.height * dpr});
  }
  return out;
""")
driver.save_screenshot("login.png")
open("login.png.boxes.json", "w").write(json.dumps(boxes))

Appium

The page source carries native bounds in screen pixels: iOS attributes are x y width height, Android is bounds="[x1,y1][x2,y2]".

import re, xml.etree.ElementTree as ET
root = ET.fromstring(driver.page_source)
boxes = []
for el in root.iter():
    text = (el.get("label") or el.get("text") or el.get("content-desc") or "").strip()
    if not text: continue
    if el.get("bounds"):
        x1, y1, x2, y2 = map(int, re.findall(r"\d+", el.get("bounds")))
        rect = dict(x=x1, y=y1, w=x2 - x1, h=y2 - y1)
    else:
        rect = dict(x=float(el.get("x")), y=float(el.get("y")), w=float(el.get("width")), h=float(el.get("height")))
    if rect["w"] > 0 and rect["h"] > 0:
        boxes.append(dict(text=text, **rect, role=el.tag))
driver.get_screenshot_as_file("home.png")
open("home.png.boxes.json", "w").write(json.dumps(boxes))

Maestro

No scripting inside the flow: dump the view hierarchy beside the screenshot and convert it once.

maestro screenshot home.png
maestro hierarchy > home.hierarchy.json
jq '[.. | objects | select(.attributes.text? and .attributes.bounds?)
     | (.attributes.bounds | capture("\\[(?<x1>\\d+),(?<y1>\\d+)\\]\\[(?<x2>\\d+),(?<y2>\\d+)\\]")) as $b
     | {text: .attributes.text, x: ($b.x1|tonumber), y: ($b.y1|tonumber),
        w: (($b.x2|tonumber) - ($b.x1|tonumber)), h: (($b.y2|tonumber) - ($b.y1|tonumber))}
     | select(.w > 0 and .h > 0)]' home.hierarchy.json > home.png.boxes.json

Findings anchored to text

An agent filing a finding through annotate no longer guesses pixels. A region can be { "text": "Continue" }, or { "text": "Continue", "nth": 1 } for the second occurrence. Matching is exact and case-sensitive against the map of the declared card version; ambiguous or missing text answers unresolved, and nothing ever picks a nearby label instead. Pixel regions {x, y, w, h} and normalized regions {nx, ny, nw, nh} remain alternatives; do not combine them.

category (a review rule id or a legacy category) and severity are required on every agent finding. With dryRun: true the batch resolves every anchor and returns the rectangle plus a version-pinned crop URL without writing anything, so a wrong anchor is caught before it lands on the board.

Optional measurement evidence

An entry can carry metrics. Omit what the runner does not know; never invent values. Capture only the comparable elements a rule needs, because assigning every text node a parent or group describes the DOM, not the intended layout relationship.

Eleven mechanical checks

vitrinka review lint ./screenshots --viewport 1440x900 --theme dark --locale cs-CZ

This writes screenshots/candidates.json; --output selects another path. Supported images are PNG, JPEG, WebP and GIF up to 64 million pixels. The viewport defaults to the image dimensions; theme and locale must describe the capture.

A candidate is a measurement for a human to verify, never an automatic fix request. Without a map only a dark-theme bright-region heuristic runs; every other check is recorded in skipped with its reason. An empty candidate list is never proof that skipped checks passed.

Judge and audit

What a program cannot measure goes to the judge: a server-side pass that reads each captured screen against the brief's rules and returns versioned, evidence-backed results. Start it with run review on a board (the ⋯ menu, or the review view of the board's Annotations panel), with vitrinka review judge <board>, or with /vitrinka:review <board> eve. It runs on the organization's AI provider.

• Pinned — each job pins the card versions and the rubric version it judged, so a result always says what it looked at.
• Audited — every proposed finding, and during calibration a sample of passes, is audited independently; the auditor sees the evidence and the claim, never which model made it. Missing evidence stays Needs evidence, never a pass.
• Yours to file — results show in the Annotations panel's review view, grouped by rule, each with its evidence crop. File finding stages it on the board; Dismiss drops it; a usefulness rating (valuable, minor, not useful) is kept apart from the AI's own assessment. Nothing is filed, resolved or dismissed for you.
• Verified after a fix — Verify a fix audits a fresh capture of the same screen against the finding.

vitrinka review judge <board> --rules spacing,copy --instructions "check the empty states"
vitrinka review judge <board> --status
vitrinka review judge export <job> --json

--section or --journey narrows the pass, --set judges a published set, and export writes the judgments as JSON to review and mark keep before filing them the way the next section shows.

Review, file, and learn

Review each candidate, set keep: true only on findings worth filing, and add the image card's cardId and cardVersion from the board document. The CLI never infers a revision from a filename.

vitrinka review file <board> screenshots/candidates.json --agent codex --dry-run --json
vitrinka review file <board> screenshots/candidates.json --agent codex --json

Both calls use the annotate batch door. The dry run resolves the same anchors and returns a crop URL per ready item; inspect the crops, then file. Filed findings are native staged annotations waiting for a human's Accept or Dismiss in the board's Annotations panel.

vitrinka review stats --project <project> --since 90d

Stats count filed, accepted, dismissed and still-staged findings per rule and agent within your board visibility. The dismiss rate is dismissed over accepted plus dismissed, so staged findings never improve it, and it is null until someone has decided. That rate is the drift signal: a rule people keep dismissing is a rule to rewrite.