Conformance and fidelity
This chapter is a draft for review, dated 2026-09-24.
Conformance is what a suite measured. Acceptance is what an authority decided about that measurement. The two answers must never be merged into one verdict (R8, D12).
This chapter specifies conformance suites, their features and the report a run emits. It then specifies the acceptance record an authority writes, the three checks that test a still-image claim, the motion checks, the editable round trips and the conformance of operations that draw nothing. It restates plan section 11 in normative form, with the corrections the review recorded as C8. The operations being measured are specified in Operations and outcomes. The authority that accepts a result is specified in Constraints, authority and local choice.
Two separate answers
A measured result and an accepted result answer different questions, so this specification keeps them in separate records (R8).
Measured conformance states the suite name and its released version, the implementation identity, the reference environment, the fixture, reference and comparator digests and the per-feature results (R8, C8). It is a fact about one run on one machine.
Contextual acceptance states the authority, the suite reference it judged and each exception it granted (R8, D12). It is a decision, and a different authority may decide otherwise about the same report.
An authority may select a suite, require stricter checks, approve references or accept an exception (R8). None of those decisions may change the meaning of a suite version already published under another identity (R8). A local waiver of a creative rule must leave measured conformance to the unmodified rule reading failed, never passed (R6, C4). The waiver is recorded as an exception on the acceptance, never on the report (R8).
Suites and features
A conformance suite is an immutable versioned set of fixtures, references, a comparator and a reference environment (plan section 3). A suite must be released under a version, and a report must never name a branch or a commit in its place (R8).
Each suite declares named features, and each feature is marked core or extended (R8). A core feature must never be skipped (R8). An extended feature may be unsupported, and the report must list it as unsupported rather than silently omit it (R8).
Requiredness is scoped to the affected operation or output (R11). Missing motion support must never block inspection or a still export that has no motion dependency (R11), which is the rule the four gates in packages/core/src/host-v1/apis.ts and schemas/tool.schema.json already follow.
Independent authors, Lolly Work instances and local people may publish suites and feature namespaces under their own namespace and version (D10, plan section 10.2). A suite namespace carries publisher identity. What names they take, and who may publish under lolly/*, is Q5 below, decided on 2026-10-16 by accepting its default.
Fixtures
A fixture must identify its inputs, its immutable dependencies, the reference environment, the features it claims, the expected artifact or typed result and the comparison rules (D3, tests/docs-shots-vector.test.ts).
Expected trees and traces may supplement output checks (D3). They must never replace them, because a valid JSON tree is not evidence of a rendered result (D3).
A fixture must declare its comparison regions itself (R9, C8). A region discovered from the difference between a candidate and its reference is exploratory evidence, never a region a fixture may declare.
ConformanceReportV1
The report shape follows Gateway API's conformance report, which the plan consulted as outside practice (R8, plan section 15).
draft shape
ConformanceReportV1
suite { name, version } a released version, never a branch or a commit
implementation { project, version, shell, mode, contact }
date
features
core { result: passed | failed, statistics, failedTests[] }
extended { result, statistics, supportedFeatures[], unsupportedFeatures[], skippedTests[] }
environment { fonts, shapingEngine, rasteriser, codecs, gpu, os }
digests { fixture, reference, comparator }This is a draft shape, not a frozen type. Nothing is added to packages/core or schemas/ in this wave, and the plan's section 18 orders the counterexample fixtures in Proof cases before any of it is frozen.
A skipped test must never be recorded as a pass (R8). A report whose suite version does not match the fixtures it ran is invalid (R8). A report must bind to the fixture, reference and comparator digests as well as the suite name and version (C8). Reports are data, and publishing one grants no trust (R8).
AcceptanceV1
draft shape
AcceptanceV1
authority { kind: work-administrator | local-person, instance, id, name }
subject { definition, instance, revision, operation }
report { suite, version, reportDigest }
decision accepted | accepted-with-exceptions | rejected
exceptions[] { target: ruleId | featureName, reason, by, date, scope }
dateAn acceptance must name the report it judged by digest (R8, C8). An acceptance describes exactly the revision named in its subject. When that revision changes the acceptance stays as history and must never describe the new revision as previously accepted (C7). Packaging, identity and migration states the same rule for receipts and signatures. An exception must name its target, its reason and the authority that granted it (R6, R8). An acceptance must never restate a measured result as passed when the report said otherwise (R8).
The governed authority is the Lolly Work administrator for the brand, recorded with the instance they administer (D12). Otherwise it is the local person using the client (D12).
Source review, use eligibility and delivery approval
AcceptanceV1 judges a conformance report. The DAM review requires separate subjects for catalog admission, upstream creative review, contextual use eligibility and final-byte delivery approval (plan 314; R8). A source review may have no conformance report. It must keep its own evidence and must never be turned into a suite pass.
A proposed source-review record must bind the decision to the source, exact reviewed file revisions or digests, reviewer authority and scope. A proof containing several files must identify its reviewed membership and revisions. Conditional approval, such as changes still required, remains conditional. Replacing a file or changing that membership must never transfer the earlier decision silently. These shared review records are not built by this amendment. Lolly Work currently treats replacement of admitted asset bytes as curation; its contribution approval is not a general review of every replacement (lolly-work/server/src/catalog/submit.ts).
Use eligibility also needs the requested operation, channel, territory and date where the applicable evidence requires them. Missing source data leaves an unknown answer; a general approved flag does not settle those conditions (packages/core/src/rights-v1.ts, policy). Signature validity, source assertions, review and eligibility remain independently reported facts.
Fixed delivery approval must bind the final staged output digest. Lolly Work's existing optional destination approval records that subject before provider egress (lolly-work/server/src/api/app.ts, lolly-work/server/src/delivery/types.ts). Ordinary exports do not acquire the same guarantee merely by referring to the same source. Each remotely processed derivative is a new artifact with its own checks and review requirements.
Adaptive publication would instead accept a pinned source and a versioned policy for permitted transformations. This is a proposed acceptance class, not a capability supplied by the draft. It requires its own validation and delivery contract before implementation and must never be presented as approval of exact delivered bytes. Existing fixed-byte delivery keeps its current meaning (R8; plan 300).
Review purpose and production proofs
A source approval assertion needs its purpose, activation target, origin and subject, including whether a person decided or an automated rule set the state (R8, R15; plan 315). Approval for an internal library must never imply approval for public delivery. A mapped flag with unknown subject stays an incomplete upstream assertion, even when the bytes have a verified digest.
Campaign review identifies exact variant revisions and proof-set membership. A responsive proof additionally identifies the viewport, rendering environment and sampled state it shows; a screenshot proves only that view. Changed copy, crop, bytes or membership needs the relevant decision again. Successful unchanged variants keep their historical decisions; a set-level policy may still require resubmission of the set. Review routing belongs to the host workflow and does not grant source, processing or delivery access.
The native 0.1 suites
| Suite | Scope | State in this draft |
|---|---|---|
still-2d/1 | PNG, SVG and PDF, including print marks, bleed and CMYK output targets. | The one native suite for 0.1 (R12). |
motion-2d/1 | Timed appearance and duration for motion output targets. | Staged after plan 196's TimelineV1 exists (R12). |
present/1 | Interactive presentation and its state transitions. | Stays an adapter until Agenda's portable runtime contract is typed (R12). |
The evidence folder records 25 raw manifest rows declaring video and one declaring render.portable, raw directory counts taken before content profile resolution (C9), while the frame clock exists and TimelineV1 does not (R12).
Still-image fidelity
D8 allows a very low pixel difference, difficult to notice without close inspection. The plan reads that intent as no visible loss on close inspection (plan section 2) and keeps it, tested by three checks rather than by one number (R9).
Each fixture declares the parameters of all three checks, its fuzz band, its regions and the facts to read back from the artifact, and a claim must pass all three (R9). A fixture that cannot state one of them must say why in the fixture, and a report that rests on it must name the check that did not run (C8).
- Whole-image fuzz band. A per-channel maximum difference and a maximum count of differing pixels, each with a lower and an upper bound, in the form web-platform-tests uses (R9). A tolerance that must accept identical output has a lower bound of zero. This catches gross drift and admits anti-aliasing.
- Region-scoped structural checks. Regions the fixture declares, such as a legal line, a logo or chart labels, compared by ink ratio and structural similarity inside the region (R9). The fixture declares the region (R9). The comparator must never derive it from the difference it is measuring (C8).
- Semantic checks on the produced artifact. Text content, page count, chart data and layer inventory, read from the artifact or from an independently extracted representation, compared through
compareSourcesinengine/src/compare.ts, which dispatches by mode tocompareTextinengine/src/compare-text.tsandcompareStructureinengine/src/compare-structure.ts(R9). Comparing authored text with the same authored text would miss a broken exporter, so the comparison must read the output (C8).
Pixel counts alone must never decide a still-image claim (R9).
The evidence and what it does not settle
A comparator probe rendered a QR code with a caption through the CLI shell as render A, then derived three variants from it. B removed the caption glyphs and left every box unchanged. C shifted the image by half a pixel. D rasterised at 1023 px and resampled to 1024 px. The caption region is 6.09% of the canvas. The probe is exploratory evidence and is not published with this specification.
| Pair | Changed pixels, whole image | SSIM, whole image | Changed pixels in the caption region | Ink ratio in the region | SSIM in the region |
|---|---|---|---|---|---|
| A vs B, caption gone | 1.46% | 0.969 | 24% | 0.00 | 0.43 |
| A vs C, half-pixel shift | 1.25% | 0.968 | 8.8% | 1.00 | 0.89 |
| A vs D, resampled | 1.29% | 0.968 | 6.9% | 1.00 | 0.95 |
The three pairs sit within a quarter of a point of each other on changed pixels and within 0.002 on SSIM. A real content loss and two harmless changes are therefore indistinguishable by a whole-image number. The region-scoped measures separate them at once. A text-presence check on the artifact would separate them with no pixels at all, by construction rather than by measurement: B was made by removing the caption glyphs from the source, and the probe ran no such check.
The probe establishes the need for complementary checks. It sets no threshold (R9, C8).
It also does not establish a validated combined comparator or cross-shell conformance (C8). Its region was discovered from the A and B difference, which a fixture must never do (C8). Its fonts were whatever the rasteriser found on one machine, and resvg-js does not report which files it resolved. The raster was composited onto white. The structural similarity is one small implementation. Nobody decoded the QR afterwards, so "nothing lost" in C and D describes the source edit only.
A second measurement, from the drawing compiler's fidelity harness in plan 295 (tests/design-draw-fidelity.browser.test.ts), uses declared regions as this chapter requires. It mounts fixture pages through the real Design tool and draws them in Chromium beside the SVG compiled from the same rows. Shapes, paths and effects give zero differing pixels. Live text differs only where runs meet, by a fraction of a pixel (0.49% of a mixed-style text box). Outlined text cannot match pixel for pixel, because the font rasteriser thickens glyph stems that a path fill does not: on macOS Chromium a page of outlined text differs from the browser's text in 5,578 pixels while every word is present and in place. The harness therefore declares one region per laid-out run, taken from the drawing operations and never from the difference, and compares the ink each side adds to the page without its words, and the centroid of that ink. Outlines measured 2.5% to 17.4% less ink with centroids at most 0.60 px apart, and on every run the harness also draws two rejected examples, a missing word and a word moved by 2 px, and fails if its limits let either pass. Those limits come from one machine, one accepted set and two rejected examples, so they are the harness's own and are not yet a number for this specification.
The same harness judges the web shell's DOM walker, the fallback for pages the drawing operations do not admit, against the same Lolly Design pages and the same regions, so the incumbent is a candidate like any other rather than the reference. The walker passes the shapes, paths, effects and text pages, and fails eight regions the drawing operations pass: a picture fitted by fill (27% of its region differs) or none (47%), because the walker writes no aspect rule for either; a picture whose fitted edge falls between pixels, which Chromium snaps and the walker does not; and a row the frame clips inside its border, which the walker clips at the frame's outer corner instead. A reference renderer is how such faults surface, and an incumbent export cannot be its own reference.
A PDF is drawn by a different rasteriser from the reference. The harness rasterises each PDF page with Poppler at one pixel per CSS pixel and compares the result with the same Lolly Design pages and regions. At the Chromium threshold, every curved region differed by about 1% for both the drawing operations and the walker, because Poppler covers an edge pixel differently. For PDF, a pixel therefore counts as different only when nothing within 1 px in the other image matches it, and a half-covered edge pixel may differ by up to 40 levels (the measured worst case was 28). A picture drawn at another size than its own is resampled by the viewer and may differ in 2% of its region (measured: 1.3% for both renderers). The harness proves those allowances still catch a fault: the same page with one rectangle moved by 2 px must fail its region, and does. With these limits the operations' PDF passes every region, and the walker's PDF still fails the clipped rows and the frame picture's corner. The numbers come from one machine and one rasteriser; they are the harness's own, not this specification's.
The Node PDF path adds artifact checks, not another fidelity reference (tests/design-ops-pdf-node.test.ts). Its tests reopen a two-page outlined RGB Lolly Design PDF and inspect the authored page sizes, metadata, output intent, page boxes and absence of artwork fonts. They verify bleed and marks on each page, and deliver a marked PDF while the CLI's browser endpoints are unavailable. The final Content Credential is read back and its data hash and signature are checked against the delivered bytes. A Unicode provenance label that Helvetica cannot encode exercises a real finishing failure: the direct Node call keeps the original WinAnsi error as its cause, a later valid export succeeds, and the CLI with --html-fallback writes neither PDF nor HTML and never escalates to a browser. Separate finishing tests decrypt the AES-256 output's strings and metadata to verify the producer, retained output intent and withheld PDF/X claim (tests/pdf-finishing.test.ts). These observations do not establish CMYK fidelity or independently certify PDF/X compliance.
Outlined strike-through adds a decorated reference page with variable weights, italic and mono runs, wrapping, colours, opacity, rotation and clipping. The same harness compares outlined SVG and Poppler-rendered PDF against the real Lolly Design HTML page. Each declared line region measures the added decoration ink against a separately rendered undecorated page, so the glyphs cannot hide a missing line. Both formats pass the existing 25% ink and 1 px centroid bands and reject lines removed or moved by 2 px. The geometry follows the pinned Chromium reference's CSS auto policy, including its local solid-line pixel rounding, with ascent read from the resolved HarfBuzz font instance. These observations qualify those fixtures on this Mac; they do not establish a universal browser policy. Pure italic or mono pages still select the web DOM walker when its vector-font registry cannot resolve the base face; mixed-base fixtures establish the admitted Node/web SVG byte parity without changing that refusal.
Outlined underline adds a second decorated page: descenders at several weights, italic and mono runs, a coloured right-aligned run on a fractional baseline, a strike and an underline on one run, a clipped translucent box, a wrapped run whose hanging space is underlined, a 15 px run with a one-pixel line, and a rotated row. The same declared line regions and undecorated grounds judge it in Chromium 153.0.8010.12 and Poppler, within the same 25% ink and 1 px centroid bands. Measured on this Mac, the SVG lines differ from the reference by at most 0.8% ink and 0.55 px, and the PDF lines by at most 13.2% and 0.56 px. Each format rejects a removed line, a line moved by 2 px and a line drawn straight through the descenders, so the check sees skip-ink and not only the line's place. The web DOM walker fails seven of the page's regions where the drawing operations pass. These observations come from one machine and one fixture page.
SVGZ tests decode the gzip bytes and compare the result with SVG from the same operations, including dimensions and metadata (tests/design-ops-svg-node.test.ts). Transport tests check that an unadmitted SVGZ page or live-text PDF makes one complete browser request, preserves the returned bytes and reports its admission reason (tests/design-session-transport.test.ts). Snapshot tests prove that a beforeExport mutation cannot rewrite the authored values passed to the bridge. Rejected integrity cases remain errors rather than admission findings. Rejected hook cases write no artifact, including an HTML substitute when --html-fallback was supplied. These are maintained implementation checks, not a released still-2d/1 suite or the full draft receipt; the reference, environment and comparator requirements above still apply to a conformance claim.
Thresholds must be calibrated on accepted and rejected examples before any number enters this specification (R9). Masks must be explicit and justified, and human review sets and reviews the bands and must never replace the comparator (R9, plan section 11.2).
The output-validation boundary
An output-validation boundary sits after rendering and before delivery of any artifact that claims acceptance (C8). An artifact that has not passed that boundary must never be delivered with an acceptance claim attached.
Comparison for a claim must run at native raster scale with pinned fonts and rasteriser, with declared colour handling and alpha treatment (R9, C8). The preview-grade comparison in engine/src/compare-visual.ts samples at most 768 px per edge with nearest-neighbour sampling on white, and it states that limit in its own result. It stays a preview tool and must never decide a conformance claim.
An undetermined result must never be a pass (R9, C8). compareSources in engine/src/compare.ts returns undetermined when a budget cut the comparison short, and engine/src/compare-budget.ts is where that budget is spent. Harmless truncation of displayed detail must be distinguished from incomplete comparison coverage (C8).
PNG alpha, PDF page boxes and colour intent must each have their own checks (C8). A white RGB fixture says nothing about CMYK print conformance, and a suite must never generalise from one to the other (C8).
A run whose text stayed a live <text> element rather than an outline must record that in its receipt, and it must never claim font-independent fidelity (R15). Evaluation and receipts specifies the receipt.
Graphics devices and numerical hosts
The first WebGPU and Rust kernel work (plan 295) supplies evidence for the fuzz band and its limits, added on 2026-10-07. It sets no threshold for any suite.
- GPU output is not a byte-identity claim. WGSL floating-point arithmetic may differ between devices and drivers. LUT grading on WebGPU is checked against a Rust and a TypeScript reference with a bound of one 8-bit code value per colour channel on the admitted corpus, and exactly for alpha, identity tables, constant tables and no-op intensity (
tests/webgpu-lut.browser.test.ts). A suite that admits a GPU renderer must declare such a band, with exact checks for the cases where exact output is the contract. - A GPU renderer declares its band per kind of region, with a negative control for each limit, added on 2026-10-14. The internal Lolly Design raster path (P3f) draws primitives with Chromium's edge coverage and no slack. Pen shapes and outlined words are filled with exact area coverage over flattened edges, after overlapping contours are merged, because a summed winding counts an overlap twice and variable-font glyphs keep their overlaps. Three references judge the fills (
tests/design-draw-webgpu-fills.browser.test.ts). Against the Lolly Design renderer, a shape region allows no positional slack and a per-pixel difference of up to 72 levels in at most 0.5% of its pixels: the worst measured region differed in 0.09%, and the same triangle moved by 2 px fails with 4.24%. An outlined word allows 25% ink difference and a centroid shift of 1.5 px times the output scale, and a missing word and a word moved by 3 px both fail. Against its own outline, a word's ink must equal its exact area within 1% and its centroid must sit within 0.05 px of the outline's area centroid: measured within 0.04% and 0.013 px on Metal, SwiftShader and Linux CI alike, and a word moved by 0.25 px fails. Chromium's SVG of the same operations is a diagnostic only, because on Metal it rasterised the outlines with quarter-step coverage and sat up to 0.36 px from the exact centroid. On Linux CI, Chromium's text sat 1.08 px from the operations' layout for an italic run after bold and plain runs at 16 px (2.62 px at 2x) while the GPU sat 0.003 px from its outline, so that distance belongs to the text layout rather than the renderer, and the Lolly Design band for words is wider than the drawing compiler's 1 px. The harness also found a fault outside the renderer: under load, headless Chrome returned a torn reference screenshot, so a reference is accepted only when two consecutive screenshots match. These limits come from one fixture set, so they belong to the harness and are not yet a number for this specification. - A stroke is the region it paints, added on 2026-10-14. The P3f raster path draws a stroke as the fill of its outline, over the shape's fills and under its words, and judges it as a shape region. A box's stroke is a CSS border: the ring between the box and the box inside the border, whose corner radius shrinks by the border width. A pen shape's stroke is centred on its path and outlined with its cap, its join and SVG's mitre limit of 4. A dashed border, a centred stroke on a box and a stroke that fades as a group over a fill or words are refused; dashes on a pen stroke follow below. On the Mac every border and pen stroke in the fixture differed from the Lolly Design renderer in 0.00% of its region but one, and a border or a pen stroke 1 px wider differs in 3.72% and 2.82%. The exception is a finding about the Lolly Design renderer, not the GPU. Lolly Design offsets a pen shape's
<svg>by its stroke pad so the outline is not clipped, and Chromium snaps a fractional offset to whole pixels, so a round-capped stroke of odd width (pad 0.5 or 1.5) draws 0.5 px right of and below its geometry, and a square cap (pad w/√2) 0.12 px. A 1 px round-capped diagonal differed in 3.5% of its region while the GPU matched Chromium's SVG of the same operations exactly, and the square-capped stroke in the fixture differed in 0.31% at 2x against 0.01% for the SVG. The drawing compiler's harness has not measured this, and the fixture keeps to whole pads except for that one square cap. - A gradient is judged by its colour as well as its outline, added on 2026-10-14. The P3f raster path paints a box's gradient over the primitive's own coverage, as CSS paints a background image: on the box inside the border, repeated across the box, after the box's colour and before its border, with the operation's opacity folded into the stops and colours interpolated premultiplied. A linear gradient follows the line the drawing operation carries, and a radial one is an ellipse at the centre of that box reaching its farthest corner. Gradient regions take the shape band, and inside each unturned gradient box, past its outline, no pixel may differ from the Lolly Design renderer by more than 2 levels. On the Mac every interior pixel sat within one level at 1x and 2x, and two controls fail every interior pixel: the gradient line moved by 2 px, and a baked OKLab ramp drawn from only its two end colours. A colour and a gradient on a box that fades as a group are refused, since painting them one after the other differs from the group.
- A frame's clip is part of its paint, added on 2026-10-14. The P3f raster path draws a rounded or elliptical frame with its own outline and border, and when the frame clips its rows, every later paint step is drawn inside the box within the border at the inner radius: each step's coverage is multiplied by the clip outline's anti-aliased coverage, as Chromium clips a board's children draw by draw. A page without a clip keeps the unclipped passes unchanged. On the Mac a rounded, bordered board and an elliptical one differed from the Lolly Design renderer in 0.01% of a region at most, rows crossing the edge included, and the same pages drawn without the clip fail in the rows that cross the edge (up to 24%) and in the frame's corners (up to 76%). Outside a rounded or elliptical frame the page is transparent, where a pixel's colour carries little information, so the harness judges both images over the same one-pixel black and white checkerboard. A single grey backdrop was tried first and missed a purple row drawn outside the frame, whose colour sat within the band of the grey.
- Dashes are placed where the browser places them, added on 2026-10-17. The P3f raster path cuts a pen stroke into the dashes Lolly Design's path hook asks for, a
stroke-dasharraywith no offset, and outlines each dash with the stroke's cap and join. The pattern restarts on each contour. On a closed contour the first dash is drawn last, and when the final dash reaches the start the first one continues it, so the two meet in a join rather than two caps. A dash of no length is a dot under a round or square cap, and a square dot stays upright. Distances along the path follow Chromium's measure rather than exact arc length: each curve is halved until its control points lie within half an output pixel of the chord, and a distance inside a chord is read linearly in the curve's parameter. On the Mac every dashed region in the fixture differed from the Lolly Design renderer in 0.05% of its pixels or less, curves and dots included, while the same dashes placed by exact arc length differed in up to 0.43%, close to the 0.5% limit. The dashed zig drawn solid fails with 2.19%, and the mitred zig with each dash 2 px longer over the same period fails with 1.51%. A dashed border stays refused, because CSS spaces a border's dashes side by side rather than along the outline. The work found a fault in the stroke outliner: outlining every dash of a stroke at once dropped a dash shorter than the stroke is wide, because the outliner reads a point within half the width of a butt end as painted and took that dash's outline for a hole. The raster path therefore outlines each dash on its own and merges the results. - A row clipped by another row's silhouette is clipped draw by draw, added on 2026-10-17. Lolly Design clips such a row with
clip-path: polygon(), four points for a rectangle and 48 for an ellipse, and the drawing operations carry that polygon in page coordinates. The P3f raster path draws each of the row's paint steps at its own coverage times the polygon's exact area coverage under the nonzero rule, and inside the frame's clip as well when the frame clips its rows. On the Mac every clipped row in the fixture differed from the Lolly Design renderer in 0.04% of its region or less, and the same rows drawn without their clips fail with 4.71% to 28.26%. A gradient in a clipped row is judged by its region alone, because the clip's edge runs through the inside that the gradient check reads. The work found a fault in the Lolly Design renderer: it writes the polygon in the clipped box's own frame without the box's mirror, so a mirrored box shows its clip mirrored about the box centre (9,600 px of a 200 by 80 box against the drawing operations' SVG). Until that is settled, the raster path refuses a clip on a mirrored row. - The reference environment of a suite must state the graphics device: the adapter, the backend and whether it is a software implementation. A pass on SwiftShader is software conformance and must never be presented as a pass on physical hardware.
- A shell running JavaScript is a numerical host, and engines differ in the last bit of
Math.hypotand other transcendental functions. In the measured geometry corpus those differences changed raw control points and internal work counts while the serialized SVG stayed equal. A semantic check on the produced artifact therefore stays the deciding check, and a suite must never claim raw floating-point identity across engines for an implementation that reads host maths. The geometry no longer reads host maths:tests/geom-portable-math.test.tspins one digest for the complete geometry workflows and their work counters, andtests/geometry-portable-math.browser.test.tsrequires Chromium, Firefox and WebKit to reproduce the same digest in the main realm and a worker. That is the shape a numerical suite can take: one checked-in answer that every engine must reproduce exactly.
Motion fidelity
Appearance is compared at normalised progress points, and total duration and the time mapping are compared separately (D9, plan section 11.3).
Sampling must include the start, the end, scene and transition boundaries, authored events and samples immediately around discontinuities (D9). Each suite must bound its sampling interval, with denser checks for fast motion (D9).
Loop behaviour and audio timing must be tested where they are claimed (D9).
Sampling is evidence at the tested points. It does not prove the absence of a defect between them, and the suite must state that limit (D9).
Editable round trips
Visual similarity is necessary and insufficient for an interchange claim (R14). A raster fallback may claim appearance and must never claim editable-vector preservation (R14).
Each route must declare preservation separately for six dimensions: appearance, editable objects, semantic data, interactions, timing and constraints (R14, engine/src/penpot-file.ts).
A route is tested by a meaningful edit after import, followed by an export, followed by a comparison across all six dimensions (R14). The required edits are moving a layer, changing a text run and swapping a token.
The first routes are Q1 below. Andy accepted them on 2026-10-16 as the table states them.
| Pilot | First route | What exists today |
|---|---|---|
| Lolly Design | To and from Penpot. | The .penpot writer in engine/src/penpot-file.ts; the editable import is parsePenpotBinfile in shells/web/src/views/design-import.ts over the pure mapper engine/src/design-map.ts (the Unpack reader in shells/web/src/views/penpot-import.ts extracts, it does not round trip). |
| Lolly Design | To native PPTX and back. | The deck reader readPptx in engine/src/pptx-read.ts, reached by the Rebrand source path in packages/node-shell/src/pptx.ts. |
| Chart | To and from CSV and JSON data, plus the SVG semantic overlay. | The portable chart contract in packages/core/src/chart-v1.ts. |
| Agenda | To ICS, portable HTML and PPTX. | The outputs listed in docs/agenda.md. |
Today's composes and compose.renderUrl paths produce a flattened image (R14). That image may pass an appearance check and must fail an editable round trip (R14). Proof cases carries it as a failing case, not as a pilot.
Conformance without a picture
An operation that draws nothing is still measurable (D11, R13). The three paragraphs below state what its suite tests.
For Lolly Rebrand, compare the approved changes against the rebrand plan, the protected content against the source and the output against a reference for that rebrand plan (R13). Partial completion is read through OutcomeV1 as Operations and outcomes specifies.
For Unpack and the other route operations, compare typed results, resource inventories, declared effects and failure behaviour (plan section 11.5, R13). A fabricated preview must never stand in for the real outcome (R13).
For interactive Agenda, test search, navigation, calendar output, state transitions and accessibility (plan section 11.5).
Material that exists today
None of this is a suite yet. These are the parts a suite would be assembled from, and each states its own limit.
| Piece | What it establishes | Where |
|---|---|---|
lolly smoke | Every tool in the active content profile renders at its defaults to its first Node-native format. A skip is never counted as a pass. | shells/cli/src/smoke.ts |
| Export characterisation | A fixed matrix of nine tools, chosen so every export branch is exercised, is rendered twice through the real web export path and classified as hashed or size-banded. Same machine only. | scripts/characterize-export.ts |
| Comparison modules | Bounded text and structure comparison with stable-id moves, an explicit undetermined result and a preview-grade visual mode that states its limits. | engine/src/compare.ts, engine/src/compare-structure.ts, engine/src/compare-visual.ts, engine/src/compare-budget.ts |
| Host conformance kit | A live host carries the members the contract declares, checked at run time rather than at compile time. | packages/core/src/host-conformance.ts |
| Docs shots | A vector baseline is compared exactly, and every raster baseline carries a written reason in one allowlist that fails in both directions. | tests/docs-shots-vector.test.ts |
When a suite exists, its first reports come from this repository's continuous integration, where the render gate, lolly smoke and the export characterisation script already run. A page listing each shell's results is the cheapest trust asset the suite can carry (plan sections 11.1 and 18).
This draft adds no conformance or reproducibility claim beyond the ones docs/determinism.md already makes, and it states no threshold and no latency target (C8).
Open points
- Q1. The first editable interchange routes per pilot. Decided on 2026-10-16 by accepting the default from plan section 16: the four routes and three edits in the editable round trips section above. Evidence that would reopen it: a customer route that outranks them.
- Q5. Who may publish a suite under the
lollynamespace, and how a local or Lolly Work suite is named. Decided on 2026-10-16 by accepting the default from plan section 16:lolly/suites are published from this repository's continuous integration only, Lolly Work instances publish underwork:<instance>/, local people publish underlocal/*and an acceptance record always states the suite it judged. Evidence that would reopen it: a partner programme that needs a shared namespace.
The other questions are carried in other chapters, each decided on 2026-10-16 (plan section 16). Q2 and Q3 are in Constraints, authority and local choice. Q4 is in Operations and outcomes. Q6 is in Status and open points.
Precedents
engine/src/compare.tsengine/src/compare-structure.tsengine/src/compare-text.tsengine/src/compare-visual.tsengine/src/compare-budget.tsengine/src/penpot-file.tsengine/src/design-map.tsengine/src/pptx-read.tspackages/core/src/host-conformance.tspackages/core/src/host-v1/apis.tspackages/node-shell/src/pptx.tspackages/core/src/chart-v1.tsschemas/tool.schema.jsonshells/cli/src/smoke.tsshells/web/src/views/design-import.tsshells/web/src/views/penpot-import.tsscripts/characterize-export.tstests/docs-shots-vector.test.tstests/design-draw-fidelity.browser.test.tstests/design-ops-svg-node.test.tstests/design-ops-pdf-node.test.tstests/design-session-transport.test.tstests/pdf-finishing.test.tstests/webgpu-lut.browser.test.tstests/geom-portable-math.test.tstests/geometry-portable-math.browser.test.tspackages/node-shell/wasm/geometry-kernel/docs/determinism.mddocs/agenda.md