ベンチマーク

The rule

No claim without a number, a date and a version. The comparison pages are the most valuable asset this site will have and the easiest to destroy, and one unsupported claim about a named competitor costs more than every page here is worth in traffic.

The rule is enforced by the schemas rather than by review: a comparison entry cannot compile without a competitor version, a measurement date and an anchor into this page, and a benchmark claim cannot compile without its value, its corpus, its run date and the engine build it measured.

The corpus

One set, committed: 30 fixtures in bench/smoke, run on every pull request. They are 15 procedurally generated scenes at 64 and 256 pixels, each rendered in two variants — clean, and Gaussian blur at sigma 0.7, which is roughly a soft rescale.

A scene is a background rectangle plus non-overlapping shapes on a grid: discs, rectangles, rounded rectangles, triangles, regular polygons, stars and a D-shape, in palettes of two to six colours. Nothing overlaps, because an occluded shape has no recoverable boundary and its truth would have to describe what a tracer ought to see rather than what the scene contains.

Ground truth is generated rather than annotated — the palette, the region count, the node count, the primitive list and boundary polylines sampled at 0.01 px all come from the generator, so there is nothing to disagree about. Placement jitter comes from a seeded SplitMix64 stream, so the committed corpus regenerates byte for byte on any machine, and the raster comes from resvg rather than our own renderer, so a rasterizer bug cannot cancel itself out at scoring time.

The metrics

Six numbers per fixture: boundary error (mean, p95 and max, in document pixels), region-count error, palette difference in Oklab under Hungarian matching, node-count ratio, output bytes and wall-clock milliseconds.

Boundary error is the symmetric point-to-polyline distance, measured against the nearest segment rather than the nearest vertex. A triangle edge has no vertex in the middle of it, so a vertex-to-vertex metric reports a perfectly traced triangle as tens of pixels of error; and measuring in one direction only rewards a tracer for emitting extra paths, or for emitting almost nothing.

Node count is reported next to boundary error on purpose. Winning on boundary error by emitting three times the nodes is not winning — it is a file nobody can edit and a cutter stutters through.

The baselines

vtracer and potrace, both open source, both run locally at their own defaults. Tuning another tracer against our own corpus would make the comparison ours rather than theirs. Potrace is bi-level and this corpus is colour, so its numbers read as a note about the corpus rather than a result about potrace.

A fixture-SVG baseline re-measures the ground truth against itself and comes back at 0.004 px on boundary p95, 1.000 on node ratio and zero on region count and palette. That residual is the two-decimal rounding in the SVG writer, and it is the honest floor of the whole method: any boundary figure below about 0.01 px is measurement noise, not quality.

A hosted service is declared in the harness and not wired up: it needs network access and its terms checked first (D13). Outputs from one are for evaluation only, never trained on, and never published without those terms clearly permitting it — an unusable comparison is better than an actionable one.

The gate

Every pull request scores the smoke set against the committed bench/baseline.json and fails on a regression greater than 5% in boundary p95, 5% in region-count error or 10% in node count. All three are one-sided: getting better never fails.

Speed is deliberately not gated. Milliseconds on a shared runner are noise, and a flaky gate is a gate people learn to ignore, so the committed baseline stores a zero for time and any published timing comes from a release build.

A gate that only runs before a release is a gate that gets argued with. Running it on every change makes a regression the problem of whoever caused it, while they still remember what they changed.

What is published here

The run above: vt 0.3.34 on the smoke set, measured 2026-09-05. Boundary error lands at 0.09 px at p95, every fixture comes back with exactly its own regions and colours, and the outline carries 1.00× the nodes the source geometry has.

Both numbers beat the week-1 engine (1.02 px and 3.25×), which was the acceptance criterion set for the curve fitter — boundary p95 AND node count, both, not either. Since 0.3.1 every boundary two regions share is fitted once and both regions carry the same curve, so stacked layers coincide instead of leaving a rim or a hairline gap between them. 0.3.2 stopped the palette stage inventing a third colour along an anti-aliased edge, and fits each boundary as the optimal polygon first and the fewest cubics second. 0.3.3 dissolves the sub-pixel slivers a snap leaves where two edges cross, and recognises circular arcs and straight lines: a boundary that is an arc of a circle comes out as that circle, exact in position and tangent, and a whole circle costs four cubics whatever size it was uploaded at. 0.3.4 stops reading the boundary off the pixel grid at all — every sample is placed from the anti-aliasing ramp in the source, to a fraction of a pixel — and puts every junction where its incident arcs and lines actually cross, so the tip of a lens is the intersection of its two circles rather than a blunt stump one pixel short of it. 0.3.5 reads a hairline as a ridge and a ramp as a ramp in the palette stage — a one-pixel stroke earns its own colour, and a pixel in the middle of a ramp can no longer claim a third colour that is not there — and cuts every closed boundary clear of its corners before the polygon search, so a rectangle keeps its corners. 0.3.6 decomposes a boundary into the lines and arcs it is made of — a rounded rectangle comes back as four lines and four quarter arcs, tangent at every join — and reads low-contrast edges against the raster noise rather than the pair of colours, so a hairline at any angle is one stroke. 0.3.7 finds a corner on the samples and puts it back on the crossing of its two sides — a star point is where the sides meet, not where the raster stopped — reads a colour the raster holds on a plateau as an ink however close it lies to a ramp, and cuts every counter as a hole of the shape that has it instead of painting a paper-coloured patch on top: the counter of an O is a second contour of the O, wound the other way, which is what an editor expects to find. 0.3.8 fits each side of a corner to its own samples and places the vertex where the two sides cross, checked against the depth the raster left out, and traces a hairline as one stroke — a centreline with a width — where the raster holds one. 0.3.9 reads the lattice a raster holds its colours on — zero on a clean fill, the codec’s step on a JPEG — so the two or three levels a JPEG block leaves of one ink are one ink again, and cuts a hairline out of the shape it lies on in cutout output instead of painting over it. 0.3.10 makes the converter trace the pixels the file stores — the browser had been colour-managing and clipping wide-gamut PNGs before the engine saw them, so the site was not running the engine that was measured — and tells a codec’s duplicate of an ink from a second ink by where its pixels lie. 0.3.11 reads the EXIF Orientation tag in the engine itself — out of a JPEG’s APP1 segment and a PNG’s eXIf chunk, in both byte orders, with the eight transforms checked against the pixels Chromium, Firefox and WebKit return for the same files — so a rotated phone photo traces upright through the converter, the API and the CLI alike, and a file with no tag decodes byte-identically: every number on this page is unchanged. 0.3.12 measures how far a JPEG’s edges overshoot their own flanks — zero on every PNG — and folds the bands that overshoot paints beside a strong edge back into the fill they rang off, so the smiley’s teeth and the football’s panels come back as the regions they are; and where the raster held nothing to measure at a junction, the curve leaving it is redrawn from the crossing’s own geometry, so the tips of a lens on a JPEG leave without a hook. 0.3.13 reads a shade in the middle of a JPEG’s colour ramp as the ramp and not a third ink — on a chroma-subsampled file the chroma lags the light across an edge, and the palette stage now measures that lag, 1 on every PNG — draws a boundary that runs between two crossings of one circle as that circle’s arc, so the same logo uploaded at 150 px, at 600 px or as a JPEG comes back as the same 19-node document, and skips the overshoot measurement wherever no two colours are close enough for it to matter, which gives every PNG its speed back. 0.3.14 stops drawing a face painted the colour it already sits in — the invisible duplicate path a layered tracer nests inside a same-coloured shape — so the icon corpus comes back with 45 fewer phantom regions and nothing in the output is described twice; the one extra region on every copy of Opera, the rocket and Nvidia turned out to be the ground truth counting two pieces that touch at a point as one shape, which no tracer can read from a raster, so it stays and is named. 0.3.15 reads a hairline by its width as well as its colour and bridges the gaps a corner or a crossing leaves in it, so a script wordmark and a thin letter come back as strokes rather than dashes, and on JPEG input a fill cut into pieces by seams is read as one fill before the palette judges it — the JPEG set is exact on 167 of 210 region counts and 190 of 210 palettes, and a bridge that had joined a JPEG logo’s two halves at a pinch was found and refused in the same release. 0.3.16 stops writing the closing line of a straight-sided loop twice — the duplicate anchor an editor shows on a traced closed path — which turned out to be the whole of the node gap on the smoke set: the output now carries exactly the source geometry’s nodes, with every render byte-identical. What is left on the smoke set is the blurred five-pointed star, whose valleys read below the corner threshold once blurred. Every triple point already arrives as one vertex, so the planar-map tracer’s remaining work is on real artwork, not here. 0.3.17 leaves this set untouched to the byte — its change is on JPEG artwork, where a pixel on a chroma-bowed ramp now goes to the pair that explains both its flanks rather than to a nearby shade. 0.3.18 likewise: the hairline reading and the residual floor moved real artwork and the large corpus, not this set. 0.3.19’s dust rule moved only the large corpus — four of its canvases that came back blank now return their artwork. 0.3.20 folds a codec’s ringing halo into the colour it rings and reads a stroke across its own rim; this set is byte-identical again, as it has been since the closing line became implicit — and it stays so in 0.3.21 and 0.3.22, whose seam, lone-crest and corner-window rules move real artwork and the large corpus’s thin diagonals, and nothing here. 0.3.23 finishes every refinement pass of the line-and-arc decomposition and refuses a pass that folds a side away, so a rounded rectangle whose corners outrun its sides comes back as four lines and four arcs rather than four bowed cubics; this set is byte-identical once more, and the movement is on the large corpus’s rounded rectangles and polygons. 0.3.24 lets a colour the mixing model explained away earn its entry when the raster holds it as a fill with an inside and it sits far from any entry the palette already has, so the chequered flag’s grey pole comes back in its own grey on a blurred JPEG; every PNG is byte-identical, this set included. 0.3.25 decides which ink a crest belongs to by the ray its pixels lie on rather than the entry they have come furthest toward, so two near-identical blues no longer trade spike tips; this set is byte-identical and every other corpus improves on region count. 0.3.26 tells a real ink from a codec’s mixture by where a colour’s crest pixels lie rather than by how far its colour sits from the inks already found, reads a map of the colours a raster settles at in place of a disc when it decides whether a shade belongs to a ramp, and refuses a bridge through a pixel another ink owns; this set is byte-identical again, and every other corpus comes back with fewer stray regions. 0.3.27 asks a crest read against a palette with no ramp in it the mixing model’s tolerance once instead of twice — the doubling guards against a codec overshooting a ramp, and a palette holding a single colour has no ramp to overshoot — so a hairline drawn in an ink a shade off its own paper keeps that ink; this set and both sets of real artwork are byte-identical, and the large corpus’s near-paper hairlines, which had traced as an empty sheet since it was generated, come back. 0.3.28 lets the raster overrule that merge where it holds two near colours apart as fills of its own — two settled plateaus, each with a core of its own, standing further apart than the raster’s own noise, on a scan with no compression lattice — and asks a hairline on a palette that holds nothing but paper no share of the canvas at all, because a share is a share of an area and a line has none; the smoke set and both sets of real artwork are byte-identical again, and the large corpus’s near-paper shapes come back in their own colour at every size but the smallest. 0.3.29 changes no engine code at all — what it fixes is the ruler: the boundary is now scored against the curves the picture actually shows as well as against every curve its author drew, because an edge a later shape is painted over is not an edge the tracer was ever given, and the release gate reads the visible column; this set hides nothing, its shapes overlapping nowhere, so every number here is the same number it was. 0.3.30 ships two rules in the palette stage: a small shape whose inside is too shallow for the depth that tells a fill from a codec’s rim is read by its shape instead — a rim is a band and a fill is compact — and the share-free hairline look leaves the ramp-less palette it was leashed to, because a halo beside a stroke can now be refused as the ramp of an ink the residual is still holding; this set and both sets of real artwork are byte-identical again, and the movement is on the large corpus’s near-paper shapes at their smallest size and its hairline grids under a codec. 0.3.31 ships one rule in the same stage: a colour the raster holds level across a line’s worth of its own width is an ink and not a ramp, so the seams of a football emoji come back in their own colour on the copy a blur and a codec had taken them from; this set, both sets of real artwork and the large corpus are byte-identical again, and the only thing that moves anywhere is that one JPEG-coded emoji, whose palette comes back exact. 0.3.32 ships three rules at once, one in the palette stage, one in the sliver pass and one in the curve fitter: an ink whose crests are arranged as a line survives the dust count that a washed hairline can never pass, a straight run’s chord may not slide sideways off the line it was detected on, and a bracket that is the edge’s own rim is not a side of the strip beside it; this set is byte-identical again, and the movement is on the large corpus’s hairlines that had traced as blank pages, a hairline lane the fitter had collapsed to no area at all, and the slivers a codec’s rim had grown along a football emoji’s pentagons. 0.3.33 ships three more, two in the palette stage and one in the curve fitter: a fill is where the snap settles and not only where the bytes repeat, so a football’s seam comes back whole from one pentagon to the next; a component that is one edge’s ramp is that edge’s rim, not artwork; and a closed fit may not spend the area its samples enclose, which gives a chequered flag’s pole edge and a hairline grid’s longest line back; this set is byte-identical again, and the movement is on a football under a codec, a flagpole’s band and the large corpus’s hairline grids, every one of whose eight movers is better. 0.3.34 ships one rule, in the stroke stage: a stroke’s outline may not be fitted wider than the stroke it is the outline of, because a budget of half a width lets both flanks of a band a pixel across settle onto its own centreline and the band come out enclosing nothing; this set and both sets of real artwork are byte-identical again, and the movement is on the large corpus’s hairline grids, whose thinnest lines come back as bands of the width they were measured at rather than as triangles of no area.

Anyone can reproduce it: the corpus, the ground truth, the baseline and the harness are all in the repository, and docs/benchmarks/methodology.md has the commands.

What is not measured yet

JPEG is measured — every fixture re-encoded at quality 75, 210 of them — and since 0.3.15 it reads 0.53 px at p95 and 1.1× the nodes, from 7.97 px and 7.6× when the palette stage kept the codec’s per-block levels as colours. That is still not the PNG number, and what remains is named in the repository: the emoji cut, and the football’s near-white body, which its small JPEG copies lose to the paper.

Two larger synthetic sets, declared and not yet generated: standard (around 700 fixtures) and full (around 4,200, on demand). A 1024-pixel size, an icon and clip-art set, and a curated real set of 60 images scored with image-similarity metrics instead of ground truth, gradient-heavy artwork tagged out of scope rather than quietly dropped.

Two more metrics: primitive recovery, and corner precision and recall. And the comparison this page exists for — a head-to-head against a hosted service with side-by-side renders and error heat maps — which is a week-12 milestone. None of these is described above as though it exists, and none of them will be until a run produces a number with a date and a version on it.