38 releases 38 tags

RSS feed

• v0.24.0 ec09cdffe4

  Compare

  v0.24.0 — a read-only query API over the published tables Stable

  joeyr released this 2026-08-06 20:51:27 -05:00 | 686 commits to main since this release

  Phase 2 of docs/query-api-plan.md: a read-only HTTP service over the tables 0.23.0 started writing.

  Nothing consumes it yet. The crawler still listens on nothing and still commits to Forgejo; lol-companion still fetches the raw URLs.

  The service

  A second entrypoint on the same image — node dist/src/api/index.js. Three routes, each one query:

  Route	Auth	Returns
  GET /v1/health	none	liveness, plus the age of the oldest artifact
  GET /v1/manifest	key	patch, publishedAt, sha256, size per artifact
  GET /v1/artifacts/{name}	key	the whole file, byte-identical, brotli, ETag

  Built on node:http rather than a framework. This is a router with one path parameter over a service that never accepts a body, and the dependency count stays at three.

  The stored brotli is served untouched when the client accepts it, so the common path spends no CPU compressing. Measured over loopback on patch 16.15:

  Artifact	File	On the wire	Served
  synergies	7.88 MB	448 KB	48 ms
  counters	1.71 MB	110 KB	20 ms
  runes	1.32 MB	50 KB	12 ms

  A manifest poll that finds nothing changed is 1.1 KB and a 304 in 2.8 ms; a conditional GET of the 7.6 MB synergies file is a 304 in 9.8 ms.

  /v1/artifacts/{name} is also the only response that reproduces the committed bytes, because it serves payload rather than reassembling rows — Postgres normalises jsonb key order. scripts/verify-api.mjs checks exactly that, in both encodings, against each raw URL.

  Keys

  Per install, never one shared key baked into the app: an asar unpacks in minutes, and a shared key cannot be revoked without breaking every install that has not updated. Per-key issuance buys revocation, per-client rate limits and accounting. It does not buy secrecy, and nothing here needs any — these are anonymised aggregates of public match data, world-readable at the Forgejo URLs today.

  npm run key:create -- --label joey-desktop     # prints the key once
  npm run key:create -- --list
  npm run key:create -- --revoke <key_id>
  

  Secrets are stored as SHA-256 deliberately, not bcrypt or argon2: the secret is 256 bits of CSPRNG output, so there is no dictionary to run and a slow KDF would spend real CPU per request to buy nothing.

  The API gets its own Postgres role — SELECT on the published tables and api_key, UPDATE on last_used_at, no DDL and no INSERT. It never holds the crawler's DSN, so participant_stat is unreadable from the process serving HTTP rather than merely unqueried. The crawler therefore creates api_key, and the API asserts the tables exist at boot so a deployment ordering mistake reads as one sentence instead of a 500 per request.

  Two corrections to the plan, both found by building it

  • Keys parse with an anchored regex, not split('_'). base64url's alphabet contains _, so splitting on the last separator lands inside roughly half of all real secrets — and it would have surfaced as an intermittent authentication failure rather than a parse error. There is a test for exactly this case.
  • The api_key_active partial index is dropped. key_id is already the primary key.

  Fixed

  • Integration tests now run one file at a time. They share lolc_test and all truncate in beforeEach, so vitest's parallel workers had a second file deleting rows a first was midway through asserting on. Invisible until two files touched the same tables.

  Deploying

  Not deployed by this release. It needs a second service in the Swarm stack (same image, command: ["node", "dist/src/api/index.js"]), the lolc_api Postgres role, and at least one key minted in the crawler container. The stack file carries all three as comments.

  Keep it on the LAN: a bearer token over plaintext is a non-starter the moment it leaves the network.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.23.0 687b624b5e

  Compare

  v0.23.0 — the published tables Stable

  joeyr released this 2026-08-05 23:32:50 -05:00 | 699 commits to main since this release

  Added

  • Every artifact is now also published into Postgres, sliced per champion and
    role. This is Phase 1 of docs/query-api-plan.md:
    lol-companion currently downloads whole files — 12.5 MB per refresh, of which
    the 7.6 MB synergies artifact has no consumer at all — and the endgame is a
    small read-only service that answers for the champions a user actually plays.
    Nothing reads the tables yet.

    Three tables. published_artifact holds the generatedFrom block, the file's
    other top-level siblings (tierList, perks/styles, items), the sha256
    of the rendered file and the file itself brotli'd. published_entry holds one
    row per (champion, role). published_current names the patch each artifact is
    serving. One transaction per artifact, with the current pointer moved last, so
    a reader that resolves a patch can never then find its entries missing.

    champion-damage.json is stored whole and deliberately not split: its record
    has no role dimension, and the damage bars need all ten champions on the board
    anyway.

    The split is verified every night rather than trusted. sliceArtifact
    rebuilds the file from its own rows, re-renders it, and refuses to publish
    unless the bytes match what was committed. Slicing is the one operation that
    can turn a validated artifact into a wrong answer — a subset is
    indistinguishable from a complete one — and a silently dropped champion or role
    is exactly the failure the classic coverage guards exist to catch. Measured
    across all ten artifacts on patch 16.15: 565 ms of slicing, 2.6 s including the
    writes, against a three-hour window.

    Storage on that patch: 4.8 MB of entries and 960 KB of archives. Entries are
    pruned to the same two-patch window the crawl rows use; the brotli'd payloads
    are not, because they are what preserves the data/archive guarantee.

  • scripts/verify-published.mjs. Checks each row's sha256 against the file
    committed at its raw URL, which is what makes running both publish targets at
    once checkable rather than hopeful. Needs the database reachable.

  • PUBLISH_DB_ENABLED (default on) backs the database publish out without
    touching the commit. Independent of PUBLISH_ENABLED: they are two outputs
    with two audiences.

  Changed

  • The Forgejo commit is now rendered once and the bytes shared with the local
    *_OUT write and the slice, so all three are provably the same file.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.22.0 1983189b72

  Compare

  v0.22.0 Stable

  joeyr released this 2026-08-04 22:23:18 -05:00 | 725 commits to main since this release

  Added

  An items name/icon dictionary in champion-builds.json. Every measurement in that file keys on a bare Riot item id, which made it the one artifact a consumer could not render without carrying a Data Dragon catalog of its own. It now ships the display strings.

  Same shape and same reason as champion-runes.json's perks and styles: an id appears in five lists per champion-role — owned.items, owned.boots, starting, core, late — so inlining the name would write a few hundred strings tens of thousands of times.

  Why the producer publishes them

  The names belong to generatedFrom.itemVersion. The ids in this file were classified boots-or-completed under one Data Dragon version. Names resolved against a different version drift every time Riot renames or reworks an item, and nothing downstream would detect it — the build list would simply start saying the wrong thing. Icon URLs are pinned to that same version.

  That is the argument against the alternative, which was to let lol-companion bundle its own catalog: it already bundles one for the overlay's gold math, so the machinery existed. But that copy has no relationship to the version this file was built against.

  No dead entries

  The dictionary is built by walking the finished builds object rather than by tallying ids as they are counted, so it holds exactly the ids the file references and no others. Any list that stops being published stops contributing for free, and a consumer can treat a lookup miss as a bug rather than as an id it was never meant to render.

  An id Data Dragon does not name is left out rather than written blank — a missing name costs a label, dropping the row would cost a measurement.

  ---

  Additive: every existing field keeps its meaning.

  Gate green: typecheck, lint, 541 tests across 29 files, build.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.21.0 04d317bd67

  Compare

  v0.21.0 Stable

  joeyr released this 2026-08-04 21:41:12 -05:00 | 739 commits to main since this release

  Added

  order.late in champion-builds.json — the completed items bought after the core, per item, with their own record. This is what lets a consumer render a six-item build without any set of six ever having been a published key.

  The gap it fills: core stops at three items and owned.items counts everything a champion finished with, unable to tell an opener from a last pick. On the first published file, approximating "what comes after the core" from owned led Caitlyn bottom's list with Hexoptics C44 — her most common first item.

  Per item, not per set

  "Items 4-6 as a set" would split {Void}, {Void, Zhonya} and {Void, Zhonya, Banshee} into three keys, because players finish at different item counts — the unbounded-key problem this file already rejects for full builds, merely moved later in the build. One row per item has none of that.

  A third denominator, and the shape says so

  late nests inside order because its population is a subset of order's: only sampled games that reached a fourth completed item count toward late.games. A reader dividing a late row by order.games would understate every share, so the number it should be divided by sits beside it — the same device that keeps owned and order apart.

  Capped at nine

  The one capped list in this artifact. Six is the obvious number and is wrong: core and late are measured independently, so an item can honestly appear in both — bought third in one game and fifth in another — and a consumer skipping duplicates while rendering six alongside a three-item core needs three spares. Nine can never come up short.

  Order is dropped within late where core keeps it. By the fourth item a build is reacting to the game rather than following a plan, and "Void Staff then Zhonya's" versus the reverse reflects who the enemy fed.

  ---

  Additive: every existing field keeps its meaning, and lol-companion reads the file unchanged until it opts in.

  Gate green: typecheck, lint, 534 tests across 29 files, build.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.20.2 317d1dfab7

  Compare

  v0.20.2 Stable

  joeyr released this 2026-08-04 21:14:24 -05:00 | 743 commits to main since this release

  Fixed

  Marksmen publish boots again. parseItemCatalog decided boot-ness from Data Dragon's Boots tag, and Data Dragon does not tag the whole tree: on 16.15.1 Gunmetal Greaves (3172) — the tier-3 every attack-speed champion upgrades into — is tagged ["AttackSpeed","LifeSteal","NonbootsMovement"] while all eight of its siblings carry Boots. It was therefore not a boot, and at 1,100g it was under the completed-item gold floor, so it was not a legendary either. It fell through both filters and disappeared from the artifact entirely.

  Visible in the first published file: 56 of 300 champion-roles had an empty boots list, Caitlyn bottom among them.

  Boot-ness is a fact about the build tree, so it is now read from the build tree — the transitive closure of into from every Boots-tagged Rift item. Over 16.15.1 that closure adds exactly 3172 and pulls in nothing priced at or above the completed floor, so no legendary is reclassified by widening it.

  Classification happens at build time from raw stored ids, so this repairs every row already in the database. No re-crawl — the next publish carries the boots.

  champion-builds.json no longer overstates the owned half's sample. Its provenance matches came from matchCount, which counts every match on the patch — including those crawled before the items column shipped in 0.17.0, which carry the empty sentinel and back none of the owned percentages. On 16.15 that reported 10,133 against a real 152.

  New Store.itemMatchCount counts through the same ITEM_ROW_FILTER the ownership aggregate runs through, so the number and the rows behind it agree. The other five artifacts keep matchCount, which is correct for them.

  ---

  Gate green: typecheck, lint, 525 tests across 29 files, build. Boots closure verified against live Data Dragon 16.15.1 (boots 19 → 20, no legendary reclassified).

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.20.1 cec7291a73

  Compare

  0.20.1 Stable

  joeyr released this 2026-08-04 20:40:02 -05:00 | 752 commits to main since this release

  Fixed

  • The owned half of champion-builds.json no longer scales with
    participants. itemInventoryRows grouped on the whole seven-slot inventory
    and returned one row per distinct bag, on the assumption that identical bags
    would collapse. They do not: a finished inventory is very nearly unique —
    which is the same fact that makes six-item builds unpublishable, so the
    argument was already written down three files away. The grouping therefore
    compressed almost nothing, and a late-patch table (~30,000 matches) would have
    handed ~300,000 array-bearing rows to a 512 MB container.

    Replaced by itemOwnershipRows, which unnests and tallies in SQL. The result
    is bounded by the thing that really is bounded — ~274 champion-roles times the
    few dozen items each is seen with — so it is roughly 14,000 rows regardless of
    how long a patch runs.

    No artifact change: the published file is byte-identical, and classification
    still happens at build time against Data Dragon.

    Caught before the first deploy of 0.20.0, so no run was affected.

  Notes

  • Deploy this rather than 0.20.0. 0.20.0 is correct but would have degraded
    as the patch accumulated rows.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.20.0 0ab0641d96

  Compare

  0.20.0 Stable

  joeyr released this 2026-08-04 16:49:16 -05:00 | 773 commits to main since this release

  Added

  • data/champion-builds.json — what players bought, and in what order. A
    new published artifact and a new URL, and the first one in this service that
    costs additional Riot requests.

    It carries two measurements from populations an order of magnitude apart,
    and the entry splits so each states its own games and wins:

    • owned — the final inventory, from every recorded match (~1,700 a
      night). item0–item6 have been captured since 0.17.0 and cost nothing.
    • order — the purchase sequence, from the ~12% of matches a match-v5
      timeline was fetched for.

    A single games would let a reader compare a 1,700-game figure against a
    200-game one without noticing. order.games: 0 is a real state, not a hole:
    no sampled match contained that champion-role.

    Completed items and boots are reported separately, because a boot is a build
    slot rather than a legendary. Classification comes from Data Dragon's
    item.json at build time, never at capture — the version used is published as
    generatedFrom.itemVersion.

    Full six-item builds are deliberately not published. At ~100 games per
    champion-role nearly every complete build observed is unique, so publishing
    them all is ~1.5 MB of games: 1 rows nobody can use. That is not the
    filtering CLAUDE.md forbids — that rule is about dropping measured rows from a
    bounded aggregation, and this is declining to publish an unbounded one,
    the same reasoning that makes a rune combo a tree pair rather than an exact
    nine-perk page.

  • Timeline sampling, and participant_stat.item_order / item_times to
    hold what it collects. TIMELINE_MAX_REQUESTS (200) and
    TIMELINE_TARGET_GAMES (100), plus BUILDS_ENABLED / BUILDS_PATH /
    BUILDS_OUT and a builds row in publish_state.

    The spend is targeted, not random. The match is fetched first, so its ten
    champion-roles are known before the decision; a timeline is fetched only while
    the least covered of them is under target. Champion-role popularity spans
    two orders of magnitude, so a uniform draw would collect a nine-hundredth Jinx
    build while Ivern jungle sat at four.

    Participants are joined to the timeline by puuid, never by position —
    the documented positional correspondence would silently credit each champion
    with somebody else's build if it ever changed. The puuid is read and
    discarded; the privacy rule that no puuid is ever written is unchanged, and a
    test asserts it.

  Changed

  • The nightly match count drops by up to ~12%, and this is the only setting
    in the service that does that. A timeline is a second request against the same
    key, so 200 of them is 200 matches not crawled — and every artifact reads
    those rows. TIMELINE_MAX_REQUESTS is a hard count rather than a ratio for
    exactly that reason.

    TIMELINE_MAX_REQUESTS=0, or BUILDS_ENABLED=false, gives the budget back.
    The first keeps the owned half publishing on its own; the second stops both.

  Notes

  • One assumption in the timeline parser is documented rather than observed.
    ITEM_UNDO is implemented per Riot's documented convention — the bought item
    in beforeId, afterId: 0 — but no captured payload with a real undo could
    be checked, because the key available at implementation time returned 403. So
    the crawl counts and logs unmatchedUndos per window instead: it should be
    ~0, and if it tracks the undo count then the fields are the other way round
    and undone purchases are being published as real ones. Worth reading on the
    first night this runs.
  • Watch the match count on the first night, since this is the release that
    trades it. It should fall by about 12% and no more; the cap is hard.
  • No consumer change is required. lol-companion does not fetch this URL.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.19.0 8ebf39aa17

  Compare

  0.19.0 Stable

  joeyr released this 2026-08-04 16:30:32 -05:00 | 775 commits to main since this release

  Added

  • data/champion-synergies.json — a new published artifact, and a new URL.
    How a champion's record at one role varies with who they were on a team with,
    modelled on op.gg's /lol/champions/<champion>/synergies/<role> page.

    It is the counters self-join with the predicate inverted: a different role on
    the same side of the win partition is an ally. Every participant has four
    of them, so the same crawl yields 40 rows per match here against counters' 10.
    No additional Riot requests, no op.gg pages, no new column.

    Nothing consumes this yet, and nothing has to. A new URL nobody fetches
    strands nobody, so it publishes now and lol-companion can mirror the schema on
    its own schedule. The usual mirror-first ordering binds from the moment a
    consumer ships.

    Two things a reader must get right:

    • Entry totals are not the sum over allies. Each game contributes four
      ally rows, so the sum is four times the champion-role's real record. The
      entry's games/wins/winRate are measured separately and are the
      champion-role's actual record — genuinely useful in their own right, and
      not the truncated-sum convention CounterEntrySchema documents.
    • Every symmetric pair is stored twice. (Evelynn JUNGLE, Lux UTILITY) and
      (Lux UTILITY, Evelynn JUNGLE) carry identical numbers by construction. The
      duplication matches how the file is read at champ select; halving it under a
      canonical ordering is a lever held in reserve.

    Size, measured at ~93 bytes per cell under the compact printer: ~3 MB at
    half occupancy, ~5.3 MB saturated, over a cell space of ~274 champion-roles
    x ~220 reachable ally champion-roles. Larger than the plan's 2 MB guess, and
    shipped as designed anyway — the same call, and the same standing compaction
    lever, as champion-runes.json going 532 KB → 2.56 MB across a patch.

  • SYNERGIES_ENABLED (default on), SYNERGIES_PATH, SYNERGIES_OUT, and a
    synergies row in publish_state so it gets the same staleness alarm as
    every other artifact. No coverage floor and no wall-clock budget: like the
    ranked rune and counters builds this is a database query, not a page crawl.

  Notes

  • No consumer change is required. lol-companion does not fetch this URL and
    is unaffected until it opts in.
  • The file grows across a patch as cells fill, in the same way runes does. Set
    SYNERGIES_ENABLED=false if the size becomes a problem before the dedup lever
    is taken.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.18.0 84c29e9b08

  Compare

  0.18.0 Stable

  joeyr released this 2026-08-04 16:18:52 -05:00 | 777 commits to main since this release

  Changed

  • champion-counters.json is now built from Riot match-v5, not scraped from
    op.gg. generatedFrom.source reads riot. No schema change and no consumer
    change: source has always been z.string().min(1), "present so a future
    second source is a value, not a fork".

    Matchups come from a self-join of participant_stat — two participants in the
    same match, at the same role, on opposite sides of the win partition. It
    needed no capture phase, unlike runes and items: every column it reads has
    been recorded since the table existed, so the swap built a complete file from
    rows already on disk. Zero additional Riot requests; ~271 fewer op.gg pages a
    night.

    What changes downstream is the sample. op.gg's file was built from a
    population and a window nobody here could see — roughly 192,000 matches on
    16.15, against the ~12,000 this service had collected at the same point. Ours
    is smaller, it is the games this crawl actually observed in the seeded ladder
    band, and it grows for as long as the patch runs. Every cell carries its own
    games.

    Two differences worth expecting rather than debugging: games per matchup
    is roughly an order of magnitude lower and climbs across a patch instead of
    arriving whole, and mirror matchups are absent — the crawl is queue 420,
    where draft pick makes champion selection exclusive across both teams, so
    Yasuo never faces Yasuo.

  • tierList still comes from op.gg, and it is now the only thing that does
    for ranked. It is op.gg's {tier, rank} bucketing, which is a score;
    deriving one here by sorting champion-roles on win rate would be inventing a
    scoring function and publishing it as though measured. Five tier-list pages a
    night, down from 276.

    A night op.gg cannot be reached publishes five empty arrays, never
    yesterday's list — the file is judged as a whole on generatedFrom.patch, so a
    stale tier list inside a current-patch artifact reads as current. The matchups
    publish as normal, and lol-companion's TierListProvider already rescrapes a
    role it finds empty.

  • COUNTERS_ENABLED now gates both sources, ranked and classic, exactly as
    RUNES_ENABLED does. COUNTERS_MIN_COVERAGE and COUNTERS_MAX_MINUTES are
    classic-only from this release: they describe a page crawl, and the ranked
    build is a database self-join.

  • One op.gg client for the whole window. The ranked tier lists and the
    classic pass used to be sequential calls that each built their own; the ranked
    half is now consumed by the Riot half, so they can overlap. Sharing one client
    means one limiter and one pacing, rather than two limiters at twice the rate.

  Removed

  • rankedPass in index.ts. The only op.gg pass is classic — the mechanism,
    deliberately, rather than a flag that could put ranked counters back on op.gg.
    src/counters.ts and the op.gg page extractors are untouched and still live:
    queue 4310 is absent from match-v5, so op.gg remains classic's only source.

  Notes

  • No consumer change is required or expected. The schema is byte-identical
    and lol-companion's mirror parses source: 'riot' today.
  • The first published file after this deploys describes whatever the crawl has
    stored for the live patch. Mid-patch that is a full file; on patch-drop
    morning it is a thin one, correctly labelled — where the op.gg build used to
    abort on its coverage floor and leave yesterday's file live instead.
  • New index participant_stat_pair_idx (patch, match_id, role), applied at
    boot. It serves the self-join; the existing rollup index leads on champion and
    cannot.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.17.0 ecce9f0801

  Compare

  0.17.0 Stable

  joeyr released this 2026-08-04 16:04:09 -05:00 | 778 commits to main since this release

  Added

  • The crawl now records each participant's final inventory. item0–item6
    are in every match payload this service already fetches, so capturing them
    costs no additional Riot requests — the same argument that made benchmarks and
    runes cheap. New participant_stat.items INTEGER[] column, populated for all
    ten participants of every match.

    Nothing is published from this yet. It ships alone and first because it is
    the only part of the item work whose cost is wall-clock: retention keeps two
    patches, so an inventory not captured while its patch is live cannot be
    recovered afterwards. Deploy it the day it merges; the artifact that reads it
    follows later.

    All seven slots are stored raw, empty slots and trinket included. Which id
    is a legendary, a boot or a ward is a Data Dragon judgement made when an
    artifact is built, and Data Dragon moves — classifying at capture time would
    mean a re-crawl after every item rework, from rows that no longer exist. The
    zeros are also the sentinel: a captured row is always seven integers, so
    items = '{}' identifies a row collected before the column existed, exactly
    as primary_style = 0 does for runes.

  Notes

  • The migration is an ADD COLUMN IF NOT EXISTS with a default, applied at
    boot alongside the existing ones. No downtime, no backfill, and rows written
    before it carry the empty-array sentinel.
  • No consumer change. No published artifact reads this column yet.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads