38 releases 38 tags

RSS feed

• v0.29.3 167b3e1d36

  Compare

  v0.29.3 Stable

  joeyr released this 2026-09-30 20:31:36 -05:00 | 216 commits to main since this release

  Fixed

  • An HTML page in place of Riot's JSON fails at once. A 200 carrying an HTML
    page — something in the path answering in Riot's place — went through the
    same retries as a truncated body: six requests and half a minute spent
    fetching the same page again, then a failure anyway. It now fails on the first
    attempt as match: HTML page instead of JSON (HTTP 200), and since 0.29.2
    that costs the one item. Only text/html is turned away. Any other content
    type is still parsed, so a mislabelled but valid response cannot fail every
    request of the night.

  • Riot responses the client does not read are released. A 404, 429, 5xx or
    other error response was never read or cancelled, which keeps its connection
    out of undici's pool until garbage collection gets to it — and 404s are
    routine here. Every body the client does not read is now cancelled at once.

  Changed

  • The Riot client tests build their client through one helper. Each test
    names only the options it changes instead of repeating the whole constructor
    call. No behaviour change.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.29.2 4491f3736e

  Compare

  v0.29.2 Stable

  joeyr released this 2026-09-30 20:12:49 -05:00 | 217 commits to main since this release

  Fixed

  • A Riot call that fails for good costs its item, not the night. A seed
    page, a seed's match list or a match that still fails after the client's
    retries is now logged and skipped, and the crawl moves on. Before, any such
    failure — a network drop longer than the retry backoff, a run of 5xx, a 400 on
    one match id — threw out of the crawl and cost all six Riot artifacts the
    night, although every row already stored was good. A failed match is not
    marked seen, so a later night whose lookback still covers it can try again.
    Failures are counted as failures on the crawl's summary line.

    Two things still end the crawl. A 401 or 403 throws as before: a bad key fails
    every call alike and is a fault on the first night. And
    MAX_CONSECUTIVE_FAILURES (20) in a row — after retries, ten minutes or more
    in which nothing succeeded — stops it with stoppedBy: 'failures', and the
    builders publish from what was stored. If nothing was, all six artifacts
    record a fault naming the last failure, not the quiet "no new rows" of a night
    that simply deduplicated to nothing.

  • Riot errors name the endpoint that failed. A request that died in transport
    after its retries surfaced undici's bare terminated, which is all a
    publish-state reason could say. It is now a RiotApiError reading
    match: transport failure after retries (terminated), with the socket error
    kept as cause, and every Riot error opens with its endpoint the same way —
    league-exp, match ids, match or timeline. Never the URL: these messages
    reach the repo's commit status, and a match-list URL carries a puuid.

  • TIMELINE_MAX_REQUESTS counts requests, not timelines. A timeline that
    needed retries — after a 429, a 5xx, or a body that died mid-read — spent up to
    six requests and was counted as one, and a timeline that failed was not
    counted at all. The budget now adds what each call actually spent, logged as
    timelineRequests. The night's last timeline can still run over by its own
    retries, since those are not known until the call ends.

  • Shutdown no longer waits out the Riot client. Its retry backoff, a 429's
    one-minute yield and the pacing after one all ignored SIGTERM, so a redeploy
    could sit in a wait longer than Swarm's stop grace period and be killed
    mid-match instead of stopping cleanly. Those waits now end on shutdown and the
    crawl stops as signal. A request already in flight is still left to finish.

  • A non-numeric Retry-After no longer skips the 429 backoff. The header may
    be an HTTP-date, which made the wait NaN and the sleep zero. It now falls
    back to the full 60-second yield.

  • A 429 on the last attempt throws at once. It used to wait out the yield
    before throwing anyway — a minute or more for nothing, and at boot for the
    one-shot preflight probe. The rate-limit penalty is still applied.

  • Retried transport failures are logged. Each retry now logs a warning with
    the URL, the attempt and the error, so a reset the retries recovered from is
    visible rather than indistinguishable from a clean night.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.29.1 e0fe88644e

  Compare

  v0.29.1 — retry a Riot response whose body dies mid-read Stable

  joeyr released this 2026-09-10 20:29:23 -05:00 | 416 commits to main since this release

  Fixed

  A Riot response whose body died mid-read walked straight past the retry.

  RiotClient.get wrapped only the fetch call in the block that retries transport failures, and read the body on the line after it. fetch resolves once the headers arrive and leaves the body streaming, so a connection dropped partway through the body rejects from res.json() — outside the guard, and therefore out of get entirely, past five perfectly good retries.

  The two phases are distinguishable and only one was ever covered: undici raises TypeError: fetch failed when the connection never came up, and TypeError: terminated (cause: read ECONNRESET, thrown from Fetch.onAborted) when it came up and then went away mid-body. Every crawl since the client was written has been exposed to the second.

  It surfaced on the window of 2026-09-10: one reset roughly four hours in threw past the crawl, and all six Riot artifacts recorded terminated against the same instant while the op.gg half — already finished at 02:01 — published its four classic files normally.

  The body read moves inside the guard, so a mid-body reset is retried with the same bounded backoff as any other transport fault. Replay is safe because everything get issues is a GET.

  No data changes and nothing needs re-crawling — the artifacts the failed window skipped are rebuilt by the next one from the retained rows.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.29.0 67c5901f05

  Compare

  0.29.0 — two-item core Stable

  joeyr released this 2026-09-02 20:46:01 -05:00 | 496 commits to main since this release

  Changed

  order.core is the first two completed items instead of the first three. CORE_ITEM_COUNT is the only boundary between core and late, and both are slices of one walk over the purchase sequence, so nothing is dropped — the third item moved into late, which keys per item rather than per sequence and therefore does not split.

  A core sequence is an ordered key, so its domain grows with every slot it carries and the sample behind each published row shrinks. Measured on the 16.17 file over the 81 champion-roles with 50 or more sampled build orders:

  core length	distinct sequences	modal row's share	median games behind it	top four rows cover
  3 (0.28.0)	36.3	18.2%	12	44.4%
  2 (0.29.0)	20.1	37.6%	27	69.1%

  Since lol-companion renders the top four rows, the build strip goes from describing under half of the sampled games to roughly seven in ten, and the games figure printed on the core block roughly doubles.

  order.late.games widens with it. Its denominator was sampled games that reached a fourth completed item and is now those that reached a third, so the situational half is measured over more games as well.

  Nothing is re-crawled for this. item_order is stored raw and classified at build time, so the next nightly run rebuilds the file with two-item cores over the whole retained window.

  Unchanged

  • LATE_ITEM_CAP stays at 9. A consumer rendering six unique items has to skip a late row for each core item it already displayed, so the cap has to clear six plus the core length; a shorter core only widens that margin.
  • lol-companion needs no change. Its builds adapter maps whatever itemIds the row carries, and its Situational block was already sized at six.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.28.0 b463527cc1

  Compare

  v0.28.0 Stable

  joeyr released this 2026-09-01 19:30:36 -05:00 | 507 commits to main since this release

  Changed

  • Benchmarks, counters, synergies and builds are built over both retained
    patches instead of one. Retention has kept the current patch and its
    predecessor since the table existed, and every aggregate filtered
    WHERE patch = $1 and walked past rows a night's crawl had already paid for.
    Those four now read WHERE patch = ANY($1::text[]) over the whole window.

    The gain is lopsided in time rather than a flat doubling. The predecessor is a
    finished crawl — ~26,000 matches on 16.15 — while the current patch holds
    whatever the nights since rollover collected: roughly 14x the sample on day
    one, ~4x by day four, ~2x by the end of a patch. On counters, whose median
    matchup cell was two games, that is the difference between publishing noise
    and publishing something a consumer can put a floor under.

    It does not make counters well-sampled. It roughly doubles a median of two.
    MATCH_LOOKBACK_HOURS and the crawl window are where the rest of that
    problem lives.

  • champion-damage and champion-runes deliberately did not change. Damage
    splits are ratios that converge in a few dozen games, and pooling would blur
    the item reworks anyone reads a damage split to find — it keeps its per-entry
    fallback. Runes are the best-sampled artifact here, because a full nine-perk
    page rides free on every match-v5 row, and the fastest-moving under a balance
    patch; they stay on the current patch under either retention setting.

  • RETAIN_PREVIOUS_PATCH=false is now a quality decision, not a storage one.
    It used to cost one thinner morning per patch. It now roughly halves the
    sample behind four artifacts for the whole patch. Its doc comment in
    config.ts, the README's "The two-patch window" and CLAUDE.md all say so.

  • generatedFrom.previousPatch is published on counters, synergies and
    builds. Required and nullable here; optional and nullable in lol-companion's
    mirror, which shipped first in 4.2.0 — those schemas are .strict(), so a
    producer that published the field first would have every install reject the
    file and freeze on cached data. generatedFrom.patch is untouched and still
    names the patch the file is for.

    Publishing the window without naming it would have left three files quietly
    claiming a one-patch sample they no longer have.

  • stale on a benchmarks bucket now means "the current patch has measured
    nothing here yet." It used to mean "served from the previous patch because
    this one had nothing", which pooling makes meaningless — every bucket may hold
    predecessor rows. The producer reads it off a new currentN column on
    benchmarkRows rather than off which of two queries the row came from, so the
    patch-day signal survives with its meaning intact and lol-companion's existing
    labelling stays correct. The damage artifact's identically named flag still
    marks a genuine either/or fallback; the two are no longer the same signal.

  • Benchmarks reversed "one fresh sample beats a stale thousand" on purpose.
    The fallback preferred a single current-patch row over five thousand beside
    it. A benchmark is a habit, and 5,001 games describe a habit better than the
    one that happens to be on this patch. The pooled mean is weighted by n, so
    the fresh sample counts for exactly what it is worth. The rule still holds for
    damage, and CLAUDE.md now scopes it to that artifact.

    Pooling is also the only lever that reaches a thin bucket without withholding
    data: raising BENCHMARK_MIN_SAMPLES does not improve a three-game bucket, it
    deletes it, and what is never published cannot be recovered downstream.

  • buildOrderCoverage stays on the current patch while the builds artifact
    pools. It steers which champions the crawl spends its rationed timeline
    requests on, and counting the predecessor's orders as coverage would starve
    the current patch — leaving the pooled file ever more predecessor-weighted
    with each night.

  Added

  • Nine integration tests for the pooled window, against real SQL, in
    test/integration/store.test.ts. Two of them are the correctness properties
    the change turns on and that no unit test can reach: that the matchup and
    synergy self-joins key on b.patch = a.patch, so a widened window sums two
    independent within-patch tallies rather than fabricating pairs between players
    who were never in a game together; and that synergyRows widens its pair
    numerators and its totals denominators together, since widening only the
    first would halve every rate in the file while still validating.

  Known risk

  • A pooled build order can silently lose an item Riot removed between the two
    patches. Purchases are stored raw and classified at build time against
    current Data Dragon, so coreSequence drops an id the catalog no longer
    carries — leaving a hole in a sequence presented as observed. Removals cluster
    at season boundaries, so this is rare, but it fails quietly rather than
    loudly. Documented on RiotBuildsOptions.previousPatch.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.27.1 08dbbfa197

  Compare

  v0.27.1 — the undo denominator Stable

  joeyr released this 2026-08-31 22:18:16 -05:00 | 518 commits to main since this release

  Instrumentation only — how undos are applied is unchanged.

  The crawl's window line now logs undos beside unmatchedUndos.

  unmatchedUndos is the tripwire on the one ITEM_UNDO assumption this service cannot verify: that an undone purchase carries the bought item in beforeId with afterId: 0. It was never checked against a captured payload, because the key available when the parser was written returned 403.

  Until now only the numerator was logged. The window of 2026-08-31 closed with unmatchedUndos=271 over 200 timelines — which is either a handful of edge cases out of thousands of undos, or every undo in the window with the convention the other way round and undone purchases being published as real ones. Nothing in the line could tell those apart.

  TimelineStats already counted undos per timeline; the crawl summed only the unmatched half and dropped it. It now carries both, so the next window answers a question that has been open since the parser shipped. Read them as a pair: an undone sale lands in unmatchedUndos legitimately, so a small non-zero floor is expected — what matters is whether the two counts track each other.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.27.0 c0cc509648

  Compare

  v0.27.0 — RETAIN_PREVIOUS_PATCH Stable

  joeyr released this 2026-08-10 21:02:59 -05:00 | 622 commits to main since this release

  Added

  • RETAIN_PREVIOUS_PATCH=false narrows retention to the current patch.

    The default is unchanged (true, the two-patch window README describes), so
    nothing moves for a deployment that does not set it.

    It is one flag with two effects, and they cannot be separated: it sets the
    window pruneToPatches enforces AND whether previousPatch reaches the
    damage and benchmarks builders as their thin-sample fallback. Pruning the rows
    while still naming the predecessor would publish
    generatedFrom.previousPatch: "16.14" over four queries that return nothing —
    provenance for a fallback that cannot fire. runRiotWindow binds it once, as
    retained, and everything downstream reads that rather than the parameter.

    What switching it off costs is one morning per patch: on rollover, a champion
    or role with no rows yet on the new patch is omitted from damage and
    benchmarks instead of being served from the previous patch flagged stale.
    Only those two artifacts have a fallback to lose — counters, runes, synergies
    and builds are current-patch-only either way — and neither data/archive nor
    the brotli'd published payloads are pruned under either setting.

    It also forgets the previous patch's seen_match ids, so rollover morning
    re-fetches some matches it already knows are off-patch and discards them
    again. Bounded, and once per patch.

    The deployed stack sets it, alongside a crawl window widened from three hours
    to six: 6-hour nights collect roughly twice the rows, and holding two patches
    of that on the shared Postgres is ~1M participant_stat rows. The wider
    window is also what makes the trade cheap — most champion-roles now have rows
    by the first morning, so the fallback rarely fires at all.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.26.0 b7ef272b91

  Compare

  v0.26.0 Stable

  joeyr released this 2026-08-08 14:04:31 -05:00 | 646 commits to main since this release

  Added

  • The crawl sets the repo's commit status, so the icon is never blank.

    Forgejo's repo list shows a tick or a cross per repo, read from the commit
    status on the tip of the default branch. Every other repo gets one from CI.
    This one never did: its tip is a nightly data commit essentially every day,
    and those carry [skip ci] and are excluded by paths-ignore on purpose, so
    no run exists to report anything.

    Blank is the worst of the three states. A cross says the crawl broke, a tick
    says it did not, and blank says only that nobody asked — which is exactly what
    a service that died a week ago also looks like.

    src/commit-status.ts now posts one status per window under the context
    crawl/nightly:

    INF publish/status > commit status set to success | sha=b6b948e…
      context=crawl/nightly description="10/10 artifacts published"
    

    It stamps whatever is on top of the branch, not a commit the run made. The
    nights worth flagging are precisely the ones that commit nothing — the crawl
    died before any artifact was reachable, the bytes were identical, or five
    published and the sixth threw and left the tip looking healthy. Statuses are
    keyed by (commit, context) with the newest winning, so re-marking last night's
    commit is what lets a run that wrote nothing still report.

    Severity decides the colour. Red is what publish-state.ts already calls
    ERROR — a thrown fault on the first night, a decline once it passes
    STALE_AFTER_NIGHTS. A single missed night stays green and says so
    (9/10 artifacts published, 1 declined), because a coverage abort on
    patch-drop morning is expected, and a cross that appears every patch cycle is
    a cross nobody reads.

    It cannot cost a window: every path out resolves, and a failed post logs under
    the new publish/status phase and is dropped. No new credential — push
    permission on the repo is what the contents API already needs, and the boot
    preflight already proves the token has it.

    A process killed outright still leaves yesterday's tick in place; closing that
    needs an observer outside the process and is not in this release.

  Changed

  • PublishTracker takes an optional onOutcome observer, which is how the
    status reporter learns what each artifact did. A store failure now emits an
    ERROR outcome rather than nothing at all — a Postgres outage that produced
    no outcomes would otherwise read downstream as a night nobody attempted.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.25.0 5e2f858048

  Compare

  v0.25.0 Stable

  joeyr released this 2026-08-07 21:42:54 -05:00 | 668 commits to main since this release

  Added

  • The query API proves its dependencies at boot instead of announcing its
    configuration. Phase 4 of docs/query-api-plan.md.

    Until now the service said one thing about itself when it started:

    INF api/config > lol-companion-data api started | version=0.24.1 commit=7ef8062
      listen=0.0.0.0:8080 database=postgres:5432/lolc_damage rate=120/min burst 60
    

    Every field on that line is configuration read back. Not one is evidence that
    anything works. The crawler's block has been the opposite since 0.19.0 —
    server=, entries=205, patch=16.15 are all answers something gave it —
    and the argument for one here is stronger, not weaker: the crawler's bad
    moment is an unattended window at 02:00, this one's is a 500 served into champ
    select, to a client on a draft timer.

    Six checks, one round trip each, run concurrently:

    INF preflight/db         | ok  server=PostgreSQL 18.4
    INF preflight/role       | ok  role=lolc_api
    INF preflight/schema     | ok  tables=4
    INF preflight/grants     | ok  select=true update=last_used_at
    INF preflight/keys       | ok  active=1 revoked=0
    INF preflight/artifacts  | ok  count=10 expected=10 patch=16.15 oldest=19h
    

    preflight/role is the one that matters most, and the crawler has no need
    for it. The whole read-only design rests on this process connecting as
    lolc_api rather than the crawler's lolc — that is what makes
    participant_stat unreadable rather than merely unqueried. Nothing already
    logged can tell you which it got: databaseLabel strips the credentials from
    the DSN on purpose, so a stack file edited in a hurry could point this process
    at the crawler's role and every line would look exactly as it does now. It has
    to be asked, so it is: SELECT current_user, compared against the new
    API_DB_ROLE, which defaults to lolc_api so a stack file that omits it
    is still checked. Running against a development database as lolc means
    setting it deliberately.

    preflight/keys is the cheapest real outage this catches. An API with
    zero active keys boots clean, answers /v1/health, and 401s every real
    request — which from the client's side is indistinguishable from a client bug.

    preflight/grants probes the write it never makes at boot. The
    last_used_at UPDATE happens on a flush timer minutes later, so a missing
    grant would otherwise surface as a background failure long after the deploy
    that caused it. UPDATE api_key SET last_used_at = last_used_at WHERE false
    matches no rows and writes nothing; Postgres checks table and column
    privileges when the statement starts regardless, so the refusal is identical
    to the one a real flush would get.

    Two of the six are fatal, and that is the one way this differs from the
    crawler's block. preflight.ts is emphatic that nothing may stop the
    crawler starting — a dead Forgejo token must not cost a night of Riot data a
    fixed token could publish tomorrow. A read service has no equivalent argument:
    it holds no work in progress, so refusing to start costs nothing that starting
    broken would not cost more of. schema is fatal because a read service
    against an unmigrated database 500s forever and cannot fix itself; role is
    fatal because it is a security boundary rather than a dependency, and
    degrading gracefully into an HTTP surface with SELECT on participant_stat is
    not degrading gracefully. Grants, keys and artifacts stay loud and non-fatal —
    each is fixable while the process runs.

  Changed

  • assertSchema stopped being the reporter for faults it knows nothing
    about. It was the first statement to touch the database, so every
    connection-level failure surfaced there regardless of cause — which is how
    0.24.0's malformed DATABASE_URL came out as a TypeError from four frames
    inside pg while its careful "the crawler owns these tables" message was
    never reached. It is now one check of six, behind preflight/db, and only
    ever speaks about tables.

  • The shape of a boot check moved to src/check.ts, shared by both
    processes. CheckOutcome, CheckStatus, the 10-second per-check timeout and
    the preflight/summary line are now defined once. The two blocks are read in
    the same Portainer viewer, often one above the other, so a parallel structure
    would have drifted into two timeouts, two summary lines and two spellings of
    "failed". What is deliberately not shared is fatality — that is a policy
    about the checks rather than their shape, so logCheckBlock takes the summary
    wording instead of deciding it.

  • ApiStore.ping() returns the server version rather than nothing, matching
    Store.ping(), so both satisfy one check interface and the boot line can say
    server=PostgreSQL 18.4 instead of proving a query returned.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.24.1 7ef8062409

  Compare

  v0.24.1 — fail at boot on an unparseable DATABASE_URL Stable

  joeyr released this 2026-08-06 21:30:37 -05:00 | 684 commits to main since this release

  Fixed

  A DATABASE_URL that Postgres cannot parse now fails at boot, naming the cause.

  Both Swarm stack entries build their DSN by interpolating a password into a URL, and a password holding a literal /, # or ? ends the authority early — the port becomes a non-numeric string and the URL is rejected. The config only asked for a non-empty string, so it loaded clean and the API started, then died on its first query four frames inside pg:

  ERR fatal | error=TypeError: Invalid URL
      at parse (/app/node_modules/pg-connection-string/index.js:30)
      at ApiStore.assertSchema (file:///app/dist/src/api/store.js:73)
  

  Nothing in that names the variable at fault or the reason, and the stack points at library code.

  DATABASE_URL is now validated in both loadConfig and loadApiConfig against pg own parser rather than a new URL lookalike — the question worth answering is whether new Pool() will accept the string, and a check that can drift from the real one is not that question.

  Two things beyond it parsed are also required, because parse cannot fail on a stray word: it resolves relative strings against a base URL, so x comes back as host base. A scheme is what distinguishes a DSN, and a non-empty host is what distinguishes postgres:///db, which parses fine and then fails as a refused connection. Both unix-socket forms still pass.

  The message names the variable and the offending characters and never quotes the DSN, which carries the password inline.

  pg-connection-string moves from a transitive dependency of pg to a declared one, since it is now imported directly. It adds no install weight.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads