38 releases 38 tags

RSS feed

• v0.8.0 d5e5b262b3

  Compare

  v0.8.0 Stable

  joeyr released this 2026-07-30 23:45:01 -05:00 | 829 commits to main since this release

  Changed

  • op.gg is now paced at one page per 12.5 s, taking the nightly crawl from
    ~14 minutes to ~2.5 hours. Derived rather than picked: the window makes 714
    requests (5 tier lists + 271 counters + 271 runes for ranked, plus 5 + 81 + 81
    for classic), and 9000 s / 714 ≈ 12.6 s.

    The old 1.5 s pacing walked the whole universe in a burst, and
    champion-counters was aborting at 55% coverage — the shape you get when
    op.gg starts serving its 403 challenge page partway through. A 403 is
    deliberately not retried, so once it starts, every remaining page fails
    instantly and coverage collapses to however far the crawl got. This does not
    prove that diagnosis, but bursting was the one thing under our control.

  • COUNTERS_MAX_MINUTES and RUNES_MAX_MINUTES raised 45 → 70, and they are
    coupled to the interval. At 12.5 s a 271-pair crawl needs ~57 minutes, so
    the old 45-minute cap would have stopped it two thirds through and then
    aborted it on coverage — turning a pacing change into a permanent outage.
    Raise the caps with the interval or not at all.

  • CLASSIC_ENABLED now defaults to OFF. Every other artifact is opt-out;
    this one is opt-in. It adds ~167 requests to a window whose ranked half is
    already failing for reasons nobody has established, and cumulative volume is
    the last thing that should grow by default while that is open.

  Notes

  The crawl window probably needs widening, and this release does not do it.
  CRAWL_START_HOUR=2 / CRAWL_END_HOUR=5 is three hours. An op.gg half taking
  2.5 h of that leaves the Riot crawl ~30 minutes where it used to have ~2h45 —
  and champion-damage and champion-benchmarks are both built from those rows,
  so they would thin out night over night. 02:00–07:00 gives each half ~2.5 h.
  That is a deployment decision, so it is flagged here rather than changed.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.7.0 73e0b261ae

  Compare

  v0.7.0 Stable

  joeyr released this 2026-07-30 23:40:15 -05:00 | 830 commits to main since this release

  Added

  • champion-runes-classic.json is published. v0.6.0 shipped the extractor
    with nothing wired to it; the classic pass now crawls rune pages alongside
    counters and publishes them.

    Its own schema, not the modern one with a flag. Classic Rift restores the
    pre-2017 rune system, which is a different data model rather than a variant:
    no keystone, no primary/secondary path, no shard rows, no selectedPerkIds —
    the things champion-runes.schema.ts is built around. A page is a flat
    multiset of four rune kinds held in quantities. Forcing it into the modern
    shape would mean inventing empty paths or making half that schema optional,
    which would weaken the validation currently catching drift on the ranked file.

    No perk dictionary either. The modern artifact hoists display strings because
    inlining measured 1.9 MB against 212 KB across ~271 pairs of ~36-option
    boards; classic is ~81 pairs carrying four runes each, so inlining costs a few
    KB and removes a rehydration step from the consumer.

  Fixed

  • The classic artifacts no longer claim a rank band they do not have.
    op.gg accepts the tier parameter for classic pages and discards it —
    iron_plus, challenger and a deliberately invalid value all return
    byte-identical rows. Recording the band the crawl asked for would be a false
    provenance claim, and anything comparing classic to ranked figures would
    silently compare a gold+ population against an everyone population. The files
    now record all.

  Notes

  Champion ids are re-keyed on the way out, and this is the subtle one. op.gg
  indexes classic CHAMPION pages by BASE id — the tier list returns 24 for Jax and
  the page lives at /champions/jax, while jade_jax 307-redirects — but champ
  select and the live client both report 60xxx, and the consumer looks entries up
  by the id it is holding. So the crawl takes base ids in and publishes classic ids
  out. A file keyed by base ids would validate, read correctly in review, and never
  match a single lookup. Rune ids need no such treatment: op.gg already publishes
  those in the 775xxx classic block, as it does items in 77xxxx. Champions are
  the exception.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.6.0 c154a17ced

  Compare

  v0.6.0 Stable

  joeyr released this 2026-07-30 21:35:39 -05:00 | 831 commits to main since this release

  Added

  • Classic Rift is crawled as its own pass, publishing
    champion-counters-classic.json. op.gg serves the mode under
    type=classic, and the pass reuses everything except that parameter and the
    output paths — same region, tier, coverage floors, pacing and time budgets —
    so the classic file describes the same population as its ranked counterpart.

    It is a separate file rather than extra rows, and that is forced. op.gg
    keys classic pages by BASE champion id: champion_id: 24 is Jax in both
    queues, measured on two different kits. Merged into one map they would collide
    on the key and one would silently win.

    Classic is the cheaper half by a wide margin — 81 (champion, role) pairs
    against ranked's ~272 — so it runs second, where a short budget costs the
    ranked artifacts nothing. Its outcomes get their own publish streaks, because
    its file goes stale independently of theirs.

  • A classic rune extractor (opgg/classic-runes-page.ts), not yet wired to
    an artifact. Classic Rift restores the pre-2017 rune system, which is a
    different data model rather than a variant: no paths, no keystone, no shards,
    just a flat multiset of Marks, Seals, Glyphs and Quintessences held in
    quantities. op.gg serves it from the same runes page under classic_runes,
    with 775xxx ids and a win COUNT rather than a rate, normalised to a percent
    on the way out to match the other artifacts. Verified against live pages.

  Fixed

  • buildTierListUrl omitted type, which would have made the classic crawl
    describe ranked champions. The tier lists are the index both op.gg crawls
    run off, and op.gg defaults the parameter to ranked when it is absent — so
    the ranked pass was correct by luck. A classic pass would have discovered the
    RANKED universe and then gone looking for those champions on classic pages,
    producing a classic-labelled artifact full of the wrong roster. That is a
    wrong file rather than an empty one, which is the failure this codebase is
    built to avoid. Found before the classic pass ever ran.

  Notes

  Two things op.gg does not publish for this mode, established by sampling every
  role: masteries (legacy_masteries is an empty array on every page) and
  synergies (play: 0, and the page itself says the sample is too small).
  Neither is crawlable, and neither is a gap on this side.

  Classic data is also not segmented by rank: the tier parameter is accepted
  and discarded, with iron_plus, challenger and a deliberately invalid value
  all returning byte-identical rows. The published file therefore describes all
  ranks, and a later release should stop labelling it gold_plus.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.5.0 ee5b20e8b2

  Compare

  0.5.0 — escalate a chronically stale artifact Stable

  joeyr released this 2026-07-30 09:07:11 -05:00 | 832 commits to main since this release

  Declining to publish is correct — the coverage floors exist so a thin crawl cannot replace a good file — and it is also completely silent. The night a failure becomes chronic logs exactly what the first night logged, and the window still reports success on the strength of the artifacts that did publish.

  Patch 16.15 is the worked example: counters aborted at 55% coverage two nights running while champion-counters.json kept serving 16.14 data, and the only trace was one WARN among 122 others.

  Added

  Each of the four artifacts now carries a miss streak in a new publish_state table.

  outcome	level
  missed, streak < STALE_AFTER_NIGHTS (3)	WARN
  missed, streak >= STALE_AFTER_NIGHTS	ERROR, naming the streak and daysSinceSuccess
  missed because something threw	ERROR on the first night
  published after a streak	INFO, naming the missed nights
  published normally	nothing — the commit line already said so

  • It counts nights, not runs. The counter advances at most once per local calendar date. RUN_ON_START and a redeploy both re-enter the window, so without that guard, deploying a fix would trip the alarm it was fixing.
  • Faults skip the streak. A coverage abort is an expected non-result; a schema regression or a 403 from Forgejo is wrong immediately, and giving it three nights would bury the one signal worth acting on.
  • The state is in Postgres, not beside the heartbeat, which lives in an ephemeral /tmp — a redeploy would reset every alarm, and a redeploy is what happens right after shipping a fix for the thing being tracked.
  • Nothing is recorded when PUBLISH_ENABLED=false. A smoke run has no publish outcome, and letting it write would leave the production streaks reading stale.

  Fixed

  • A damage publish failure no longer costs the benchmarks artifact. The two come from one crawl and were documented as publishing independently, but shared a single try/catch — so a schema regression in the first skipped the second, built from rows already paid for.
  • The heartbeat is no longer touched after a Riot half that published nothing. runRiotWindow returned void and the caller marked the window successful unconditionally, so a night that dedup'd to zero new rows still reported health to the container healthcheck.

  Upgrading

  No schema change to any published artifact; consumers need no update. The migration is additive and idempotent. One new optional env var, STALE_AFTER_NIGHTS (default 3).

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.4.1 7760c9cce5

  Compare

  0.4.1 — build identity in the boot line Stable

  joeyr released this 2026-07-29 09:17:41 -05:00 | 840 commits to main since this release

  Changed

  The boot line now identifies the build and the config that decides whether a night produces anything.

  version and commit are stamped into the image by CI via --build-arg, using the same values it tags the image with. This service sleeps 21 hours a day and Swarm pins :latest without ever re-resolving it, so a deploy that silently did not pull was indistinguishable from one that did until the next window opened — telling them apart meant resolving the running task's image digest against the registry by hand. A version of dev in a container means the build args never reached docker build.

  The line also carries what is otherwise invisible until too late: publishEnabled (false crawls all night and commits nothing), both coverage floors (what withheld counters and runes on 2026-07-28), lookbackHours, the target owner/repo@branch, and archiveDir.

  DATABASE_URL is never logged — it carries the password inline. It is reduced to host/database.

  Upgrade notes

  No consumer change; logging and build metadata only.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.4.0 b4c4d2df63

  Compare

  0.4.0 — canonical-name fix and patch archiving Stable

  joeyr released this 2026-07-29 09:06:04 -05:00 | 842 commits to main since this release

  Fixed

  Champion display names stopped being unique, and it took both op.gg artifacts down.
  Patch 16.15 added 60 Jade_* entries to Data Dragon (173 champions -> 233), each carrying its base champion's display name. ChampionDirectory built its name->id maps last-write-wins, so for the 18 names whose base alias sorts before the Jade_ block, the 60001+ id won. That id picks the op.gg slug, and jade_annie is not a page: 31 of 272 (champion, role) pairs failed, coverage fell to 88.6%, and the 90% floor correctly withheld both counters and runes on the night of 2026-07-28.

  Name lookups now keep the lowest id. The id->slug direction never needed a fix.

  Added

  A patch's final artifacts are archived instead of overwritten. The four canonical paths still hold whatever patch is live — lol-companion fetches one fixed URL forever, unchanged. On the first run of a new patch the outgoing file is copied to ARCHIVE_DIR/<old patch>/<same basename>, e.g. data/archive/16.14/champion-runes.json.

  The trigger is the committed file's own generatedFrom.patch differing from the one being published, so no schedule or stored state is involved and a failed copy retries the next night. The bytes come from the read the no-op check already performs. Existing archives are never rewritten, and an archiving failure is logged rather than costing the publish.

  Kept indefinitely: ~3 MB per patch across the four files.

  Upgrade notes

  No consumer change. ARCHIVE_DIR defaults to data/archive; the deployed stack sets it explicitly.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.3.1 8538db6dd7

  Compare

  v0.3.1 Stable

  joeyr released this 2026-07-28 09:03:03 -05:00 | 845 commits to main since this release

  Changed

  • Log lines name their severity — INFO, WARN, ERROR — instead of
    pino's numeric scale. The Portainer dashboard renders whatever the level
    field holds, so a nightly run showed a bare 30 where the severity should be,
    readable only if you already know that debug is 20 and error is 50. The
    LOG_LEVEL threshold is unaffected; it took names already.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads
• v0.3.0 f3741794c8

  Compare

  v0.3.0 Stable

  joeyr released this 2026-07-28 08:56:18 -05:00 | 846 commits to main since this release

  Changed

  • The counters and runes artifacts are published readably instead of
    minified. Both were committed as a single line, which kept them small but
    made them unreadable in the Forgejo UI and gave git no line structure to delta
    between nightly rewrites. They now use a width-bounded printer: structure on
    its own lines, but any subtree whose minified form fits in 120 columns stays
    inline, so a matchup or a rune board row remains on one line. Measured on
    patch 16.14 — runes 398 KB → 539 KB (12,747 lines) against 1.47 MB at full
    indentation; counters 911 KB → 1.08 MB (16,220 lines) against 2.01 MB. The
    damage and benchmarks artifacts are unchanged, still fully indented.

    The JSON data is byte-for-byte equivalent, so this is not a schema change
    and consumers need no update. The first run after deploying rewrites both
    files once, then nightly diffs resume — and are now legible.

  • The publish option that controlled this is now an ArtifactFormat
    (INDENTED_FORMAT / COMPACT_FORMAT) rather than a bare indent number.

  Added

  • Images are tagged with the package version (:X.Y.Z) alongside :latest
    and the commit SHA, so a running container can be matched to the vX.Y.Z tag
    and release it was cut from.

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    0 downloads