cat METHODOLOGY.md

How the score is built

Relay Compare aggregates independent probe data on AI API relay stations so you can pick one without relying on Telegram rumors. Every component of the Quality Score is disclosed below — including what is real today versus what is a placeholder for the paid probe pipeline.

Quality score composition weighted sum · 0–10

authenticity 40%latency 25%uptime 20%price 15%

score = 10 × ( 0.40·auth + 0.25·latency_norm + 0.20·uptime_30d + 0.15·price_fair )
// each term normalised to 0–1 · latency_norm = clamp(1 − p50_ms / 2000, 0, 1)

Note: while auth and price are placeholders we fill both with a neutral 0.5. Under those placeholders the maximum achievable score today is ≈ 7.75, so no relay can reach the Trustworthy ≥ 8.0 floor until the authenticity probe lands. This is by design — a top band should never be reachable from latency and uptime alone.

40%WEIGHT

Model authenticity placeholder

MVP placeholder — every relay currently receives the same neutral value. Post-MVP we probe each relay with fingerprint prompts and compare responses to the canonical upstream model to detect distillation, silent model substitution, or cached response reuse.

25%WEIGHT

Latency live

End-to-end probe response time from awesome-ai-api, measured on a small completion request. Post-MVP: measured from five global locations and averaged so a single fast region cannot inflate the score.

20%WEIGHT

Uptime live

30-day rolling availability from daily reachability probes. A probe counts as down on connection failure, TLS error, or any non-2xx response to a well-formed request.

15%WEIGHT

Price fairness roadmap

Post-MVP. Compares each relay’s claimed pricing to the upstream models.dev baseline. Relays charging more than 3× the baseline get flagged as Overpriced; relays charging less than the upstream cost get an extra authenticity review, since that is usually how a substituted model is subsidised.

Trust bands derived from score + history

Trustworthy

≥ 8.0

Plus ≥ 14 days of probe history and no substitution flag in the last 30 days.

Medium

6.0 – 7.9

Or any relay with fewer than 14 days of history, regardless of score.

Risky

< 6.0

Or an active substitution / overpricing flag, or 3+ outages in 7 days.

Bands are a floor, never a ceiling: a relay can be demoted by a flag even with a high score, but never promoted above what its score allows.

Data sources refreshed daily via GitHub Action

awesome-ai-apiclaws-zh.github.io

Open-source daily probe of AI API relays. We consume the raw gateways.json and refresh our snapshot every day at 06:00 UTC.

feeds latency · uptime · model listlicense MIT

models.devgithub.com

Structured per-model pricing and context-window data for 195 flagship models across 8 providers. MIT-licensed with attribution.

feeds price baselinelicense MIT

kina.menkina.men

Tracks every Visit → click on gorelay.online with country, device, browser, and referrer. Powers the paid dashboard.

feeds traffic reportsretention 90 days

Probe pipeline what happens at 06:00 UTC

01 fetch gateways.json ← awesome-ai-api
02 fetch models.dev/api.json ← price baseline
03 join on provider + model id
04 score each relay → formula above
05 band trustworthy | medium | risky
06 diff vs yesterday → alerts for subscribers
07 commit data/snapshot-YYYY-MM-DD.json

Every snapshot is committed to the public repo, so any score on the site can be reproduced from the day’s raw inputs.

Roadmap

shippedDaily snapshot + public rankingsLatency, uptime and model list for every tracked relay.
in progressMulti-region latency probesFive global locations (US-E, EU-W, SG, TYO, CN-HK) averaged per relay.
nextResponse-fingerprint model authenticity probesTurns the 40% placeholder into a live signal.
nextPrice scraping + fairness scoringAgainst the models.dev baseline; overpricing flags.
laterPer-user watchlists + email alertsOn relay degradation or band change.
laterWHOIS / domain-age / operator background checksFed into the trust band as a demotion signal.

Reserve the paid plan on the Subscribe page.