# Rate limiting Fibre's read path without IP allowlists **URL:** \ **Category:** Research\ **Created:** [September 4, 2026, 8:26pm UTC](https://forum.celestia.org/t/rate-limiting-fibres-read-path-without-ip-allowlists/2295 "2026-09-04T20:26:11Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![utku](https://forum.celestia.org/user_avatar/forum.celestia.org/utku/32/1656_2.png) [@utku](https://forum.celestia.org/u/utku)\ **Post date:** [September 4, 2026, 8:26pm UTC](https://forum.celestia.org/t/rate-limiting-fibres-read-path-without-ip-allowlists/2295/1 "2026-09-04T20:26:11Z")
ADR-029 gave `UploadShard` a solid, stake-derived occupancy budget, and explicitly scoped the read path out: `DownloadShard` never grows the store, so it was left unthrottled. That gap is worth closing before Fibre carries real traffic, and I want to argue for a specific way to close it, because the obvious way (rate-limit by peer) doesn’t work here. ## The problem with per-peer limits `DownloadShard` is public to any client that completes the TLS handshake. No client cert, no mTLS, no request-level auth (see `fibre_server.md`, Transport Security). So the only thing a naive limiter has to key on is the connection: source IP, maybe ASN. That’s the standard move for a public read endpoint, and it’s the wrong one here, because it can’t tell apart the three kinds of traffic hitting this RPC: 1. **Honest readers** , fetching a shard they’re entitled to, for a blob that’s genuinely live. 2. **Honest external observers** , tools that re-derive shard assignment from chain state and deliberately query validators (including off-window, where `NOT_FOUND` is the correct answer) to check they’re actually serving what they signed for. 3. **Abuse** : scraping, DoS attempts, or a client that never checks whether a blob exists before asking every validator for it. A peer-based limiter treats all three identically, because peer identity carries no information about which of these a request is. It also invites a fix that seems obvious and is actually harmful: allowlisting known monitor IPs. That lets a validator pass every check a monitor throws at it while quietly dropping real readers, and the selective serving becomes undetectable precisely because the only external check has a fast lane. Good external observability is a reason not to special-case the observer’s traffic. ## What actually distinguishes the traffic A legitimate request, reader or observer, has two properties that abuse usually doesn’t: - It asks for a `BlobID` that corresponds to a real, currently-live publication, meaning an on-chain `MsgPayForFibre` settlement still inside its retention window. - It asks the validator actually assigned that shard by the deterministic, stake-weighted assignment (`Set.Assign`). That’s public information, computable by any outside party from the commitment and the validator set, using the same deterministic function celestia-app runs. Neither requires trusting or authenticating the client. Assignment is a pure function of on-chain data. I verified this independently, off-chain, bit-for-bit against the reference implementation across ~890 scenarios. Any honest client can do the same computation before opening a connection, and should, if it wants the shard on the first try. ## The server already has this for free This isn’t a new mechanism, it’s already in the code path. `DownloadShard` (`fibre/server_download.go`) decodes the `BlobID`, then calls `store.Get(ctx, commitment)`. That lookup already answers both questions: - **Existence** : no separate on-chain query needed. The store is the durable record of what got settled and admitted. - **Assignment** : guaranteed by construction rather than by an extra check. `UploadShard` runs `verifyAssignment` before ever calling `store.Put` (`fibre/server_upload.go`). If a shard is in the store, this validator was the assignee at upload time. A hit on `Get` is a validated assignment. The cost asymmetry is large too. `Store.Get` is a pebble prefix scan over small metadata keys; bulk shard bytes are deliberately kept out of pebble. A miss returns `ErrStoreNotFound` without touching a shard file. A hit reads the actual shard off disk, up to `MaxShardSize` (129.83 MiB per ADR-029’s sizing example), and serializes it back over gRPC. So “is this real” is nearly free, and “serve it” is expensive by orders of magnitude. That’s the shape a rate limiter wants: a cheap admission check gating an expensive payload. ## What this suggests Rate limit on the classification of the request, not on who’s asking. A commitment genuinely in the store, within its retention window, is the honest-reader path, and functionally the same request as an observer’s. Leave it alone, or apply the loosest limit. A commitment that was never real, or asking a validator never assigned it, is what abusive scanning looks like: hammering every validator hoping one sticks, without doing the free assignment computation first. Limit this hard per source. A client that skipped the free check is either broken or hostile. `NOT_FOUND` after a publication’s `must_serve_until` has passed is also expected, and shouldn’t feed any abusive-peer score. It’s the shape of a well-behaved observer’s tail probe, checking that pruning happened on schedule. For scale: in my own measurements a single probe (DNS, TCP, TLS and download, judged separately) runs p50 14ms / p95 24ms, with 4-5 points scheduled per assigned validator per publication. That’s a very different traffic pattern from a scraper hammering commitments that don’t exist. The tolerance for a missing shard also has to account for real pruning lag. I measured prune landing at `pruneAt + ~1m45s` rather than exactly at `pruneAt`, from the 60s prune loop plus the minute-granularity prune key. Without that, honest late prunes get flagged as something worse. None of this needs new on-chain data, a new RPC, or client identity. It’s the same `store.Get` that already runs, read as a classifier instead of a binary success or failure. ## Open questions - Does this belong directly in `fibre/server_download.go`, cheap enough to always run, or as an operator-tunable wrapper? - Should the rate itself derive from stake like ADR-029’s upload budget, or is a flat per-source limit fine for reads, given there’s no disk-occupancy risk, only CPU and bandwidth? - Assignment constants (`MinRowsPerValidator`, `LivenessThreshold`, row counts) aren’t among `x/fibre`’s five governance params, they’re compiled into the validator binary. A classifier has to pull them from the binary version, so a silent cross-version mismatch is possible. That pinning needs deliberate design rather than a fixed table assumed to hold forever. I can run this classifier logic against a multi-node devnet, with real publications, real assignment and real prune timing, plus a synthetic mix of valid, stale and wrong-validator requests, and come back with actual hit/miss and cost numbers for the three buckets above. Happy to do that pass if it would help move this past a proposal.