Reading through specs/src/fibre_server.md and CIP-51 ahead of Fibre going live. Three things that don’t resolve from the docs.
1.ShardRetention (default 4h) is described in fibre_server.md as an on-chain, governance-changeable x/fibre parameter, with pruneAt = max(ExpiresAt, creation_timestamp + ShardRetention). CIP-51’s parameter table lists only GasPerBlobByte, WithdrawalDelay, PaymentPromiseTimeout, PaymentPromiseRetentionWindow and PaymentPromiseHeightWindow. ShardRetention isn’t in it. Should the CIP table be updated, or is this intentionally implementation-level?
2. The server spec notes that the reference implementation stores before signing, so a signature means the shard was accepted and stored. That holds for the reference server. Is there an intended path for detecting a modified server that signs without storing, or one that prunes earlier than pruneAt? Or is the intended answer honest majority plus reputational pressure, consistent with treating social consensus as the final layer?
3.DownloadShard is public to any client that completes the server-authenticated TLS handshake, and the spec notes no per-peer rate limiting is currently implemented. Is periodic external probing of registered Fibre endpoints (measuring per-validator shard availability, and whether a given commitment is still reconstructable) something the core team plans to ship, or is it expected to come from operators? If the latter, what probe rate would be acceptable given the current lack of rate limiting?
1.ShardRetention (default 4h) is described in fibre_server.md as an on-chain, governance-changeable x/fibre parameter, with pruneAt = max(ExpiresAt, creation_timestamp + ShardRetention). CIP-51’s parameter table lists only GasPerBlobByte, WithdrawalDelay, PaymentPromiseTimeout, PaymentPromiseRetentionWindow and PaymentPromiseHeightWindow. ShardRetention isn’t in it. Should the CIP table be updated, or is this intentionally implementation-level?
2. The server spec notes that the reference implementation stores before signing, so a signature means the shard was accepted and stored. That holds for the reference server. Is there an intended path for detecting a modified server that signs without storing, or one that prunes earlier than pruneAt? Or is the intended answer honest majority plus reputational pressure, consistent with treating social consensus as the final layer?
Yes, we rely on an honest majority to serve the data for the specified period of time. And by design, we only need 1/3rd of the validator set to be honest to be able to retrieve the rows. And if that’s not possible, that means more than 2/3rd of the validator set are malicious and social consensus is needed.
3.DownloadShard is public to any client that completes the server-authenticated TLS handshake, and the spec notes no per-peer rate limiting is currently implemented. Is periodic external probing of registered Fibre endpoints (measuring per-validator shard availability, and whether a given commitment is still reconstructable) something the core team plans to ship, or is it expected to come from operators? If the latter, what probe rate would be acceptable given the current lack of rate limiting?
We’re planning to implement rate limiting for downloads in the next iterations. Also, the community is welcome to propose solutions on how to effectively rate limite download without impacting honest readers.
For making sure the data is available, that can be ensured by the security assumptions of the protocol. Additionaly, the community will generally build dashboards that will:
Periodically check whether the provided fibre server addresses, in the x/valaddr module are reachable
On rate limiting: I’ve been building an external verifier for this and have some measurements that might be useful input for that design. It watches the chain for MsgPayForFibre, recomputes the deterministic shard assignment for the promise height, and probes only the validators actually assigned rows for that blob, verifying returned rows against the commitment. Repeat measurements across the retention window rather than a single check, since the obligation is continuous serving rather than initial acceptance.
Cost per probe against a local devnet: p50 ~14ms, p95 ~24ms for dial plus DownloadShard plus full row verification. Per publication that’s one probe per assigned validator per scheduled point, roughly 4-5 points across the window. So a well-behaved external verifier generates traffic that is regular and predictable rather than bursty, and its request pattern is derivable from chain state, which might be a useful distinguishing signal.
One thing worth considering in the design: a naive per-peer limit can’t tell a legitimate verifier from abuse, and if verification traffic gets throttled the same way, the community dashboards you mention become unreliable. A cheaper signal might be that legitimate probes only ever request BlobIDs that correspond to publications the requesting party can point to on-chain, and only from validators actually assigned to them.
Also measured, in case it’s useful: actual prune happens at pruneAt + ~1m45s in practice, from the 60s prune loop plus the minute-granularity prune key. Any monitoring that treats the first post-pruneAtNOT_FOUND as a fault will produce false positives on every publication.
Code is at GitHub - plsgiveup/fibre · GitHub if any of it is useful. Happy to write up the rate-limiting angle properly if that would help.
