Celestia DHT Network Health Weekly Reports

Celestia DHT Network Health Weekly Reports

High Level Description

The ProbeLab team (https://probelab.io - @guissou, @cortze, @kasteph, @dennis-tra) has developed and deployed infrastructure to monitor several critical metrics for Celestia’s DHT network. We have adapted the Nebula crawler (GitHub - dennis-tra/nebula: 🌌 A network agnostic DHT crawler, monitor, and measurement tool that exposes timely information about DHT networks.) to be compatible with the Celestia network and are gathering results that reflect the structure and health of the P2P network at the DHT level.

In this post we’re presenting a summary of what is included in the reports, but for a more complete picture of what’s there, head to: Week 2024-37 | ProbeLab for the latest report.

Why you should care

These metrics:

  • give an overview of the network structure, network size and agent adoption breakdown. This helps in understanding the robustness and diversity of the network,
  • provide accurate geographic distribution of nodes in the network per agent version (and client, where applicable) over time, which can highlight regional trends and potential vulnerabilities or strengths in specific areas,
  • make it easy to spot drastic changes in the structure and setup of the network,
  • allow for monitoring of new protocol version uptake/adoption, and provide insights on whether there are adoption barriers,
  • reveal the infrastructure setup (e.g., data center-hosted vs non-data center-hosted) and cloud provider distribution per client implementation,
  • show the breakdown of nodes supporting particular network-layer protocols,
  • depict the percentage of reachable vs unreachable node records in the DHT network.

Overview of Results

We’re presenting a small fraction of the results given at https://probelab.io to give an idea of the metrics listed. Please head there for the complete reports from Week 15 (mid-April), 2024 onwards.

Agent distribution and adoption over time

Churn analysis per agent version

Country distribution of Full Nodes over time

Cloud provider distribution of Full Nodes

Cloud vs non-cloud distribution of Full Nodes over time

Additional results that one can find at: https://probelab.io include the supported protocols, as well as the count of nodes that support each protocol over time; the number of connection and crawl errors over time, together with the error type; and finally, the reachable vs unreachable DHT node records over time.

What’s next

ProbeLab is building tooling and deploying infrastructure to monitor the population and uptime of Light Nodes in the Celestia network. The relevant plots and results will be published either as part of these weekly reports or separately in the coming month. Monitor https://probelab.io and stay tuned - we will follow up with a separate post when these results are public.

How to contribute

We hope you’ll find the reports useful. If there are important metrics that you believe should be part of these weekly reports, or have questions regarding the existing ones, comment below, or get in touch with the team: Contact.

2 Likes

The latest report Week 2024-38 is out!

In this report, we have changed our methodology to count stale node records in the DHT. We used to count the number of crawl_errors reported by Nebula, but we figured counting connect_errors better reflects the reachability of nodes.

As a result, the latest and future reports are showing higher numbers of stale records than previous reports. It isn’t due to an increase in the number of stale records in the network, but due to a difference in the way they are counted.

1 Like