Skip to content

bgpdata/bgpdata

Repository files navigation




https://bgp-data.net — A BGP Data Aggregation Service.

BGPDATA helps researchers and network operators gain insights into their networks by providing a scalable and reliable way to analyze and inspect historical and live global BGP data from Route Views and RIPE NCC RIS, as well as from direct peers who graciously choose to contribute additional data to the system.

Our tool is an open-source initiative dedicated to advancing real-time BGP analysis through community-driven development. By leveraging community-operated infrastructure, we aim to accelerate research and innovation in network analysis by empowering researchers to rapidly deploy and validate their experiments with the global Internet community. Unlike proprietary solutions, BGPDATA is non-profit and committed to limitless open collaboration, driving the broader Internet community's advancement forward.

You can actively contribute to BGPDATA's operation by joining the underlying distributed computing cluster SUPERCLUSTR. This collaborative platform enables BGPDATA and other projects to benefit the Internet research community through shared resources and expertise.

Data Flow

We collect both historical and live BGP data from Route Views and RIPE NCC RIS collectors to provide comprehensive insights.

Prerequisites

Before you begin, ensure you have the following installed on your system:

Getting Started

  1. Clone the repository:
git clone --recurse-submodules [email protected]:bgpdata/bgpdata.git
cd bgpdata
  1. Start the project:
jinja2 docker-compose.jinja values.yaml | docker compose -f - up

Note: This will start collecting from all RouteViews and RIS Collectors. You can further specify to collect from only a specific subset of collector hosts by modifying the values.yaml file`.

  1. Open:
    http://localhost:8080 (Web)
    http://localhost:3000 (Grafana)
    and try whois -h localhost AS3582 (WHOIS)

Contributing

We welcome contributions from researchers and developers! If you need help implementing your ideas or getting started with the project, our team is here to assist you. Whether you're looking to contribute code, suggest improvements, or integrate BGPDATA into your research, we're happy to help you get up and running.

Feel free to reach out to us at [email protected] - we're committed to making the onboarding process as smooth as possible and helping you implement your ideas effectively.

Production Deployment

For production deployment, we use Docker Swarm, but any other orchestration tool may be used. The recommended system requirements are a Manager and Worker Node with each 60 GB of RAM, 60 GB Swap, 1 TB Datacenter SSD Storage and 48 vCPU cores. Initial system provisioning may require up to 3 hours to complete, contingent upon your specific configuration parameters. Once initialization is complete, resource utilization will stabilize at optimal levels.

# Customize the .env file, then:
curl -fsSL https://downloads.bgp-data.net/docker-compose.yml | docker stack deploy -c - bgpdata

Contact us at [email protected] if you need any help.

ACKs

  • RIS for providing the data and collector infrastructure
  • Route Views for providing the data and collector infrastructure
  • OpenBMP for providing the invaluable OpenBMP Suite used in-depth in this project
  • Tim Evens for his leading role in the development of the OpenBMP Suite
  • RIPEstat for their incredible infrastructure and data visualization
  • RIPE Atlas for providing the RIPE Atlas infrastructure and providing mesaurement data for this project
  • Massimo Candela for his outstanding work on BGPlay
  • Christian Teuschel for his inspiration for this project and operation of RIPEstat at the RIPE NCC
  • Ben Cartwright-Cox for the development of BGP.tools and service to the Internet Community
  • CAIDA BGPStream for providing a versatile BGP data streaming library
  • Postmark for providing the email service

License

See LICENSE