Skip to content

Add save and load route table feature to nats-client #295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 31, 2022

Conversation

domdom82
Copy link
Contributor

@domdom82 domdom82 commented Oct 26, 2022

  • A short explanation of the proposed change:
  • Added "save" and "load" functionality to the nats client in routing_utils
  • The "save" function stores a gorouter's routing table in a json file
  • The "load" function plays back the json file by registering all routes with gorouter again
  • An explanation of the use cases your change solves
    The main purpose of this functionality is to be able to easily load scenarios where the route table is in a specific state.
    This is helpful in debugging problems like the ones mentioned here and here or here

Developers can simply craft a route table as desired instead of having to tweak apps and NATS to behave in such a way that the issue is reproduced.

  • Instructions to functionally test the behavior change using operator interfaces (BOSH manifest, logs, curl, and metrics)
  1. Deploy nats and gorouter as described in the README.md
  2. Provide a minimal gorouter.yml config file for the nats-client to read. If you started gorouter without a config file, you may use this one:
---

status:
  port: 8082
  user:
  pass:

nats:
  tls_enabled: false
  hosts:
  - hostname: localhost
    port: 4222
  1. Add a few routes. You may use the nats-client or simply curl to send some router.register NATS messages.
  2. Save the current route table using
./nats_client gorouter.yml save routes.json
Saving route table to routes.json
Done
  1. You should now have a routes.json file with a pretty-printed output of the /routes endpoint of gorouter. It could look like this:
{
  "api.cf.my.domain.com": [
    {
      "address": "10.0.65.66:9024",
      "private_instance_id": "767a8085-3e34-47d6-47b3-1ac87020edad",
      "protocol": "http1",
      "server_cert_domain_san": "api.cf.my.domain.com",
      "tags": {
        "component": "CloudController"
      },
      "tls": true,
      "ttl": 120
    }
  ],
  "cf-app.cf.my.domain.com": [
    {
      "address": "10.0.73.0:61015",
      "private_instance_id": "32e285ab-9b42-4de8-7141-3b87",
      "protocol": "http1",
      "server_cert_domain_san": "32e285ab-9b42-4de8-7141-3b87",
      "tags": {
        "app_id": "e74be186-20f5-4ea8-844b-77895c477c10",
        "app_name": "cf-app",
        "component": "route-emitter",
        "instance_id": "0",
        "organization_id": "bc578e33-bce1-4bce-91c3-a302bcb27ee3",
        "organization_name": "ACME",
        "process_id": "e74be186-20f5-4ea8-844b-77895c477c10",
        "process_instance_id": "32e285ab-9b42-4de8-7141-3b87",
        "process_type": "web",
        "source_id": "e74be186-20f5-4ea8-844b-77895c477c10",
        "space_id": "e646f0e9-1aa3-41f3-b177-9427ec5e7c84",
        "space_name": "dev"
      },
      "tls": true,
      "ttl": 120
    },
(...)
}
  1. You may now edit the routes.json file, e.g. add another route or change an existing one, add more endpoints etc. Be aware that this only works well with TLS routes as non-TLS routes will be pruned shortly after.
  2. Use the nats-client to upload the changed routes to gorouter:
./nats_client gorouter.yml load routes.json
Loading route table from routes.json
Done
  1. You can then test the changed routes by curling them, e.g. if you changed the route to cf-app.cf.my.domain.com:
curl -v http://localhost:8081 -H "Host: cf-app.cf.my.domain.com"
(...)
  1. You should see the effects in the behavior of gorouter. It will use the changed route instead. Be aware that your changes may be overwritten by route-emitters of the app you changed. So for best results it is recommended to inject a new route that does not exist on any app.
  • Expected result after the change

nats-client can save and restore gorouter route tables using NATS.

  • Current result before the change

nats-client can only publish and subscribe to NATS.

  • Links to any other associated PRs

  • I have viewed signed and have submitted the Contributor License Agreement

  • I have made this pull request to the develop branch

  • I have run all the unit tests using scripts/run-unit-tests-in-docker

  • (Optional) I have run Routing Acceptance Tests and Routing Smoke Tests on bosh lite

  • (Optional) I have run CF Acceptance Tests on bosh lite

@domdom82 domdom82 marked this pull request as ready for review October 27, 2022 08:23
@domdom82 domdom82 force-pushed the nats_client_save_load_routes branch 3 times, most recently from 83cd22e to be8e3a9 Compare October 27, 2022 14:04
@ameowlia
Copy link
Member

This will be great for debugging.

❓ Can you add these notes for how to use it in a doc in the docs dir?

I'll let you assign a reviewer for a more thorough review. Thanks!

@domdom82
Copy link
Contributor Author

Can you add these notes for how to use it in a doc in the docs dir?

Good idea! Will do.

@domdom82 domdom82 force-pushed the nats_client_save_load_routes branch from be8e3a9 to 32f0e45 Compare October 28, 2022 10:22
@domdom82 domdom82 requested a review from b1tamara October 28, 2022 13:50
@b1tamara
Copy link
Contributor

LGFM

@domdom82 domdom82 merged commit 7f61a9d into cloudfoundry:develop Oct 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants