Skip to content

Simplify durable execution events monitoring of example for beginners #161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
y12studio opened this issue Jul 11, 2024 · 5 comments
Open

Simplify durable execution events monitoring of example for beginners #161

y12studio opened this issue Jul 11, 2024 · 5 comments

Comments

@y12studio
Copy link

Currently, newcomers to Restate example face a challenge when trying to understand the sequence of durable execution events. They need to monitor multiple terminals simultaneously, which can be overwhelming and confusing. To enhance the learning experience and provide clearer guidance, I recommend introducing a new section in the basics-typescript/README.md file, specifically focused on 'Running Examples with Docker Compose'.

Expected benefits:

  1. Reduced complexity for beginners setting up the environment
  2. Clearer visualization of the durable execution event flow
  3. Decreased cognitive load by eliminating the need to switch between multiple terminals
  4. Improved overall learning experience for new users

By implementing this change, we can significantly lower the entry barrier for beginners, helping them grasp the core concepts of durable execution more quickly and effectively.

$ docker compose up

restate_dev-1  | 2024-07-11T11:52:03.830098Z INFO restate_server
restate_dev-1  |   Starting Restate Server 1.0.2 (604592b x86_64-unknown-linux-gnu 2024-06-20)
restate_dev-1  |     node_name: "97a302f1d0ef"
restate_dev-1  |     config_source: [default]
restate_dev-1  |     base_dir: /restate-data/97a302f1d0ef/
app-1          | 
app-1          | up to date, audited 75 packages in 437ms
app-1          | 
app-1          | 9 packages are looking for funding
app-1          |   run `npm fund` for details
app-1          | 
app-1          | found 0 vulnerabilities
app-1          | 
app-1          | > @restatedev/[email protected] example-1
app-1          | > CRASH_PROCESS=true ./src/utils/restart-process.sh 'ts-node-dev --transpile-only src/1_durable_execution.ts'
app-1          | 
app-1          | [INFO] 11:52:04 ts-node-dev ver. 2.0.0 (using ts-node ver. 10.9.2, typescript ver. 5.5.3)
app-1          | [restate] [2024-07-11T11:52:04.718Z] INFO:  Listening on 9080...
restate_dev-1  | 2024-07-11T11:52:06.648193Z INFO restate_grpc_util
restate_dev-1  |   Server 'metadata-store-grpc' listening
restate_dev-1  |     net.host.addr: 0.0.0.0
restate_dev-1  |     net.host.port: 5123
restate_dev-1  | 2024-07-11T11:52:07.277841Z INFO restate_node
restate_dev-1  |   My Node ID is N0:5
restate_dev-1  | 2024-07-11T11:52:07.321325Z INFO restate_ingress_http::server
restate_dev-1  |   Ingress HTTP listening
restate_dev-1  |     net.host.addr: 0.0.0.0
restate_dev-1  |     net.host.port: 8080
restate_dev-1  | 2024-07-11T11:52:07.321461Z INFO restate_network::networking
restate_dev-1  |   Connection to node N0:5 failed with cannot connect: The service is currently unavailable error trying to connect: tcp connect error: Connection refused (os error 111), next retry is attempt 2/10
restate_dev-1  |   in restate_network::networking::send
restate_dev-1  |     to: N0
restate_dev-1  |     msg: GetProcessorsStateRequest
restate_dev-1  | 2024-07-11T11:52:07.322072Z INFO restate_grpc_util
restate_dev-1  |   Server 'node-grpc' listening
restate_dev-1  |     net.host.addr: 0.0.0.0
restate_dev-1  |     net.host.port: 5122
restate_dev-1  | 2024-07-11T11:52:07.322096Z INFO restate_admin::service
restate_dev-1  |   Admin API listening
restate_dev-1  |     net.host.addr: 0.0.0.0
restate_dev-1  |     net.host.port: 9070
restate_dev-1  | 2024-07-11T11:52:07.869352Z INFO restate_worker::partition_processor_manager
restate_dev-1  |   Plan applied from attaching to controller N0:5
curl-1         |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
curl-1         |                                  Dload  Upload   Total   Spent    Left  Speed
curl-1         | == Info: Host app:9080 was resolved.
curl-1         | == Info: IPv6: (none)
curl-1         | == Info: IPv4: 172.28.0.4
curl-1         | == Info:   Trying 172.28.0.4:9080...
curl-1         | == Info: Connected to app (172.28.0.4) port 9080
curl-1         | == Info: [HTTP/2] [1] OPENED stream for http://app:9080/
curl-1         | == Info: [HTTP/2] [1] [:method: GET]
curl-1         | == Info: [HTTP/2] [1] [:scheme: http]
curl-1         | == Info: [HTTP/2] [1] [:authority: app:9080]
curl-1         | == Info: [HTTP/2] [1] [:path: /]
curl-1         | == Info: [HTTP/2] [1] [user-agent: curl/8.8.0]
curl-1         | == Info: [HTTP/2] [1] [accept: */*]
curl-1         | => Send header, 69 bytes (0x45)
curl-1         | 0000: GET / HTTP/2
curl-1         | 000e: Host: app:9080
curl-1         | 001e: User-Agent: curl/8.8.0
curl-1         | 0036: Accept: */*
curl-1         | 0043: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl-1         | == Info: Request completely sent off
curl-1         | <= Recv header, 13 bytes (0xd)
curl-1         | 0000: HTTP/2 404 
curl-1         | <= Recv header, 32 bytes (0x20)
curl-1         | 0000: content-type: application/json
curl-1         | <= Recv header, 48 bytes (0x30)
curl-1         | 0000: x-restate-server: restate-sdk-typescript/1.0.1
curl-1         | <= Recv header, 37 bytes (0x25)
curl-1         | 0000: date: Thu, 11 Jul 2024 11:52:08 GMT
curl-1         | <= Recv header, 2 bytes (0x2)
curl-1         | 0000: 
curl-1         | <= Recv data, 0 bytes (0x0)
curl-1         | == Info: Connection #0 to host app left intact
curl-1         | == Info: Host restate_dev:9070 was resolved.
curl-1         | == Info: IPv6: (none)
curl-1         | == Info: IPv4: 172.28.0.3
curl-1         | == Info:   Trying 172.28.0.3:9070...
curl-1         | == Info: Connected to restate_dev (172.28.0.3) port 9070
curl-1         | => Send header, 143 bytes (0x8f)
curl-1         | 0000: POST /deployments HTTP/1.1
curl-1         | 001c: Host: restate_dev:9070
curl-1         | 0034: User-Agent: curl/8.8.0
curl-1         | 004c: Accept: */*
curl-1         | 0059: content-type: application/json
curl-1         | 0079: Content-Length: 26
curl-1         | 008d: 
curl-1         | => Send data, 26 bytes (0x1a)
curl-1         | 0000: {"uri": "http://app:9080"}
restate_dev-1  | 2024-07-11T11:52:15.647891Z INFO restate_admin::schema_registry::updater
restate_dev-1  |   Overwriting existing service schemas
restate_dev-1  |     rpc.service: roleUpdate
curl-1         | == Info: upload completely sent off: 26 bytes
curl-1         | <= Recv header, 22 bytes (0x16)
curl-1         | 0000: HTTP/1.1 201 Created
curl-1         | <= Recv header, 32 bytes (0x20)
curl-1         | 0000: content-type: application/json
curl-1         | <= Recv header, 51 bytes (0x33)
curl-1         | 0000: location: /deployments/dp_12GPapzNSRdYp6feoUi7Xln
curl-1         | <= Recv header, 21 bytes (0x15)
curl-1         | 0000: content-length: 392
curl-1         | <= Recv header, 37 bytes (0x25)
curl-1         | 0000: date: Thu, 11 Jul 2024 11:52:15 GMT
curl-1         | <= Recv header, 2 bytes (0x2)
curl-1         | 0000: 
curl-1         | <= Recv data, 392 bytes (0x188)
curl-1         | 0000: {"id":"dp_12GPapzNSRdYp6feoUi7Xln","services":[{"name":"roleUpda
curl-1         | 0040: te","handlers":[{"name":"applyRoleUpdate","ty":"Shared","input_d
curl-1         | 0080: escription":"one of [\"none\", \"value of content-type 'applicat
curl-1         | 00c0: ion/json'\"]","output_description":"value of content-type 'appli
curl-1         | 0100: cation/json'"}],"ty":"Service","deployment_id":"dp_12GPapzNSRdYp
curl-1         | 0140: 6feoUi7Xln","revision":3,"public":true,"idempotency_retention":"
curl-1         | 0180: 1day"}]}
curl-1         | {"id":"dp_12GPapzNSRdYp6feoUi7Xln","services":[{"name":"roleUpdate","handlers":[{"name":"applyRoleUpdate","ty":"Shared","input_description":"one of [\"none\", \"value of content-type 'application/json'\"]","output_description":"value of content-type 'application/json'"}],"ty":"Service","deployment_id":"dp_12GPapzNSRdYp6feoUi7Xln","revision":3,"public":true,"idempotency_retention":"1day"}]}== Info: Connection #0 to host restate_dev left intact
curl-1         |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
curl-1         |                                  Dload  Upload   Total   Spent    Left  Speed
restate_dev-1  | 2024-07-11T11:52:17.053785Z INFO restate_ingress_http::handler::service_handler
restate_dev-1  |   Processing ingress request
restate_dev-1  |   in restate_ingress_http::handler::tracing::ingress_invoke
restate_dev-1  |     otel.name: "ingress_invoke roleUpdate/applyRoleUpdate"
restate_dev-1  |     rpc.system: "restate"
restate_dev-1  |     rpc.service: roleUpdate
restate_dev-1  |     rpc.method: applyRoleUpdate
restate_dev-1  |     restate.invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     restate.invocation.target: roleUpdate/applyRoleUpdate
restate_dev-1  |     client.socket.address: 172.28.0.2
restate_dev-1  |     client.socket.port: 55558
restate_dev-1  | 2024-07-11T11:52:17.257259Z INFO restate_invoker_impl::invocation_task::service_protocol_runner
restate_dev-1  |   Executing invocation at deployment
restate_dev-1  |     invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     deployment.address: http://app:9080/
restate_dev-1  |     deployment.service_protocol_version: 1
restate_dev-1  |     path: /invoke/roleUpdate/applyRoleUpdate
app-1          | [restate] [roleUpdate/applyRoleUpdate][inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1][2024-07-11T11:52:17.261Z] INFO:  Invoking function.
app-1          | >>> Applied role content-manager for user Sam Beckett
app-1          | >>> Applied permission add:allow for user Sam Beckett
app-1          | A failure happened!
app-1          | --- CRASHING THE PROCESS ---
restate_dev-1  | 2024-07-11T11:52:18.063267Z WARN restate_invoker_impl
restate_dev-1  |   Error when executing the invocation, retrying in 54ms 631us 480ns.
restate_dev-1  |     error: [RT0010] error reading a body from connection: stream closed because of a broken pipe
restate_dev-1  |     restate.error.code: RT0010
restate_dev-1  |     restate.invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     restate.invocation.target: roleUpdate/applyRoleUpdate
restate_dev-1  | 
restate_dev-1  | 2024-07-11T11:52:18.118795Z INFO restate_invoker_impl::invocation_task::service_protocol_runner
restate_dev-1  |   Executing invocation at deployment
restate_dev-1  |     invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     deployment.address: http://app:9080/
restate_dev-1  |     deployment.service_protocol_version: 1
restate_dev-1  |     path: /invoke/roleUpdate/applyRoleUpdate
restate_dev-1  | 2024-07-11T11:52:18.119492Z WARN restate_invoker_impl
restate_dev-1  |   Error when executing the invocation, retrying in 128ms 882us 643ns.
restate_dev-1  |     error: [RT0010] error trying to connect: tcp connect error: Connection refused (os error 111)
restate_dev-1  |     restate.error.code: RT0010
restate_dev-1  |     restate.invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     restate.invocation.target: roleUpdate/applyRoleUpdate
restate_dev-1  | 
restate_dev-1  | 2024-07-11T11:52:18.250147Z INFO restate_invoker_impl::invocation_task::service_protocol_runner
restate_dev-1  |   Executing invocation at deployment
restate_dev-1  |     invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     deployment.address: http://app:9080/
restate_dev-1  |     deployment.service_protocol_version: 1
restate_dev-1  |     path: /invoke/roleUpdate/applyRoleUpdate
restate_dev-1  | 2024-07-11T11:52:18.250849Z WARN restate_invoker_impl
restate_dev-1  |   Error when executing the invocation, retrying in 224ms 605us 681ns.
restate_dev-1  |     error: [RT0010] error trying to connect: tcp connect error: Connection refused (os error 111)
restate_dev-1  |     restate.error.code: RT0010
restate_dev-1  |     restate.invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     restate.invocation.target: roleUpdate/applyRoleUpdate
restate_dev-1  | 
restate_dev-1  | 2024-07-11T11:52:18.476585Z INFO restate_invoker_impl::invocation_task::service_protocol_runner
restate_dev-1  |   Executing invocation at deployment
restate_dev-1  |     invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     deployment.address: http://app:9080/
restate_dev-1  |     deployment.service_protocol_version: 1
restate_dev-1  |     path: /invoke/roleUpdate/applyRoleUpdate
restate_dev-1  | 2024-07-11T11:52:18.477523Z WARN restate_invoker_impl
restate_dev-1  |   Error when executing the invocation, retrying in 475ms 95us 396ns.
restate_dev-1  |     error: [RT0010] error trying to connect: tcp connect error: Connection refused (os error 111)
restate_dev-1  |     restate.error.code: RT0010
restate_dev-1  |     restate.invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     restate.invocation.target: roleUpdate/applyRoleUpdate
restate_dev-1  | 
restate_dev-1  | 2024-07-11T11:52:18.953491Z INFO restate_invoker_impl::invocation_task::service_protocol_runner
restate_dev-1  |   Executing invocation at deployment
restate_dev-1  |     invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     deployment.address: http://app:9080/
restate_dev-1  |     deployment.service_protocol_version: 1
restate_dev-1  |     path: /invoke/roleUpdate/applyRoleUpdate
restate_dev-1  | 2024-07-11T11:52:18.954341Z WARN restate_invoker_impl
restate_dev-1  |   Error when executing the invocation, retrying in 964ms 638us 613ns.
restate_dev-1  |     error: [RT0010] error trying to connect: tcp connect error: Connection refused (os error 111)
restate_dev-1  |     restate.error.code: RT0010
restate_dev-1  |     restate.invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     restate.invocation.target: roleUpdate/applyRoleUpdate
restate_dev-1  | 
app-1          | [INFO] 11:52:19 ts-node-dev ver. 2.0.0 (using ts-node ver. 10.9.2, typescript ver. 5.5.3)
app-1          | [restate] [2024-07-11T11:52:19.391Z] INFO:  Listening on 9080...
restate_dev-1  | 2024-07-11T11:52:19.920411Z INFO restate_invoker_impl::invocation_task::service_protocol_runner
restate_dev-1  |   Executing invocation at deployment
restate_dev-1  |     invocation.id: inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1
restate_dev-1  |     deployment.address: http://app:9080/
restate_dev-1  |     deployment.service_protocol_version: 1
restate_dev-1  |     path: /invoke/roleUpdate/applyRoleUpdate
app-1          | [restate] [roleUpdate/applyRoleUpdate][inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1][2024-07-11T11:52:19.926Z] INFO:  Resuming (replaying) function.
app-1          | >>> Applied permission remove:allow for user Sam Beckett
app-1          | >>> Applied permission share:block for user Sam Beckett
app-1          | [restate] [roleUpdate/applyRoleUpdate][inv_1b1fQx4ev6kC7bv9gbULzV6puP3iWyhtC1][2024-07-11T11:52:20.321Z] INFO:  Function completed successfully.
100   315    0     0  100   315      0     90  0:00:03  0:00:03 --:--:--    90

compose.yaml

services:
  app:
    image: node:20.15
    working_dir: /app
    ports:
      - "9080:9080"
    command:
      - /bin/bash
      - -c
      - |
        if [ ! -d "examples" ]; then
          git clone --depth=1 https://github.com/restatedev/examples.git
        fi
        cd examples/basics/basics-typescript/
        npm install
        npm run example-1
  restate_dev:
    image: docker.io/restatedev/restate:1.0.2
    ports:
      - 8080:8080 
      - 9070:9070 
      - 9071:9071
  curl:
    image: docker.io/curlimages/curl:8.8.0
    command:
      - /bin/sh
      - -c 
      - |
        sleep 5
        curl --http2-prior-knowledge --retry 8 http://app:9080 --trace-ascii -
        curl -s http://restate_dev:9070/deployments -H 'content-type: application/json' -d '{"uri": "http://app:9080"}' --trace-ascii -
        sleep 1
        curl http://restate_dev:8080/roleUpdate/applyRoleUpdate -H 'content-type: application/json' -d \
        '{
            "userId": "Sam Beckett",
            "role": { "roleKey": "content-manager", "roleDescription": "Add/remove documents" },
            "permissions" : [
              { "permissionKey": "add", "setting": "allow" },
              { "permissionKey": "remove", "setting": "allow" },
              { "permissionKey": "share", "setting": "block" }
            ]
        }'
        sleep 1000
@tillrohrmann
Copy link
Contributor

Thanks a lot for your suggestion @y12studio. I like your idea to make it easier for first-time Restate users to get things up and running. The one thing I would actually exclude from the docker compose script is the actual curl. Do you think that such a compose file would be helpful?

services:
  endpoint:
    image: node:20.15
    working_dir: /endpoint
    ports:
      - "9080:9080"
    command:
      - /bin/bash
      - -c
      - |
        if [ ! -d "examples" ]; then
          git clone --depth=1 https://github.com/restatedev/examples.git
        fi
        cd examples/basics/basics-typescript/
        npm install
        npm run ${EXAMPLE:-example-1}
    healthcheck:
      test: curl --http2-prior-knowledge http://localhost:9080 || exit 1
      interval: 5s
      retries: 10
      start_period: 5s
      timeout: 10s
  restate:
    image: docker.io/restatedev/restate:latest
    pull_policy: always
    ports:
      - 8080:8080
      - 9070:9070
      - 9071:9071
  register_endpoint:
    image: docker.io/curlimages/curl:8.8.0
    depends_on:
      endpoint:
        condition: service_healthy
      restate:
        condition: service_started
    command:
      - /bin/sh
      - -c
      - |
        curl --retry 10 http://restate:9070/deployments -H 'content-type: application/json' -d '{"uri": "http://endpoint:9080"}'
        echo "Endpoint successfully registered"

@pavan4
Copy link

pavan4 commented Mar 12, 2025

This should be updated as examples got updated

services:
  endpoint:
    image: node:20.15
    working_dir: /endpoint
    ports:
      - "9080:9080"
    command:
      - /bin/bash
      - -c
      - |
        if [ ! -d "examples" ]; then
          git clone --depth=1 https://github.com/restatedev/examples.git
        fi
        cd examples/typescript/basics
        npm install
        npm run example-0 # or 2, 3
    healthcheck:
      test: curl --http2-prior-knowledge http://localhost:9080 || exit 1
      interval: 5s
      retries: 10
      start_period: 5s
      timeout: 10s
  restate:
    image: docker.io/restatedev/restate:latest
    pull_policy: always
    ports:
      - 8080:8080
      - 9070:9070
      - 9071:9071
  register_endpoint:
    image: docker.io/curlimages/curl:8.8.0
    depends_on:
      endpoint:
        condition: service_healthy
      restate:
        condition: service_started
    command:
      - /bin/sh
      - -c
      - |
        curl --retry 10 http://restate:9070/deployments -H 'content-type: application/json' -d '{"uri": "http://endpoint:9080"}'
        echo "Endpoint successfully registered"

@tillrohrmann
Copy link
Contributor

@pavan4 do you want to open a PR for such a docker compose setup?

@pavan4
Copy link

pavan4 commented Mar 20, 2025

@tillrohrmann #258

@tillrohrmann
Copy link
Contributor

Thanks @pavan4. I've pinged Giselle, who maintains our examples to take a look at it.

@pavan4 pavan4 mentioned this issue Mar 21, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pavan4 @y12studio @tillrohrmann