Nexus: worker, workflow-backed operations, and workflow caller #813

dandavison · 2025-04-02T19:06:35Z

Initial Python Temporal Nexus implementation.

Temporal SDK PR to accompany

Nexus SDK PR Initial implementation nexus-rpc/sdk-python#1
samples-python PR Nexus samples samples-python#174

Contains Nexus worker, components for users to define workflow-backed Nexus operations, and the ability to start and cancel a Nexus operation from a workflow.

Notes for reviewers

nexusrpc.handler.start_workflow is a top-level static function, but currently there's still a public contextvar object nexusrpc.handler.temporal_operation_context. Let's settle on approach there. If we're using module-level getters I think they need to be named something like get_client() etc, despite the fact that we have activity.metric_meter() ? Little bit more discussion needed there.
The workflow caller is in there, but needs more cleaning up. Feel free to ignore those files for now.
I'll do another cleanup pass on docstrings, API docs, etc.

cretz · 2025-06-25T14:19:31Z

README.md

@@ -94,6 +94,7 @@ informal introduction to the features and their implementation.
        - [Heartbeating and Cancellation](#heartbeating-and-cancellation)
        - [Worker Shutdown](#worker-shutdown)
      - [Testing](#testing-1)
+    - [Nexus](#nexus)


I do not see this section

cretz · 2025-06-25T14:20:00Z

pyproject.toml

+[tool.uv.sources]
+nexus-rpc = { path = "../nexus-sdk-python", editable = true }


Let's make sure not to merge until this is proper dependency

cretz · 2025-06-25T14:21:22Z

temporalio/client.py

        priority: temporalio.common.Priority = temporalio.common.Priority.default,
+        # The following options are deliberately not exposed in overloads


Unresolved (but pedantic, don't have to take if not wanted, I am just afraid of users using them)

cretz · 2025-06-25T14:23:53Z

temporalio/client.py

+    nexus_completion_callbacks: Sequence[NexusCompletionCallback]
+    workflow_event_links: Sequence[temporalio.api.common.v1.Link.WorkflowEvent]
+    request_id: Optional[str]


May want to mention here these are unstable/experimental

cretz · 2025-06-25T14:24:23Z

temporalio/client.py

@@ -7231,6 +7260,17 @@ def api_key(self, value: Optional[str]) -> None:
        self.service_client.update_api_key(value)


+@dataclass(frozen=True)
+class NexusCompletionCallback:


May want to mention this is unstable/experimental and also not really for user use (I understand exposing because it's exposed in the interceptor)

cretz · 2025-06-25T14:56:40Z

temporalio/nexus/handler/_token.py

+    if not all(
+        c in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_-"
+        for c in s
+    ):
+        raise ValueError(
+            "invalid base64URL encoded string: contains invalid characters"
+        )


Pedantic and although not really documented well, I think urlsafe_b64decode will fail on invalid chars (though can check there are no trailing = if needed)

cretz · 2025-06-25T15:01:16Z

temporalio/worker/_interceptor.py

@@ -3,22 +3,27 @@
 from __future__ import annotations


This is in conflict with the statement on the PR description of "Feel free to ignore those files for now". Ignoring workflow caller stuff for now.

cretz · 2025-06-25T15:08:53Z

temporalio/worker/_nexus.py

+        # TODO(nexus-prerelease): headers
+        try:
+            await self._handler.cancel_operation(ctx, request.operation_token)
+        except Exception as err:


So this is the case where I think we should consider catching BaseException. I think we need to swallow all user exceptions that can occur for any reason and respond back

cretz · 2025-06-25T15:09:43Z

temporalio/worker/_nexus.py

+        try:
+            await self._handler.cancel_operation(ctx, request.operation_token)
+        except Exception as err:
+            logger.exception("Failed to execute Nexus cancel operation method")


We had traditionally treated these types of things as "warning" level not "error" level because it is not an SDK error, it is a user-code error, the SDK is just fine.

cretz · 2025-06-25T15:13:44Z

temporalio/worker/_nexus.py

+        try:
+            await self._handler.cancel_operation(ctx, request.operation_token)
+        except Exception as err:
+            logger.exception("Failed to execute Nexus cancel operation method")


If possible, use the contextual logger that has the Nexus info on it for this and start exception

cretz · 2025-06-25T15:47:37Z

temporalio/nexus/handler/_operation_handlers.py

+
+    def __init__(
+        self,
+        start: Optional[


Just for posterity, noting that I disagree with this approach vs a simple approach of instantiating this class with the start workflow args. I think making users do nested functions for something as simple as tying a workflow to an operation is rough.

OK let's not treat that discussion as closed.

- Expose nexus.client(), nexus.info() - Don't expose contextvar

Accidentally broken at f8077c5

dandavison force-pushed the nexus branch 2 times, most recently from a845a6f to 7a121bb Compare April 5, 2025 15:08

dandavison force-pushed the nexus branch 10 times, most recently from 862e4f9 to 8ac0192 Compare April 17, 2025 12:17

dandavison force-pushed the nexus branch 4 times, most recently from 8940d51 to 520aecf Compare April 26, 2025 16:16

dandavison force-pushed the nexus branch 2 times, most recently from 2fc971f to 8bd6011 Compare April 29, 2025 13:11

dandavison force-pushed the nexus branch from 18e268a to 2eb967a Compare May 6, 2025 15:55

dandavison changed the title ~~Nexus prototype~~ Nexus May 8, 2025

dandavison force-pushed the nexus branch from 7876ecc to 7fe6868 Compare May 8, 2025 22:07

dandavison force-pushed the nexus branch 2 times, most recently from 5d54f48 to 8ddfb94 Compare May 24, 2025 01:49

dandavison force-pushed the nexus branch 7 times, most recently from fa3e8ec to f7bf47b Compare May 27, 2025 20:34

dandavison added 2 commits June 25, 2025 11:06

Parameterize workflow_run_operation tests

1af01d3

Failing test case

1319732

cretz reviewed Jun 25, 2025

View reviewed changes

dandavison added 15 commits June 25, 2025 18:27

Test: clean up imports

320e1f1

Respond to upstream: sync_operation_handler

f8077c5

New workflow_run_operation_handler

f74784a

Delete reference to obsolete __nexus_service_metadata__

8e0af0a

TODO

d9bef4e

Use get_callable_name utility

6836d4b

Fix test: 'not an async def` message changed

2a4b7cf

Refactor

d45317e

Reorganize: temporalio.nexus.handler -> temporalo.nexus

740a822

Fix signatures of start_method on workflow caller side

64e4b01

from temporalio import nexus everywhere

f0dba64

Qualify client.WorkflowHandle in temporalio.nexus

8c76a6f

Fixup: no coverage for these

97272ed

Rename: nexus.WorkflowHandle

5ff174c

nexus.WorkflowHandle.{to,from}_token()

8f4b524

dandavison force-pushed the nexus branch from 4121452 to 8f4b524 Compare June 26, 2025 03:15

dandavison added 6 commits June 26, 2025 06:50

Respond to upstream: sync_operation_handler -> sync_operation

e090847

workflow_run_operation_handler -> workflow_run_operation

a398dc6

from nexus import workflow_run_operation

83791fd

Respond to upstream: operation_handler is not in the public API

7cf8722

New nexus operation context API

0e221c8

- Expose nexus.client(), nexus.info() - Don't expose contextvar

Fix broken test

e201590

Accidentally broken at f8077c5

dandavison force-pushed the nexus branch from 33ea9fc to e201590 Compare June 26, 2025 12:37

dandavison added 3 commits June 26, 2025 08:54

Fix another test

0ffc419

Move Failure tests utility

ec3d28a

Fix test

2bda761

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Nexus: worker, workflow-backed operations, and workflow caller #813

Nexus: worker, workflow-backed operations, and workflow caller #813

Uh oh!

dandavison commented Apr 2, 2025 •

edited

Loading

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

cretz Jun 25, 2025

Uh oh!

dandavison Jun 25, 2025

Uh oh!

Uh oh!

		[tool.uv.sources]
		nexus-rpc = { path = "../nexus-sdk-python", editable = true }

		priority: temporalio.common.Priority = temporalio.common.Priority.default,
		# The following options are deliberately not exposed in overloads

Nexus: worker, workflow-backed operations, and workflow caller #813

Are you sure you want to change the base?

Nexus: worker, workflow-backed operations, and workflow caller #813

Uh oh!

Conversation

dandavison commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notes for reviewers

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dandavison commented Apr 2, 2025 •

edited

Loading