feat: V2 library worker #1667
feat: V2 library worker #1667
Conversation
…e/azure-functions-python-worker into hallvictoria/library-worker
| is_deferred_binding: typing.Optional[bool] = False) -> typing.Any: | ||
| binding = get_binding(binding, is_deferred_binding) | ||
| is_deferred_binding: Optional[bool] = False) -> Any: | ||
| binding_obj = get_binding(binding, is_deferred_binding) |
There was a problem hiding this comment.
Renamed this variable so we're not overwriting binding
| @@ -4,12 +4,31 @@ | |||
| from dataclasses import dataclass | |||
| from enum import Enum | |||
|
|
|||
| from . import rpcexception | |||
|
|
|||
| class RpcException: | |||
There was a problem hiding this comment.
Moved this class into retrycontext.py.
| from ._thirdparty import typing_inspect | ||
| from .constants import HTTP_TRIGGER | ||
| from .protos import BindingInfo | ||
| from .bindings.meta import (has_implicit_output, |
There was a problem hiding this comment.
Refactoring: instead of importing the module as bindings_utils, we import the relevant methods and use them directly.
| return_type) | ||
|
|
||
| def add_indexed_function(self, function): | ||
| def add_indexed_function(self, function, protos): |
There was a problem hiding this comment.
For V2, we only need add_indexed_function
| @@ -0,0 +1,437 @@ | |||
| # Copyright (c) Microsoft Corporation. All rights reserved. | |||
There was a problem hiding this comment.
This file replaces dispatcher.py in the previous worker. Here is where all the definitions for init, load, metadata, env reload, and invocation request are.
| from .utils.tracing import serialize_exception | ||
| from .utils.validators import validate_script_file_name | ||
|
|
||
| metadata_result: Optional[List] = None |
There was a problem hiding this comment.
We have global definitions for variables that are used across different methods and need to be saved.
|
|
||
|
|
||
| async def worker_init_request(request): | ||
| logger.info("V2 Library Worker: received WorkerInitRequest," |
There was a problem hiding this comment.
This is a new log added, and mirrors a log added in env reload. Here we log if the worker is V1 or V2 and what version is being used.
| init_request = request.request.worker_init_request | ||
| host_capabilities = init_request.capabilities | ||
| _host = request.properties.get("host") | ||
| protos = request.properties.get("protos") |
There was a problem hiding this comment.
_host and protos are two variables sent from the proxy worker. They are needed for streaming and handling protos objects, respectively
| if is_envvar_true(PYTHON_APPLICATIONINSIGHTS_ENABLE_TELEMETRY): | ||
| initialize_azure_monitor() | ||
|
|
||
| if is_envvar_true(PYTHON_ENABLE_OPENTELEMETRY): |
There was a problem hiding this comment.
otel logic has been refactored into otel.py. Requesting a double check on this refactoring!
|
|
||
| # Index in init by default | ||
| try: | ||
| load_function_metadata( |
There was a problem hiding this comment.
We index in init by default now. With the extended timeout limit, we will not hit the same issues as previously.
|
|
||
|
|
||
| async def function_load_request(request): | ||
| logger.debug("V2 Library Worker: received WorkerLoadRequest") |
There was a problem hiding this comment.
This is a no-op for the V2 library worker, but this method still needs to be defined in order to work with the proxy worker.
| function_id) | ||
| assert fi is not None | ||
| logger.info("Function name: %s, Function Type: %s", | ||
| fi.name, |
There was a problem hiding this comment.
This is the third and last new log added. We log the function name and type (async vs. sync). This information was previously logged, so we log it here as well to maintain comprehensive logging.
| configure_opentelemetry(fi_context) | ||
|
|
||
| # Extensions are not supported | ||
| call_result = await execute_async(fi.func, args) |
There was a problem hiding this comment.
Extensions are not supported in 3.13+
| + str(indexed_function_bindings_logs)) | ||
| indexed_function_logs.append(function_log) | ||
|
|
||
| log_data = { |
There was a problem hiding this comment.
This formats the log into a JSON that we can easily parse in Kusto. We log the app setting state here alone -- this ensures that the app settings that are logged are accurate.
| import traceback | ||
|
|
||
| # Logging Prefixes | ||
| SDK_LOG_PREFIX = "azure.functions" |
There was a problem hiding this comment.
Todo: change log prefix?
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. | ||
|
|
||
| import asyncio |
There was a problem hiding this comment.
Refactor: moved methods related to the loop & executing out from dispatcher.py
| @@ -49,31 +47,7 @@ extends: | |||
| - stage: Build | |||
| jobs: | |||
| - template: /eng/templates/official/jobs/build-artifacts.yml@self | |||
There was a problem hiding this comment.
There will be only unit tests for the library worker. E2E tests will remain in the proxy worker.
Description
The library worker holds the main logic for responding to the host, indexing function apps, and executing functions.
Fixes #
PR information
Quality of Code and Contribution Guidelines