diff --git a/azurefunctions-extensions-bindings-servicebus/LICENSE b/azurefunctions-extensions-bindings-servicebus/LICENSE new file mode 100644 index 0000000..63447fd --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/LICENSE @@ -0,0 +1,21 @@ +Copyright (c) Microsoft Corporation. + +MIT License + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/MANIFEST.in b/azurefunctions-extensions-bindings-servicebus/MANIFEST.in new file mode 100644 index 0000000..e1ae5ad --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/MANIFEST.in @@ -0,0 +1,3 @@ +recursive-include azure *.py *.pyi +recursive-include tests *.py +include LICENSE README.md \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/README.md b/azurefunctions-extensions-bindings-servicebus/README.md new file mode 100644 index 0000000..0326d87 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/README.md @@ -0,0 +1,102 @@ +# Azure Functions Extensions Bindings ServiceBus library for Python +This library allows ServiceBus Triggers in Python Function Apps to recognize and bind to client types from the +Azure ServiceBus sdk. + +The SDK types can be generated from: + +* ServiceBus Triggers + +The supported ServiceBus SDK types include: + +* ServiceBusReceivedMessage + +[Source code](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-servicebus) +| +[Package (PyPi)](https://pypi.org/project/azurefunctions-extensions-bindings-servicebus/) +| [Samples](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-servicebus/samples) + + +## Getting started + +### Prerequisites +* Python 3.9 or later is required to use this package. For more details, please read our page on [Python Functions version support policy](https://learn.microsoft.com/en-us/azure/azure-functions/functions-versions?tabs=isolated-process%2Cv4&pivots=programming-language-python#languages). + +* You must have an [Azure subscription](https://azure.microsoft.com/free/) and a +[ServiceBus Resource](https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-service-bus?tabs=isolated-process%2Cextensionv5%2Cextensionv3&pivots=programming-language-python) to use this package. + +### Install the package +Install the Azure Functions Extensions Bindings ServiceBus library for Python with pip: + +```bash +pip install azurefunctions-extensions-bindings-servicebus +``` + + +### Bind to the SDK-type +The Azure Functions Extensions Bindings ServiceBus library for Python allows you to create a function app with a ServiceBus Trigger +and define the type as a ServiceBusReceivedMessage. Instead of receiving +a ServiceBusMessage, when the function is executed, the type returned will be the defined SDK-type and have all the +properties and methods available as seen in the Azure ServiceBus library for Python. + + +```python +import logging +import azure.functions as func +import azurefunctions.extensions.bindings.servicebus as servicebus + +app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION) + +@app.service_bus_queue_trigger(arg_name="receivedmessage", + queue_name="QUEUE_NAME", + connection="SERVICEBUS_CONNECTION") +def servicebus_queue_trigger(receivedmessage: servicebus.ServiceBusReceivedMessage): + logging.info("Python ServiceBus queue trigger processed message.") + logging.info("Receiving: %s\n" + "Body: %s\n" + "Enqueued time: %s\n" + "Lock Token: %s\n" + "Locked until : %s\n" + "Message ID: %s\n" + "Sequence number: %s\n", + receivedmessage, + receivedmessage.body, + receivedmessage.enqueued_time_utc, + receivedmessage.lock_token, + receivedmessage.locked_until, + receivedmessage.message_id, + receivedmessage.sequence_number) +``` + +## Troubleshooting +### General +The SDK-types raise exceptions defined in [Azure Core](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md). + +This list can be used for reference to catch thrown exceptions. To get the specific error code of the exception, use the `error_code` attribute, i.e, `exception.error_code`. + +## Next steps + +### More sample code + +Get started with our [ServiceBus samples](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-servicebus/samples). + +Several samples are available in this GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Azure ServiceBus: + +* [servicebus_samples_single](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single) - Examples for using the ServiceBusReceivedMessage type: + * From ServiceBus Queue Trigger (Single Message) + * From ServiceBus Topic Trigger (Single Message) + +* [servicebus_samples_batch](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-servicebus/samples/service_samples_batch) - Examples for interacting with batches: + * From ServiceBus Queue Trigger (Batch) + * From ServiceBus Topic Trigger (Batch) + + +### Additional documentation +For more information on the Azure ServiceBus SDK, see the [Azure ServiceBus SDK documentation](https://learn.microsoft.com/en-us/python/api/overview/azure/servicebus-readme?view=azure-python) on docs.microsoft.com +and the [Azure ServiceBus README](https://github.com/Azure/azure-sdk-for-python/blob/azure-servicebus_7.14.1/sdk/servicebus/azure-servicebus/README.md). + +## Contributing +This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com. + +When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. + +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/azurefunctions/__init__.py b/azurefunctions-extensions-bindings-servicebus/azurefunctions/__init__.py new file mode 100644 index 0000000..8db66d3 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/azurefunctions/__init__.py @@ -0,0 +1 @@ +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/__init__.py b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/__init__.py new file mode 100644 index 0000000..8db66d3 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/__init__.py @@ -0,0 +1 @@ +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/__init__.py b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/__init__.py new file mode 100644 index 0000000..8db66d3 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/__init__.py @@ -0,0 +1 @@ +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/__init__.py b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/__init__.py new file mode 100644 index 0000000..1d7c21c --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/__init__.py @@ -0,0 +1,12 @@ +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. + +from .serviceBusReceivedMessage import ServiceBusReceivedMessage +from .serviceBusConverter import ServiceBusConverter + +__all__ = [ + "ServiceBusReceivedMessage", + "ServiceBusConverter", +] + +__version__ = '1.0.0a1' diff --git a/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/serviceBusConverter.py b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/serviceBusConverter.py new file mode 100644 index 0000000..9897835 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/serviceBusConverter.py @@ -0,0 +1,74 @@ +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. + +import collections.abc +from typing import Any, get_args, get_origin + +from azurefunctions.extensions.base import Datum, InConverter +from .serviceBusReceivedMessage import ServiceBusReceivedMessage + + +class ServiceBusConverter( + InConverter, + binding='serviceBusTrigger', trigger=True +): + @classmethod + def check_input_type_annotation(cls, pytype: type) -> bool: + if pytype is None: + return False + + # The annotation is a class/type (not an object) - not iterable + if (isinstance(pytype, type) + and issubclass(pytype, ServiceBusReceivedMessage)): + return True + + # An iterable who only has one inner type and is a subclass of SdkType + return cls._is_iterable_supported_type(pytype) + + @classmethod + def _is_iterable_supported_type(cls, annotation: type) -> bool: + # Check base type from type hint. Ex: List from List[SdkType] + base_type = get_origin(annotation) + if (base_type is None + or not issubclass(base_type, collections.abc.Iterable)): + return False + + inner_types = get_args(annotation) + if inner_types is None or len(inner_types) != 1: + return False + + inner_type = inner_types[0] + + return (isinstance(inner_type, type) + and issubclass(inner_type, ServiceBusReceivedMessage)) + + @classmethod + def decode(cls, data: Datum, *, trigger_metadata, pytype) -> Any: + """ + ServiceBus allows for batches to be sent. The cardinality can be one or many. + When the cardinality is one: + - The data is of type "model_binding_data" - each event is an independent + function invocation + When the cardinality is many: + - The data is of type "collection_model_binding_data" - all events are sent + in a single function invocation + - collection_model_binding_data has 1 or more model_binding_data objects + """ + if data is None or data.type is None: + return None + + data_type = data.type + + if data_type == "model_binding_data": + return ServiceBusReceivedMessage(data=data.value).get_sdk_type() + elif data_type == "collection_model_binding_data": + try: + return [ServiceBusReceivedMessage(data=mbd).get_sdk_type() + for mbd in data.value.model_binding_data] + except Exception as e: + raise ValueError("Failed to decode incoming ServiceBus batch: " + + repr(e)) from e + else: + raise ValueError( + "Unexpected type of data received for the 'servicebus' binding: " + + repr(data.type)) diff --git a/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/serviceBusReceivedMessage.py b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/serviceBusReceivedMessage.py new file mode 100644 index 0000000..e5e7e7f --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/serviceBusReceivedMessage.py @@ -0,0 +1,33 @@ +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. + +from azure.servicebus import ServiceBusReceivedMessage as ServiceBusReceivedMessageSdk +from azurefunctions.extensions.base import Datum, SdkType +from .utils import get_decoded_message + + +class ServiceBusReceivedMessage(SdkType): + def __init__(self, *, data: Datum) -> None: + # model_binding_data properties + self._data = data + self._version = None + self._source = None + self._content_type = None + self._content = None + self._decoded_message = None + if self._data: + self._version = data.version + self._source = data.source + self._content_type = data.content_type + self._content = data.content + self._decoded_message = get_decoded_message(self._content) + + def get_sdk_type(self): + """ + Returns a ServiceBusReceivedMessage. + Message settling is not yet supported. + """ + if self._decoded_message: + return ServiceBusReceivedMessageSdk(self._decoded_message, receiver=None) + else: + return None diff --git a/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/utils.py b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/utils.py new file mode 100644 index 0000000..0d7fea9 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/azurefunctions/extensions/bindings/servicebus/utils.py @@ -0,0 +1,53 @@ +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. + +import uamqp +import uuid + + +_X_OPT_LOCK_TOKEN = b"x-opt-lock-token" + + +def get_lock_token(message: bytes, index: int) -> str: + # Get the lock token from the message + lock_token_encoded = message[:index] + + # Convert the lock token to a UUID using the first 16 bytes + lock_token_uuid = uuid.UUID(bytes=lock_token_encoded[:16]) + + return lock_token_uuid + + +def get_amqp_message(message: bytes, index: int): + """ + Get the amqp message from the model_binding_data content + and create the message. + """ + amqp_message = message[index + len(_X_OPT_LOCK_TOKEN):] + decoded_message = uamqp.Message().decode_from_bytes(amqp_message) + + return decoded_message + + +def get_decoded_message(content: bytes): + """ + First, find the end of the lock token. Then, + get the lock token UUID and create the delivery + annotations dictionary. Finally, get the amqp message + and set the delivery annotations. Once the delivery + annotations have been set, the amqp message is ready to + return. + """ + if content: + try: + index = content.find(_X_OPT_LOCK_TOKEN) + + lock_token = get_lock_token(content, index) + delivery_anno_dict = {_X_OPT_LOCK_TOKEN: lock_token} + + decoded_message = get_amqp_message(content, index) + decoded_message.delivery_annotations = delivery_anno_dict + return decoded_message + except Exception as e: + raise ValueError(f"Failed to decode ServiceBus content: {e}") from e + return None diff --git a/azurefunctions-extensions-bindings-servicebus/pyproject.toml b/azurefunctions-extensions-bindings-servicebus/pyproject.toml new file mode 100644 index 0000000..cacdfb2 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/pyproject.toml @@ -0,0 +1,52 @@ +[build-system] +requires = ["setuptools >= 61.0"] +build-backend = "setuptools.build_meta" + +[project] +name = "azurefunctions-extensions-bindings-servicebus" +dynamic = ["version"] +requires-python = ">=3.9" +authors = [{ name = "Azure Functions team at Microsoft Corp.", email = "azurefunctions@microsoft.com"}] +description = "ServiceBus Python worker extension for Azure Functions." +readme = "README.md" +license = {text = "MIT License"} +classifiers= [ + 'License :: OSI Approved :: MIT License', + 'Intended Audience :: Developers', + 'Programming Language :: Python :: 3', + 'Programming Language :: Python :: 3.9', + 'Programming Language :: Python :: 3.10', + 'Programming Language :: Python :: 3.11', + 'Programming Language :: Python :: 3.12', + 'Programming Language :: Python :: 3.13', + 'Operating System :: Microsoft :: Windows', + 'Operating System :: POSIX', + 'Operating System :: MacOS :: MacOS X', + 'Environment :: Web Environment', + 'Development Status :: 5 - Production/Stable', + ] +dependencies = [ + 'azurefunctions-extensions-base', + 'azure-servicebus~=7.14.2', + 'uamqp~=1.6.11' + ] + +[project.optional-dependencies] +dev = [ + 'flake8', + 'mypy', + 'pytest', + 'pytest-cov', + 'coverage', + 'pytest-instafail', + 'pre-commit' + ] + +[tool.setuptools.dynamic] +version = {attr = "azurefunctions.extensions.bindings.servicebus.__version__"} + +[tool.setuptools.packages.find] +exclude = [ + 'azurefunctions.extensions.bindings','azurefunctions.extensions', + 'azurefunctions', 'tests', 'samples' + ] diff --git a/azurefunctions-extensions-bindings-servicebus/samples/README.md b/azurefunctions-extensions-bindings-servicebus/samples/README.md new file mode 100644 index 0000000..947f5f3 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/README.md @@ -0,0 +1,61 @@ +--- +page_type: sample +languages: + - python +products: + - azure + - azure-functions + - azure-functions-extensions + - azurefunctions-extensions-bindings-servicebus +urlFragment: extension-servicebus-samples +--- + +# Azure Functions Extension ServiceBus library for Python samples + +These are code samples that show common scenario operations with the Azure Functions Extension ServiceBus library. + +These samples relate to the Azure ServiceBus client library being used as part of a Python Function App. For +examples on how to use the Azure ServiceBus client library, please see [Azure ServiceBus samples](https://github.com/Azure/azure-sdk-for-python/tree/azure-servicebus_7.14.1/sdk/servicebus/azure-servicebus/samples) + +* [servicebus_samples_single](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single) - Examples for using the ServiceBusReceivedMessage type: + * From ServiceBus Queue Trigger (Single Message) + * From ServiceBus Topic Trigger (Single Message) +* [servicebus_samples_batch](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch) - Examples for using the ServiceBusReceivedMessage type: + * From ServiceBus Queue Trigger (Batch) + * From ServiceBus Topic Trigger (Batch) + + +## Prerequisites +* Python 3.9 or later is required to use this package. For more details, please read our page on [Python Functions version support policy](https://learn.microsoft.com/en-us/azure/azure-functions/functions-versions?tabs=isolated-process%2Cv4&pivots=programming-language-python#languages). + +## Setup + +1. Install [Core Tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?tabs=windows%2Cisolated-process%2Cnode-v4%2Cpython-v2%2Chttp-trigger%2Ccontainer-apps&pivots=programming-language-python) +2. Install the Azure Functions Extension ServiceBus library for Python with [pip](https://pypi.org/project/pip/): + +```bash +pip install azurefunctions-extensions-bindings-servicebus +``` + +3. Clone or download this sample repository +4. Open the sample folder in Visual Studio Code or your IDE of choice. + +## Running the samples + +1. Open a terminal window and `cd` to the directory that the sample you wish to run is saved in. +2. Set the environment variables specified in the sample file you wish to run. +3. Install the required dependencies +```bash +pip install -r requirements.txt +``` +4. Start the Functions runtime +```bash +func start +``` +5. Execute the function by triggering the ServiceBus entity. + +## Next steps + +Visit the [SDK-type bindings in Python reference documentation](https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=get-started%2Casgi%2Capplication-level&pivots=python-mode-decorators#sdk-type-bindings-preview) to learn more about how to use SDK-type bindings in a Python Function App and the +[API reference documentation](https://learn.microsoft.com/en-us/python/api/overview/azure/service-bus?view=azure-python) to learn more about +what you can do with the Azure ServiceBus client library. \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/function_app.py b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/function_app.py new file mode 100644 index 0000000..4017a0a --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/function_app.py @@ -0,0 +1,79 @@ +# coding: utf-8 + +# ------------------------------------------------------------------------- +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. See License.txt in the project root for +# license information. +# -------------------------------------------------------------------------- + +import logging +from typing import List + +import azure.functions as func +import azurefunctions.extensions.bindings.servicebus as servicebus + +app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION) + +""" +FOLDER: servicebus_samples +DESCRIPTION: + These samples demonstrate how to obtain a ServiceBusReceivedMessage + from a ServiceBus Trigger. +USAGE: + Set the environment variables with your own values before running the + sample: + For running the ServiceBus queue trigger function: + 1) QUEUE_NAME - the name of the ServiceBus queue + 2) SERVICEBUS_CONNECTION - the connection string for the ServiceBus entity + For running the ServiceBus topic trigger function: + 1) TOPIC_NAME - the name of the ServiceBus topic + 2) SERVICEBUS_CONNECTION - the connection string for the ServiceBus entity + 3) SUBSCRIPTION_NAME - the name of the Subscription +""" + + +@app.service_bus_queue_trigger(arg_name="receivedmessage", + queue_name="QUEUE_NAME", + connection="SERVICEBUS_CONNECTION", + cardinality="many") +def servicebus_queue_trigger(receivedmessage: List[servicebus.ServiceBusReceivedMessage]): + logging.info("Python ServiceBus queue trigger processed message.") + for message in receivedmessage: + logging.info("Receiving: %s\n" + "Body: %s\n" + "Enqueued time: %s\n" + "Lock Token: %s\n" + "Locked until : %s\n" + "Message ID: %s\n" + "Sequence number: %s\n", + message, + message.body, + message.enqueued_time_utc, + message.lock_token, + message.locked_until, + message.message_id, + message.sequence_number) + + +@app.service_bus_topic_trigger(arg_name="receivedmessage", + topic_name="TOPIC_NAME", + connection="SERVICEBUS_CONNECTION", + subscription_name="SUBSCRIPTION_NAME", + cardinality="many") +def servicebus_topic_trigger(receivedmessage: List[servicebus.ServiceBusReceivedMessage]): + logging.info("Python ServiceBus topic trigger processed message.") + for message in receivedmessage: + logging.info("Receiving: %s\n" + "Body: %s\n" + "Enqueued time: %s\n" + "Lock Token: %s\n" + "Locked until : %s\n" + "Message ID: %s\n" + "Sequence number: %s\n", + message, + message.body, + message.enqueued_time_utc, + message.lock_token, + message.locked_until, + message.message_id, + message.sequence_number) diff --git a/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/host.json b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/host.json new file mode 100644 index 0000000..9df9136 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/host.json @@ -0,0 +1,15 @@ +{ + "version": "2.0", + "logging": { + "applicationInsights": { + "samplingSettings": { + "isEnabled": true, + "excludedTypes": "Request" + } + } + }, + "extensionBundle": { + "id": "Microsoft.Azure.Functions.ExtensionBundle", + "version": "[4.*, 5.0.0)" + } +} \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/local.settings.json b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/local.settings.json new file mode 100644 index 0000000..967f18b --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/local.settings.json @@ -0,0 +1,11 @@ +{ + "IsEncrypted": false, + "Values": { + "FUNCTIONS_WORKER_RUNTIME": "python", + "AzureWebJobsStorage": "UseDevelopmentStorage=true", + "QUEUE_NAME": "", + "SERVICEBUS_CONNECTION": "", + "TOPIC_NAME": "", + "SUBSCRIPTION_NAME": "" + } +} \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/requirements.txt b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/requirements.txt new file mode 100644 index 0000000..7b4c53b --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_batch/requirements.txt @@ -0,0 +1,6 @@ +# DO NOT include azure-functions-worker in this file +# The Python Worker is managed by Azure Functions platform +# Manually managing azure-functions-worker may cause unexpected issues + +azure-functions +azurefunctions-extensions-bindings-servicebus \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/function_app.py b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/function_app.py new file mode 100644 index 0000000..c877378 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/function_app.py @@ -0,0 +1,74 @@ +# coding: utf-8 + +# ------------------------------------------------------------------------- +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. See License.txt in the project root for +# license information. +# -------------------------------------------------------------------------- + +import logging + +import azure.functions as func +import azurefunctions.extensions.bindings.servicebus as servicebus + +app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION) + +""" +FOLDER: servicebus_samples +DESCRIPTION: + These samples demonstrate how to obtain a ServiceBusReceivedMessage + from a ServiceBus Trigger. +USAGE: + Set the environment variables with your own values before running the + sample: + For running the ServiceBus queue trigger function: + 1) QUEUE_NAME - the name of the ServiceBus queue + 2) SERVICEBUS_CONNECTION - the connection string for the ServiceBus entity + For running the ServiceBus topic trigger function: + 1) TOPIC_NAME - the name of the ServiceBus topic + 2) SERVICEBUS_CONNECTION - the connection string for the ServiceBus entity + 3) SUBSCRIPTION_NAME - the name of the Subscription +""" + + +@app.service_bus_queue_trigger(arg_name="receivedmessage", + queue_name="QUEUE_NAME", + connection="SERVICEBUS_CONNECTION") +def servicebus_queue_trigger(receivedmessage: servicebus.ServiceBusReceivedMessage): + logging.info("Python ServiceBus queue trigger processed message.") + logging.info("Receiving: %s\n" + "Body: %s\n" + "Enqueued time: %s\n" + "Lock Token: %s\n" + "Locked until : %s\n" + "Message ID: %s\n" + "Sequence number: %s\n", + receivedmessage, + receivedmessage.body, + receivedmessage.enqueued_time_utc, + receivedmessage.lock_token, + receivedmessage.locked_until, + receivedmessage.message_id, + receivedmessage.sequence_number) + + +@app.service_bus_topic_trigger(arg_name="receivedmessage", + topic_name="TOPIC_NAME", + connection="SERVICEBUS_CONNECTION", + subscription_name="SUBSCRIPTION_NAME") +def servicebus_topic_trigger(receivedmessage: servicebus.ServiceBusReceivedMessage): + logging.info("Python ServiceBus topic trigger processed message.") + logging.info("Receiving: %s\n" + "Body: %s\n" + "Enqueued time: %s\n" + "Lock Token: %s\n" + "Locked until : %s\n" + "Message ID: %s\n" + "Sequence number: %s\n", + receivedmessage, + receivedmessage.body, + receivedmessage.enqueued_time_utc, + receivedmessage.lock_token, + receivedmessage.locked_until, + receivedmessage.message_id, + receivedmessage.sequence_number) \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/host.json b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/host.json new file mode 100644 index 0000000..9df9136 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/host.json @@ -0,0 +1,15 @@ +{ + "version": "2.0", + "logging": { + "applicationInsights": { + "samplingSettings": { + "isEnabled": true, + "excludedTypes": "Request" + } + } + }, + "extensionBundle": { + "id": "Microsoft.Azure.Functions.ExtensionBundle", + "version": "[4.*, 5.0.0)" + } +} \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/local.settings.json b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/local.settings.json new file mode 100644 index 0000000..967f18b --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/local.settings.json @@ -0,0 +1,11 @@ +{ + "IsEncrypted": false, + "Values": { + "FUNCTIONS_WORKER_RUNTIME": "python", + "AzureWebJobsStorage": "UseDevelopmentStorage=true", + "QUEUE_NAME": "", + "SERVICEBUS_CONNECTION": "", + "TOPIC_NAME": "", + "SUBSCRIPTION_NAME": "" + } +} \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/requirements.txt b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/requirements.txt new file mode 100644 index 0000000..7b4c53b --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/samples/servicebus_samples_single/requirements.txt @@ -0,0 +1,6 @@ +# DO NOT include azure-functions-worker in this file +# The Python Worker is managed by Azure Functions platform +# Manually managing azure-functions-worker may cause unexpected issues + +azure-functions +azurefunctions-extensions-bindings-servicebus \ No newline at end of file diff --git a/azurefunctions-extensions-bindings-servicebus/tests/__init__.py b/azurefunctions-extensions-bindings-servicebus/tests/__init__.py new file mode 100644 index 0000000..2b28c76 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/tests/__init__.py @@ -0,0 +1,18 @@ +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. + +import os.path +import sys +import unittest +import unittest.runner + + +def suite(): + test_loader = unittest.TestLoader() + return test_loader.discover(os.path.dirname(__file__), pattern="test_*.py") + + +if __name__ == "__main__": + runner = unittest.runner.TextTestRunner() + result = runner.run(suite()) + sys.exit(not result.wasSuccessful()) diff --git a/azurefunctions-extensions-bindings-servicebus/tests/test_code_quality.py b/azurefunctions-extensions-bindings-servicebus/tests/test_code_quality.py new file mode 100644 index 0000000..fcf1ff5 --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/tests/test_code_quality.py @@ -0,0 +1,53 @@ +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. +import pathlib +import subprocess +import sys +import unittest + +ROOT_PATH = pathlib.Path(__file__).parent.parent.parent + + +class TestCodeQuality(unittest.TestCase): + def test_mypy(self): + try: + import mypy # NoQA + except ImportError as e: + raise unittest.SkipTest('mypy module is missing') from e + + try: + subprocess.run( + [sys.executable, '-m', 'mypy', '-m', + 'azurefunctions-extensions-bindings-servicebus'], + check=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + cwd=str(ROOT_PATH)) + except subprocess.CalledProcessError as ex: + output = ex.output.decode() + raise AssertionError( + f'mypy validation failed:\n{output}') from None + + def test_flake8(self): + try: + import flake8 # NoQA + except ImportError as e: + raise unittest.SkipTest('flake8 module is missing') from e + + config_path = ROOT_PATH / '.flake8' + if not config_path.exists(): + raise unittest.SkipTest('could not locate the .flake8 file') + + try: + subprocess.run( + [sys.executable, '-m', 'flake8', + 'azurefunctions-extensions-bindings-servicebus', + '--config', str(config_path)], + check=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + cwd=str(ROOT_PATH)) + except subprocess.CalledProcessError as ex: + output = ex.output.decode() + raise AssertionError( + f'flake8 validation failed:\n{output}') from None diff --git a/azurefunctions-extensions-bindings-servicebus/tests/test_servicebus.py b/azurefunctions-extensions-bindings-servicebus/tests/test_servicebus.py new file mode 100644 index 0000000..1bc85ce --- /dev/null +++ b/azurefunctions-extensions-bindings-servicebus/tests/test_servicebus.py @@ -0,0 +1,153 @@ +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. + +import unittest +from typing import List, Optional + + +from azure.servicebus import ServiceBusReceivedMessage as ServiceBusSDK +from azurefunctions.extensions.base import Datum + +from azurefunctions.extensions.bindings.servicebus import (ServiceBusReceivedMessage, + ServiceBusConverter) +from azurefunctions.extensions.bindings.servicebus.utils import get_decoded_message + + +SERVICEBUS_SAMPLE_CONTENT = b"_\241S\374f\335OI\202]\356\033|4<\373\000Sp\300\013\005@@pH\031\010\000@R\001\000Sq\301$\002\243\020x-opt-lock-token\230\374S\241_\335fIO\202]\356\033|4<\373\000Sr\301U\006\243\023x-opt-enqueued-time\203\000\000\001\216v\307\333\310\243\025x-opt-sequence-numberU\014\243\022x-opt-locked-until\203\000\000\001\216v\310\3067\000Ss\300?\r\241 f00d2a33551440389d68e299d31adc7c@@@@@@@\203\000\000\001\216\276\340\343\310\203\000\000\001\216v\307\333\310@@@\000Su\240\005hello" # noqa: E501 + + +# Mock classes for testing +class MockMBD: + def __init__(self, version: str, source: str, content_type: str, content: str): + self.version = version + self.source = source + self.content_type = content_type + self.content = content + + +class MockCMBD: + def __init__(self, model_binding_data_list: List[MockMBD]): + self.model_binding_data = model_binding_data_list + + @property + def data_type(self) -> Optional[int]: + return self._data_type.value if self._data_type else None + + @property + def direction(self) -> int: + return self._direction.value + + +class TestServiceBus(unittest.TestCase): + def test_input_type(self): + check_input_type = ServiceBusConverter.check_input_type_annotation + self.assertTrue(check_input_type(ServiceBusReceivedMessage)) + self.assertFalse(check_input_type(str)) + self.assertFalse(check_input_type("hello")) + self.assertFalse(check_input_type(bytes)) + self.assertFalse(check_input_type(bytearray)) + self.assertTrue(check_input_type(List[ServiceBusReceivedMessage])) + self.assertTrue(check_input_type(list[ServiceBusReceivedMessage])) + self.assertTrue(check_input_type(tuple[ServiceBusReceivedMessage])) + self.assertTrue(check_input_type(set[ServiceBusReceivedMessage])) + self.assertFalse(check_input_type(dict[str, ServiceBusReceivedMessage])) + + def test_input_none(self): + result = ServiceBusConverter.decode( + data=None, trigger_metadata=None, pytype=ServiceBusReceivedMessage + ) + self.assertIsNone(result) + + datum: Datum = Datum(value=b"string_content", type=None) + result = ServiceBusConverter.decode( + data=datum, trigger_metadata=None, pytype=ServiceBusReceivedMessage + ) + self.assertIsNone(result) + + def test_input_incorrect_type(self): + datum: Datum = Datum(value=b"string_content", type="bytearray") + with self.assertRaises(ValueError): + ServiceBusConverter.decode( + data=datum, trigger_metadata=None, pytype=ServiceBusReceivedMessage + ) + + def test_input_empty_mbd(self): + datum: Datum = Datum(value={}, type="model_binding_data") + result: ServiceBusReceivedMessage = ServiceBusConverter.decode( + data=datum, trigger_metadata=None, pytype=ServiceBusReceivedMessage + ) + self.assertIsNone(result) + + def test_input_empty_cmbd(self): + datum: Datum = Datum(value=MockCMBD([None]), + type="collection_model_binding_data") + result: ServiceBusReceivedMessage = ServiceBusConverter.decode( + data=datum, trigger_metadata=None, pytype=ServiceBusReceivedMessage + ) + self.assertEqual(result, [None]) + + def test_input_populated_mbd(self): + sample_mbd = MockMBD( + version="1.0", + source="AzureServiceBusReceivedMessage", + content_type="application/octet-stream", + content=SERVICEBUS_SAMPLE_CONTENT + ) + + datum: Datum = Datum(value=sample_mbd, type="model_binding_data") + result: ServiceBusReceivedMessage = ServiceBusConverter.decode( + data=datum, trigger_metadata=None, pytype=ServiceBusReceivedMessage + ) + + self.assertIsNotNone(result) + self.assertIsInstance(result, ServiceBusSDK) + + sdk_result = ServiceBusReceivedMessage(data=datum.value).get_sdk_type() + + self.assertIsNotNone(sdk_result) + self.assertIsInstance(sdk_result, ServiceBusSDK) + + def test_input_populated_cmbd(self): + sample_mbd = MockMBD( + version="1.0", + source="AzureServiceBusReceivedMessage", + content_type="application/octet-stream", + content=SERVICEBUS_SAMPLE_CONTENT + ) + + datum: Datum = Datum(value=MockCMBD([sample_mbd, sample_mbd]), + type="collection_model_binding_data") + result: ServiceBusReceivedMessage = ServiceBusConverter.decode( + data=datum, trigger_metadata=None, pytype=ServiceBusReceivedMessage + ) + + self.assertIsNotNone(result) + for event_data in result: + self.assertIsInstance(event_data, ServiceBusSDK) + + sdk_results = [] + for mbd in datum.value.model_binding_data: + sdk_results.append(ServiceBusReceivedMessage(data=mbd).get_sdk_type()) + + self.assertNotEqual(sdk_results, [None, None]) + for event_data in sdk_results: + self.assertIsInstance(event_data, ServiceBusSDK) + + def test_input_invalid_datum_type(self): + with self.assertRaises(ValueError) as e: + datum: Datum = Datum(value="hello", type="str") + _: ServiceBusReceivedMessage = ServiceBusConverter.decode( + data=datum, trigger_metadata=None, pytype="" + ) + self.assertEqual( + e.exception.args[0], + "Unexpected type of data received for the 'servicebus' binding: 'str'", + ) + + def test_input_get_decoded_message_ex(self): + with self.assertRaises(ValueError) as e: + _ = get_decoded_message("Invalid message") + self.assertEqual( + e.exception.args[0], + "Failed to decode ServiceBus content: must be str, not bytes", + ) diff --git a/eng/ci/ci-servicebus-tests.yml b/eng/ci/ci-servicebus-tests.yml new file mode 100644 index 0000000..b0e7c7c --- /dev/null +++ b/eng/ci/ci-servicebus-tests.yml @@ -0,0 +1,35 @@ +trigger: none # ensure this is not ran as a CI build + +pr: + branches: + include: + - dev + - release/* + +resources: + repositories: + - repository: 1es + type: git + name: 1ESPipelineTemplates/1ESPipelineTemplates + ref: refs/tags/release + - repository: eng + type: git + name: engineering + ref: refs/tags/release + +variables: + - template: /ci/variables/build.yml@eng + - template: /ci/variables/cfs.yml@eng + +extends: + template: v1/1ES.Unofficial.PipelineTemplate.yml@1es + parameters: + pool: + name: 1es-pool-azfunc + image: 1es-windows-2022 + os: windows + + stages: + - stage: RunServiceBusUnitTests + jobs: + - template: /eng/templates/official/jobs/servicebus-unit-tests.yml@self diff --git a/eng/ci/official-build.yml b/eng/ci/official-build.yml index 28f6ec8..a341b53 100644 --- a/eng/ci/official-build.yml +++ b/eng/ci/official-build.yml @@ -51,6 +51,14 @@ extends: dependsOn: Build jobs: - template: /eng/templates/official/jobs/blob-unit-tests.yml@self + - stage: RunEventHubTests + dependsOn: Build + jobs: + - template: /eng/templates/official/jobs/eventhub-unit-tests.yml@self + - stage: RunServiceBusTests + dependsOn: Build + jobs: + - template: /eng/templates/official/jobs/servicebus-unit-tests.yml@self - stage: RunFastApiTests dependsOn: Build jobs: diff --git a/eng/ci/public-build.yml b/eng/ci/public-build.yml index 61ed7a9..5368b2f 100644 --- a/eng/ci/public-build.yml +++ b/eng/ci/public-build.yml @@ -43,3 +43,23 @@ extends: - stage: Build jobs: - template: /eng/templates/jobs/build.yml@self + - stage: RunBaseTests + dependsOn: Build + jobs: + - template: /eng/templates/official/jobs/base-unit-tests.yml@self + - stage: RunBlobTests + dependsOn: Build + jobs: + - template: /eng/templates/official/jobs/blob-unit-tests.yml@self + - stage: RunEventHubTests + dependsOn: Build + jobs: + - template: /eng/templates/official/jobs/eventhub-unit-tests.yml@self + - stage: RunServiceBusTests + dependsOn: Build + jobs: + - template: /eng/templates/official/jobs/servicebus-unit-tests.yml@self + - stage: RunFastApiTests + dependsOn: Build + jobs: + - template: /eng/templates/official/jobs/fastapi-unit-tests.yml@self diff --git a/eng/templates/jobs/build.yml b/eng/templates/jobs/build.yml index 3c3c7c4..ad70054 100644 --- a/eng/templates/jobs/build.yml +++ b/eng/templates/jobs/build.yml @@ -13,6 +13,9 @@ jobs: eventhub_extension: EXTENSION_DIRECTORY: 'azurefunctions-extensions-bindings-eventhub' EXTENSION_NAME: 'EventHub' + servicebus_extension: + EXTENSION_DIRECTORY: 'azurefunctions-extensions-bindings-servicebus' + EXTENSION_NAME: 'ServiceBus' fastapi_extension: EXTENSION_DIRECTORY: 'azurefunctions-extensions-http-fastapi' EXTENSION_NAME: 'Http' diff --git a/eng/templates/official/jobs/build-artifacts.yml b/eng/templates/official/jobs/build-artifacts.yml index f41a08e..307cac8 100644 --- a/eng/templates/official/jobs/build-artifacts.yml +++ b/eng/templates/official/jobs/build-artifacts.yml @@ -13,6 +13,9 @@ jobs: eventhub_extension: EXTENSION_DIRECTORY: 'azurefunctions-extensions-bindings-eventhub' EXTENSION_NAME: 'EventHub' + servicebus_extension: + EXTENSION_DIRECTORY: 'azurefunctions-extensions-bindings-servicebus' + EXTENSION_NAME: 'ServiceBus' fastapi_extension: EXTENSION_DIRECTORY: 'azurefunctions-extensions-http-fastapi' EXTENSION_NAME: 'FastAPI' diff --git a/eng/templates/official/jobs/servicebus-unit-tests.yml b/eng/templates/official/jobs/servicebus-unit-tests.yml new file mode 100644 index 0000000..770e851 --- /dev/null +++ b/eng/templates/official/jobs/servicebus-unit-tests.yml @@ -0,0 +1,29 @@ +jobs: + - job: "TestPython" + displayName: "Run ServiceBus Tests" + + strategy: + matrix: + python39: + PYTHON_VERSION: '3.9' + python310: + PYTHON_VERSION: '3.10' + python311: + PYTHON_VERSION: '3.11' + python312: + PYTHON_VERSION: '3.12' + + steps: + - task: UsePythonVersion@0 + inputs: + versionSpec: $(PYTHON_VERSION) + - bash: | + python -m pip install --upgrade pip + cd azurefunctions-extensions-bindings-servicebus + python -m pip install -U -e .[dev] + displayName: 'Install dependencies' + - bash: | + python -m pytest -q --instafail azurefunctions-extensions-bindings-servicebus/tests/ + env: + AzureWebJobsStorage: $(AzureWebJobsStorage) + displayName: "Running ServiceBus $(PYTHON_VERSION) Python Extension Tests"