Skip to content

feat: finalize param inputs + multiple fixes #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 4, 2023

Conversation

Salakar
Copy link
Member

@Salakar Salakar commented May 4, 2023

Fixes #74
Fixes #75
Fixes #76
Fixes #81

Testing

Deployed the following sample:

from firebase_functions import storage_fn, params
from firebase_admin import initialize_app

initialize_app()

bucket = params.StringParam(
    "BUCKET",
    label="storage bucket",
    description="The bucket to resize images from.",
    input=params.ResourceInput(type=params.ResourceType.STORAGE_BUCKET),
    default=params.STORAGE_BUCKET,
)

output_path = params.StringParam(
    "OUTPUT_PATH",
    label="storage bucket output path",
    description=
    "The path of in the bucket where processed images will be stored.",
    input=params.TextInput(
        example="/images/processed",
        validation_regex=r"^\/.*$",
        validation_error_message=
        "Must be a valid path starting with a forward slash",
    ),
    default="/images/processed",
)

image_type = params.ListParam(
    "IMAGE_TYPE",
    label="convert image to preferred types",
    description="The image types you'd like your source image to convert to.",
    input=params.MultiSelectInput([
        params.SelectOption(value="jpeg", label="jpeg"),
        params.SelectOption(value="png", label="png"),
        params.SelectOption(value="webp", label="webp"),
    ]),
    default=["jpeg", "png"],
)

delete_original = params.BoolParam(
    "DELETE_ORIGINAL_FILE",
    label="delete the original file",
    description=
    "Do you want to automatically delete the original file from the Cloud Storage?",
    input=params.SelectInput([
        params.SelectOption(value=True, label="Delete on any resize attempt"),
        params.SelectOption(value=False, label="Don't delete"),
    ],),
    default=True,
)

image_resize_api_secret = params.SecretParam(
    "IMAGE_RESIZE_API_SECRET",
    label="image resize api secret",
    description="The fake secret key to use for the image resize API.",
)


@storage_fn.on_object_finalized(
    bucket=bucket,
    secrets=[image_resize_api_secret],
)
def resize_images(event: storage_fn.CloudEvent[storage_fn.StorageObjectData]):
    """
    This function will be triggered when a new object is created in the bucket.
    """
    print("Got a new image:", event)
    print("Selected image types:", image_type.value)
    print("Selected bucket resource:", bucket.value)
    print("Selected output location:", output_path.value)
    print("Testing a not so secret api key:", image_resize_api_secret.value)
    print("Should original images be deleted?:", delete_original.value)
    # TODO: Implement your image resize logic

Deployment successful:

? Would you like to proceed with deletion? Selecting no will continue the rest of the deployments. No
i  functions: creating Python 3.10 function resize_images(us-central1)...
✔  functions[resize_images(us-central1)] Successful create operation.
i  functions: cleaning up build files...

✔  Deploy complete!

Project Console: https://console.firebase.google.com/project/python-functions-testing/overview

image

Logs when a file is uploaded to storage;

2023-05-04 03:04:23.447 BST
Got a new image: CloudEvent(specversion='1.0', id='7594586441232904', source='//storage.googleapis.com/projects/_/buckets/python-functions-testing.appspot.com', type='google.cloud.storage.object.v1.finalized', time=datetime.datetime(2023, 5, 4, 2, 4, 22, 153261, tzinfo=datetime.timezone.utc), data=StorageObjectData(bucket='python-functions-testing.appspot.com', cache_control=None, component_count=None, content_disposition="inline; filename*=utf-8''IMG_5468.JPG", content_encoding=None, content_language=None, content_type='image/jpeg', crc32c='f9KoMQ==', customer_encryption=None, etag='CJeGubrJ2v4CEAE=', generation='1683165862052631', id='python-functions-testing.appspot.com/testing/IMG_5468.JPG/1683165862052631', kind='storage#object', md5_hash='MySB6kxWobchiW35zlXouA==', media_link='https://storage.googleapis.com/download/storage/v1/b/python-functions-testing.appspot.com/o/testing%2FIMG_5468.JPG?generation=1683165862052631&alt=media', metadata={'firebaseStorageDownloadTokens': '0a6b227c-cc28-4aff-be10-e47a735bebfe'}, metageneration='1', name='testing/IMG_5468.JPG', self_link='https://www.googleapis.com/storage/v1/b/python-functions-testing.appspot.com/o/testing%2FIMG_5468.JPG', size='298482', storage_class='STANDARD', time_created='2023-05-04T02:04:22.153Z', time_deleted=None, time_storage_class_updated='2023-05-04T02:04:22.153Z', updated='2023-05-04T02:04:22.153Z'), subject='objects/testing/IMG_5468.JPG')
2023-05-04 03:04:23.447 BST
Selected image types: ['jpeg', 'png']
2023-05-04 03:04:23.447 BST
Selected bucket resource: python-functions-testing.appspot.com
2023-05-04 03:04:23.447 BST
Selected output location: /a/good/path
2023-05-04 03:04:23.447 BST
Testing a not so secret api key: ffffff
2023-05-04 03:04:23.447 BST
Should original images be deleted?: True

A string param with ResourceInput to select a storage bucket, the default is set to the builtin default param params.STORAGE_BUCKET and you can see the input prompts with this value pre selected:
Pasted Graphic 7


A string param with text input to select a path:


Pasted Graphic 8

and tests regex validation:

Pasted Graphic 11


A list param with multi select + default selection correctly showing:

Pasted Graphic 13


A bool param with select input (also shows correct default value):

Pasted Graphic 14


A secret param:

image

(shows #76 is now fixed and secrets are being created correctly now)


The CLI generates a .env file:

BUCKET=python-functions-testing.appspot.com
OUTPUT_PATH=/a/good/path
IMAGE_TYPE=jpeg,png
DELETE_ORIGINAL_FILE=true

The generated manifest yaml:

endpoints:
  resize_images:
    availableMemoryMb: null
    concurrency: null
    entryPoint: resize_images
    eventTrigger:
      eventFilters:
        bucket: '{{ params.BUCKET }}'
      eventType: google.cloud.storage.object.v1.finalized
      retry: false
    ingressSettings: null
    labels: {}
    maxInstances: null
    minInstances: null
    platform: gcfv2
    secretEnvironmentVariables:
    - key: IMAGE_RESIZE_API_SECRET
    serviceAccountEmail: null
    timeoutSeconds: null
params:
- default: '{{ params.STORAGE_BUCKET }}'
  description: The bucket to resize images from.
  input:
    resource:
      type: storage.googleapis.com/Bucket
  label: storage bucket
  name: BUCKET
  type: string
- default: /images/processed
  description: The path of in the bucket where processed images will be stored.
  input:
    text:
      example: /images/processed
      validationErrorMessage: Must be a valid path starting with a forward slash
      validationRegex: ^\/.*$
  label: storage bucket output path
  name: OUTPUT_PATH
  type: string
- default:
  - jpeg
  - png
  description: The image types you'd like your source image to convert to.
  input:
    multiSelect:
      options:
      - label: jpeg
        value: jpeg
      - label: png
        value: png
      - label: webp
        value: webp
  label: convert image to preferred types
  name: IMAGE_TYPE
  type: list
- default: true
  description: Do you want to automatically delete the original file from the Cloud
    Storage?
  input:
    select:
      options:
      - label: Delete on any resize attempt
        value: true
      - label: Don't delete
        value: false
  label: delete the original file
  name: DELETE_ORIGINAL_FILE
  type: boolean
- description: The fake secret key to use for the image resize API.
  label: image resize api secret
  name: IMAGE_RESIZE_API_SECRET
  type: secret
requiredAPIs: []
specVersion: v1alpha1

@Salakar Salakar force-pushed the param-inputs branch 4 times, most recently from d8e6a5d to fbdaec7 Compare May 4, 2023 02:01
@Salakar Salakar marked this pull request as ready for review May 4, 2023 02:22
return []
# Otherwise, split the string by commas.
# (This is for emulator & the Firebase CLI generated .env file, the environment
# variable is a comma-separated list.)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@taeold FYI I had to handle both scenarios here; local .env is generated as comma delimited string:

image

cloud run env is json array string:

image

Copy link
Collaborator

@taeold taeold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Let's get rid of ListParam - we don't want people to use it :'(

if self.default is not None:
return self.default
return self.default.value if isinstance(
self.default, Expression) else self.default
return False


@_dataclasses.dataclass(frozen=True)
class ListParam(Param[list]):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry - after all this work, I think the team is considering yanking ListParam support all together 🤦🏼 .

It isn't documented in https://firebase.google.com/docs/functions/config-env.

It is documented in JS SDK reference doc, and the team feels that it was a mistake to release it prematurely :'(

@taeold taeold merged commit 11f799f into firebase:main May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants