Redo image building related pages

content/user-guide/core-concepts/building-images/cloud-image-builder.md

@@ -0,0 +1,16 @@
+---
+title: Cloud image builder
+weight: 2
+variants: -flyte +serverless +byoc +selfmanaged
+---
+
+## Cloud image builder
+
+When `builder="union"` is specified in an `ImageSpec`, {{< key product_name >}} will build the image using its `ImageBuilder` service in the cloud and registered, either in the in {{< key product_name >}}'s own internal container registry (the default) or if a
+If no `registry` parameter is supplied, the resulting image will be registered
+
+If a `registry` parameter is supplied, then
+
+From there it will be pulled and installed in the task container when it spins up.
+All this is done transparently and does not require any set up by the user.
+

content/user-guide/core-concepts/building-images/image-spec-development-cycle.md

@@ -0,0 +1,113 @@
+---
+title: ImageSpec
+weight: 1
+variants: +flyte +serverless +byoc +selfmanaged
+---
+
+# ImageSpec
+
+When deploying a task, {{< key product_name >}} needs to know how to build the container image that will be used to run that task.
+You specify this information in an `ImageSpec` object that is passed to the `@{{< key kit_as >}}.task` decorator.
+
+Here is a simple example:
+
+{{< variant flyte >}}
+{{< markdown >}}
+
+```python
+import {{< key kit_import >}}
+
+image_spec = union.ImageSpec(
+    builder="default",
+    registry=registry=os.environ.get("IMAGE_REGISTRY", None),
+    name="say-hello-image",
+    requirements="uv.lock",
+)
+
+@{{< key kit_as >}}.task(container_image=image_spec)
+def say_hello(name: str) -> str:
+    return f"Hello, {name}!"
+
+@{{< key kit_as >}}.workflow
+def hello_world_wf(name: str = "world") -> str:
+    greeting = say_hello(name=name)
+    return greeting
+```
+
+{{< /markdown >}}
+{{< /variant >}}
+{{< variant serverless byoc selfmanaged >}}
+{{< markdown >}}
+
+```python
+import {{< key kit_import >}}
+
+image_spec = union.ImageSpec(
+    builder="union",
+    name="say-hello-image",
+    requirements="uv.lock",
+)
+
+@{{< key kit_as >}}.task(container_image=image_spec)
+def say_hello(name: str) -> str:
+    return f"Hello, {name}!"
+
+@{{< key kit_as >}}.workflow
+def hello_world_wf(name: str = "world") -> str:
+    greeting = say_hello(name=name)
+    return greeting
+```
+
+{{< /markdown >}}
+{{< /variant >}}
+
+Here, the `ImageSpec` class is used to specify the container image to be used for the `say_hello` task.
+ImageSpec supports [a large set of configuration parameters](../../api-reference/flytekit-sdk/packages/flytekit.image_spec.image_spec.md), here we show a few basic ones:
+
+{{< variant flyte >}}
+{{< markdown >}}
+
+* `builder`: Specifies how the image is built.
+  * In this case we specify `default`.
+    This will [build the image on your local machine](./local-image-builder).
+
+  > [!NOTE]
+  > In Flyte OSS, omitting the `builder` parameter is equivalent to specifying `builder="default"`.
+
+* `registry` Specifies the container registry to which the image will be pushed after it is built.
+  * In this case we use an environment variable to hold the URL of the registry.
+  * This is the registry to which the image will be pushed once it is built, and from which it will be pulled when the task using that image is instantiated.
+  * See [Local image builder > Container registry](./local-image-builder#container-registry).
+
+{{< /markdown >}}
+{{< /variant >}}
+{{< variant serverless byoc selfmanaged >}}
+{{< markdown >}}
+
+* `builder`: Specifies how the image is built.
+  * In this case we specify `union`.
+    This will build the image using {{< key product_name >}}'s built-in [Cloud image builder](./cloud-image-builder).
+  * Alternatively, you can specify `default`.
+    This will [build the image on your local machine](./local-image-builder).
+    This will also require adding a [`registry` parameter](./local-image-builder#container-registry).
+
+  > [!NOTE]
+  > In {{< key product_name >}} Serverless, omitting the `builder` parameter is equivalent to specifying `builder="union"`.
+  > In {{< key product_name >}} BYOC and Self-managed, omitting the `builder` parameter is equivalent to specifying `builder="default"`
+  > (this differentiation exists for purposes of backward compatibility).
+
+{{< /markdown >}}
+{{< /variant >}}
+
+* The `name` parameter specifies the name of the image. This name will be used to identify the image in the container registry.
+
+* The `requirements` parameter specifies the path to a file (relative to the directory in which the
+  `{{< key cli >}} run` or `{{< key cli >}} register` command is invoked) that specifies the dependencies to be installed in the image.
+  The file may be:
+  * A `requirements.txt` file.
+  * A `uv.lock` file generated by the `uv sync` command.
+  * A `poetry.lock` file generated by the `poetry install` command.
+  * A `pyproject.toml` file.
+
+When you execute the `{{< key cli >}} run` or `{{< key cli >}} register` command, {{< key product_name >}} will build the container image defined in `ImageSpec` block
+(as well as registering the tasks and workflows defined in your code).

content/user-guide/core-concepts/tasks/task-software-environment/image-spec.md → content/user-guide/core-concepts/building-images/image-spec-task-environment.md

@@ -4,7 +4,7 @@ weight: 1
 variants: +flyte -serverless +byoc +selfmanaged
 ---
 
-# Local image builduing
+# Local image building
 
 With {{< key product_name >}}, every task in a workflow runs within its own dedicated container.
 Since a container requires a container image to run, every task in {{< key product_name >}} must have a container image associated with it.

content/user-guide/core-concepts/building-images/local-image-builder.md

@@ -0,0 +1,53 @@
+---
+title: Local image builder
+weight: 3
+variants: +flyte +serverless +byoc +selfmanaged
+---
+
+# Local image builder
+
+When `builder="default"` is specified, {{< key product_name >}} will build the image locally on your machine with `docker` and push it to the registry you specify in the `registry` parameter.
+From there it will be pulled and installed in the task container when it spins up.
+
+### Local container engine
+
+To enable local image building you must have an [OCI-compatible](https://opencontainers.org/) container engine, like [Docker](https://docs.docker.com/get-docker/), installed and running locally.
+Other options include [Podman](https://podman.io/), [LXD](https://linuxcontainers.org/lxd/introduction/), or [Containerd](https://containerd.io/).
+
+### Accessibility of the container registry and image
+
+When building images locally you will need to specify the URL of the registry in the `registry` parameter of the `ImageSpec`.
+
+The registry you specify must be accessible both to your local machine and to your {{< key product_name >}} cluster:
+
+* Your local machine must be able to *push* the image it builds to the registry.
+  * This usually requires authenticating your local docker to the registry by doing a `docker login`. See the example below.
+* Your {{< key product_name >}} cluster must be able to *pull* the image from the registry when it spins up a task that uses that image.
+  * The most common option here is use a public registry and make the image publicly available. See the example below.
+
+A common choice is to use the GitHub Container Registry (GHCR) that comes as part of your GitHub account.
+For more information, see [Working with the Container registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry).
+
+You will still need to set up your local Docker client to authenticate to the registry in order for `{{< key cli >}}` to be able to push the built image.
+
+Follow the directions in [Working with the Container registry > Authenticating to the Container registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-to-the-container-registry).
+
+You may use another container registry if you prefer,
+such as [Docker Hub](https://hub.docker.com/),
+[Amazon Elastic Container Registry (ECR)](../integrations/enabling-aws-resources/enabling-aws-ecr),
+or [Google Artifact Registry (GAR)](../integrations/enabling-gcp-resources/enabling-google-artifact-registry).
+
+
+### Make your image accessible to {{< key product_name >}}
+
+In addition to making sure your registry is accessible from your local machine, you will need to ensure that the specific image, once pushed to the registry, is itself publicly accessible.
+
+> [!NOTE] Make your image public
+> Note that in the case of our example registry (GHCR), making the image public can only be done once the image _has been_ pushed.
+> This means that you will need to register your workflow first, then make the image public and then run the workflow from the {{< key product_name >}} UI.
+> If you try to run the workflow before making the image public (for example by doing a `{{< key cli >}} run` which both registers and runs immediately)
+> the workflow execution will fail with an `ImagePullBackOff `error.
+
+In the GitHub Container Registry, switch the visibility of your container image to Public. For more information, see [Configuring a package's access control and visibility](https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility#about-inheritance-of-access-permissions-and-visibility).
+
+At this point, you can run the workflow from the {{< key product_name >}} interface.

content/user-guide/core-concepts/building-images/private-image-registries.md

@@ -0,0 +1,76 @@
+---
+title: Private image registries
+weight: 3
+variants: +flyte +serverless +byoc +selfmanaged
+---
+
+# Private image registries
+
+The images used by your {{< key product_name >}} tasks must be stored in a container registry from which {{< key product_name >}} will pull it when it spins up the container for a task execution.
+The key constraint here is that the registry, and the image itself, must be read-accessible to your {{< key product_name >}} cluster, in order for the image pull to succeed.
+
+{{< variant serverless byoc selfmanaged >}}
+{{< markdown >}}
+
+So far, we have discussed two options:
+
+* Using the built-in {{< key product_name >}} [Cloud image builder]() (by specifying the `ImageSpec` parameter `builder="union"`) which pushes the image to the {{< key product_name >}} internal container registry.
+  * Since the registry is built into {{< key product_name >}}, it is automatically accessible to your {{< key product_name >}} cluster.
+* Building your image on your local machine (by specifying the `ImageSpec` parameter `builder="default"`) and pushing the image to  a public registry (like the GitHub Packages registry) (by specifying the `ImageSpec` parameter `registry`).
+  * In this case the image is accessible to your {{< key product_name >}} cluster because it is public.
+
+If you want to keep your images private, clearly the public registry option will not work.
+The internal {{< key product_name >}} registry is private, so that is an option is you are concerned only with keeping your images non-public.
+
+However, there may be cases where you want to keep your images both *private*, and *in you own registry*.
+In that case, you will need to set up your {{< key product_name >}} cluster with the correct credentials to be able to pull the images from that private registry.
+
+{{< /markdown >}}
+{{< /variant >}}
+
+{{< variant flyte >}}
+{{< markdown >}}
+
+We have discussed building your image on your local machine (by specifying the `ImageSpec` parameter `builder="default"`) and pushing the image to a public registry (like the GitHub Packages registry) (by specifying the `ImageSpec` parameter `registry`). In this case the image is accessible to your {{< key product_name >}} cluster because it is public.
+
+However, if you want to **keep your images private**, clearly the public registry option will not work.
+You will need to set up your {{< key product_name >}} cluster with the correct credentials to be able to pull the images from a private registry.
+
+
+You can use different private container registries to host your images, such as AWS ECR, Docker Hub, GitLab Container Registry, and GitHub Container Registry.
+
+To pull private images, ensure that you have the command line tools and login information associated with the registry.
+
+## Create a secret
+
+First create a secret that contains all the credentials needed to log into the registry.
+
+## Configure `imagePullSecrets`
+
+Then, you’ll need to specify an `imagePullSecrets` configuration to pull a private image using one of two methods below.
+
+### Service Account
+
+You can use the default or new service account for this option:
+
+1. Add your `imagePullSecrets` configuration to the service account.
+
+2. Use this service account to log into the private registry and pull the image.
+
+3. When you create a task/workflow execution this service account should be specified.
+
+### Custom Pod Template
+
+This option uses a custom pod template to create a pod.
+This template is automatically added to every pod that Flyte creates.
+
+1. Add your `imagePullSecrets` configuration to the custom pod template.
+
+2. Update FlytePropeller about the pod created in the previous step.
+
+3. FlytePropeller adds `imagePullSecrets`, along with other customization for the pod, to the `PodSpec`, which should look similar to this manifest.
+
+The pods with their keys can log in and access the images in the private registry. Once you set up the token to authenticate with the private registry, you can pull images from them.
+
+{{< /markdown >}}
+{{< /variant >}}

content/user-guide/development-cycle/accessing-aws-s3.md

@@ -1,6 +1,6 @@
 ---
 title: Accessing AWS S3 buckets
-weight: 14
+weight: 16
 variants: -flyte +serverless +byoc +selfmanaged
 ---
 

content/user-guide/development-cycle/ci-cd-deployment.md

@@ -1,6 +1,6 @@
 ---
 title: CI/CD deployment
-weight: 17
+weight: 19
 variants: -flyte -serverless +byoc +selfmanaged
 ---
 

content/user-guide/development-cycle/debugging-with-interactive-tasks.md

@@ -1,6 +1,6 @@
 ---
 title: Debugging with interactive tasks
-weight: 11
+weight: 13
 variants: +flyte +serverless +byoc +selfmanaged
 ---
 

content/user-guide/development-cycle/decks.md

@@ -1,6 +1,6 @@
 ---
 title: Decks
-weight: 19
+weight: 22
 variants: +flyte +serverless +byoc +selfmanaged
 ---
 

content/user-guide/development-cycle/details-of-pyflyte-run.md

@@ -1,6 +1,6 @@
 ---
 title: Details of pyflyte run
-weight: 10
+weight: 12
 variants: +flyte -serverless -byoc -selfmanaged
 ---
 

content/user-guide/development-cycle/details-of-union-run.md

@@ -1,6 +1,6 @@
 ---
 title: Details of union run
-weight: 10
+weight: 12
 variants: -flyte +serverless +byoc +selfmanaged
 ---