Skip to content

Commit c3139aa

Browse files
committed
update README
1 parent 31b29b9 commit c3139aa

File tree

3 files changed

+17
-5
lines changed

3 files changed

+17
-5
lines changed

airbyte_cdk/sources/declarative/declarative_component_schema.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1593,7 +1593,7 @@ definitions:
15931593
- "$ref": "#/definitions/HttpRequester"
15941594
- "$ref": "#/definitions/CustomRequester"
15951595
download_target_extractor:
1596-
description: Responsible for fetching the url where the file is located. This is applied on each records and not on the HTTP response
1596+
description: Responsible for fetching the final result `urls` provided by the completed / finished / ready async job.
15971597
anyOf:
15981598
- "$ref": "#/definitions/DpathExtractor"
15991599
- "$ref": "#/definitions/CustomRecordExtractor"

airbyte_cdk/sources/declarative/models/declarative_component_schema.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
# Copyright (c) 2025 Airbyte, Inc., all rights reserved.
2+
13
# generated by datamodel-codegen:
24
# filename: declarative_component_schema.yaml
35

airbyte_cdk/sources/declarative/requesters/README.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,19 @@
1+
# Download Target and Download Requester
2+
3+
- The `creation_response` and `polling_response` interpolation contexts are always available during the job download step of the process.
4+
5+
- The`download_target` interpolation context is generated by the `download_target_extractor` and made available to the job download step as well.
6+
- if `download_target_requester` is not provided, `download_target_extractor` will get urls from the `polling_response`
7+
- if `download_target_requester` is provided, an additional request will be made to fetch job download targets and `download_target_extractor` will operate on that response
8+
9+
## Some important considerations
10+
11+
- **Note:** If the `download_target_extractor` and `download_target_requester` are not defined, a single job download request will be made without the `download_target` context.
12+
- **Note:** The `download_target_extractor` is required (not optional) if using a `download_target_requester`
13+
114
# AsyncHttpJobRepository sequence diagram
215

316
- Components marked as optional are not required and can be ignored.
4-
- # TODO update
5-
- if `download_target_requester` is not provided, `download_target_extractor` will get urls from the `polling_response`
6-
- interpolation_context, e.g. `creation_response` or `polling_response` can be obtained from stream_slice
717

818
```mermaid
919
---
@@ -38,7 +48,7 @@ sequenceDiagram
3848
UrlRequester -->> AsyncHttpJobRepository: Download URLs
3949
4050
AsyncHttpJobRepository ->> DownloadRetriever: Download reports
41-
DownloadRetriever ->> Reporting Server: Retrieve report data (interpolation_context: `url`, `creation_response`, `polling_response`)
51+
DownloadRetriever ->> Reporting Server: Retrieve report data (interpolation_context: `download_target`, `creation_response`, `polling_response`)
4252
Reporting Server -->> DownloadRetriever: Report data
4353
DownloadRetriever -->> AsyncHttpJobRepository: Report data
4454
else Status: Failed

0 commit comments

Comments
 (0)