fix: improve stream template info in declarative_component_schema #717
Conversation
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Testing This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@lmossman/improve-stream-template-info#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch lmossman/improve-stream-template-infoHelpful ResourcesPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
There was a problem hiding this comment.
Pull Request Overview
This PR improves the documentation and examples for stream template information in the declarative component schema. It updates examples to be more relevant and adds documentation for the components_values variable.
- Updated examples in the schema to show more realistic stream template usage patterns
- Added documentation for the
components_valuesinterpolation variable with description and examples
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
airbyte_cdk/sources/declarative/declarative_component_schema.yaml
Outdated
Show resolved
Hide resolved
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 WalkthroughWalkthroughUpdated schema examples and interpolation metadata in declarative_component_schema.yaml: field_path examples for DpathExtractor and ComponentMappingDefinition now use top-level/name and retriever/requester paths; added a new interpolation variable Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant U as User Config
participant CR as Components Resolver
participant INT as Interpolation Engine
participant SG as Stream Generator
U->>CR: Define components (e.g., name, id)
CR-->>INT: Provide components_values object
U->>INT: Templates with field_path (e.g., ["retriever","requester","url"], ["name"])
INT-->>SG: Resolved values using components_values and context
SG-->>U: Generated stream(s)
note over INT,SG: New variable: components_values available during interpolation
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Would you like me to propose alternate example paths that include nested arrays or HTTP-header-related examples for completeness, wdyt? Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (2)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
airbyte_cdk/sources/declarative/declarative_component_schema.yaml (1)
1608-1609: Schema keyword typo breaks JSON Schema validation (additional_properties → additionalProperties).This field uses additional_properties instead of additionalProperties. JSON Schema won’t recognize it, so this object won’t accept arbitrary params as intended.
Patch?
- $parameters: - type: object - additional_properties: true + $parameters: + type: object + additionalProperties: true
🧹 Nitpick comments (5)
airbyte_cdk/sources/declarative/declarative_component_schema.yaml (5)
4145-4148: Updated field_path examples improve clarity; consider adding a defensive example using default values.Nice switch to name/retriever context and showcasing components_values in paths. Would you add one more example that demonstrates a safe default to avoid KeyError during interpolation, e.g. ["retriever", "requester", "{{ components_values.get('field', 'path') }}"] to guide users toward resilient templates, wdyt?
1868-1872: AI summary inconsistent with DpathExtractor examples; do we also want to update these to match the new guidance?The PR summary mentions replacing data-rooted examples with name/retriever/requester paths for DpathExtractor. This block still shows the old ["data", ...] examples. If the intent is to align extractor examples with the component templating guidance, shall we update to something like:
- ["name"]
- ["retriever", "requester", "url"]
- ["retriever", "requester", "{{ parameters.name }}"]
- ["*", "**", "name"]
wdyt?
4683-4684: Minor grammar: duplicate word in config variable description.There's a small typo: “the the”. Want to fix it?
- description: The connector configuration. The object's keys are the same as the the keys defined in the connection specification. + description: The connector configuration. The object's keys are the same as the keys defined in the connection specification.
4719-4721: Type of last_page_size should be an integer, not object.The description says “Number of records…”, but type is object. Should we correct it to integer?
- type: object + type: integer
2899-2904: Typos in OAuth interpolation context names (Decorer → Decoder) could confuse readers.Within the examples, base64Decorer/urlDecorer appear; elsewhere you use Decoder/Decoder. Want to normalize?
- + base64Decorer - decode from `base64` encoded string, {{ {{my_string_variable_or_string_value}} | base64Decoder }} + + base64Decoder - decode from `base64` encoded string, {{ {{my_string_variable_or_string_value}} | base64Decoder }} - + urlDecorer - decode the input url-encoded string into text format, {{ urlDecoder:https%3A%2F%2Fairbyte.io | urlDecoder}} + + urlDecoder - decode the input url-encoded string into text format, {{ urlDecoder:https%3A%2F%2Fairbyte.io | urlDecoder }}Also, minor spacing in the second example after the closing braces added for consistency. wdyt?
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
airbyte_cdk/sources/declarative/declarative_component_schema.yaml(2 hunks)
🧰 Additional context used
🪛 YAMLlint (1.37.1)
airbyte_cdk/sources/declarative/declarative_component_schema.yaml
[error] 4783-4783: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: Check: source-pokeapi
- GitHub Check: Check: destination-motherduck
- GitHub Check: Check: source-shopify
- GitHub Check: Check: source-intercom
- GitHub Check: Check: source-hardcoded-records
- GitHub Check: SDM Docker Image Build
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (Fast)
- GitHub Check: Analyze (python)
airbyte_cdk/sources/declarative/declarative_component_schema.yaml
Outdated
Show resolved
Hide resolved
There were 2 problems with the info about stream template fields in the declarative_component_schema:
The
components_valuesvariable was not documented in theinterpolationsection, which meant it never showed up in the jinja suggestions menu in the Builder UI. This PR adds it so that it shows up in the UI like this:The examples for the
field_pathin components mappings were not really practical. This PR replaces the examples with ones that users might actually want to use.Summary by CodeRabbit
New Features
Documentation