Skip to content

Adding specification for the new data stream settings APIs #4416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

masseyke
Copy link
Member

This adds specifications for the new data stream settings APIs.

@masseyke masseyke requested a review from a team as a code owner May 27, 2025 18:45
@masseyke masseyke added specification skip-backport This pull request should not be backported labels May 27, 2025
indices-get-field-mapping,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-mapping
indices-get-index,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get
indices-get-mapping,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-mapping
indices-get-settings,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings
indices-get-template,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-index-template
indices-get-template-v1,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-template
indices-open-close,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-open
indices-put-data-stream-settings,https://www.elastic.co/guide/en/elasticsearch/reference/master/data-streams.html
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only URL in this whole file that links to a docs v2 page. @lcawl @georgewallace Does this need to be changed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had just put in the placeholder we had used on the elasticsearch side. I can change it to something else. Looking now to see what fits best.

@masseyke masseyke requested a review from JoshMock May 27, 2025 20:41
Copy link
Contributor

@lcawl lcawl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a lot of minor suggestions, but the important one is related to the @doc_tag. It would also be nice to have request and response examples per https://github.com/elastic/elasticsearch-specification/blob/main/docs/add-new-api.md#add-endpoint-request-and-response-examples

Copy link
Contributor

Following you can find the validation results for the APIs you have changed.

API Status Request Response
indices.add_block 🟢 2/2 2/2
indices.analyze 🟢 246/246 246/246
indices.cancel_migrate_reindex 🟢 2/2 2/2
indices.clear_cache 🟢 4/4 4/4
indices.clone 🟢 5/5 5/5
indices.close 🟢 49/49 49/49
indices.create_data_stream 🟢 31/31 31/31
indices.create_from 🟢 4/4 4/4
indices.create 🔴 1179/1225 1225/1225
indices.data_streams_stats 🟢 4/4 4/4
indices.delete_alias 🔴 12/12 5/12
indices.delete_data_lifecycle Missing test Missing test
indices.delete_data_stream_options 🟠 Missing type Missing type
indices.delete_data_stream 🟢 28/28 28/28
indices.delete_index_template 🟢 1/1 1/1
indices.delete_template 🟢 9/9 9/9
indices.delete 🟢 105/105 105/105
indices.disk_usage 🟢 6/6 6/6
indices.downsample 🟢 4/4 4/4
indices.exists_alias 🟢 37/37 37/37
indices.exists_index_template Missing test Missing test
indices.exists_template 🟢 15/15 15/15
indices.exists 🟢 39/39 39/39
indices.explain_data_lifecycle Missing test Missing test
indices.field_usage_stats 🟢 5/5 5/5
indices.flush 🟢 41/41 41/41
indices.forcemerge 🟢 12/12 12/12
indices.get_alias 🟢 68/68 68/68
indices.get_data_lifecycle_stats Missing test Missing test
indices.get_data_lifecycle Missing test Missing test
indices.get_data_stream_options 🟠 Missing type Missing type
indices.get_data_stream_settings Missing test Missing test
indices.get_data_stream 🟢 11/11 11/11
indices.get_field_mapping 🟢 15/15 15/15
indices.get_index_template 🟢 21/21 21/21
indices.get_mapping 🟢 142/142 142/142
indices.get_migrate_reindex_status 🟢 1/1 1/1
indices.get_settings 🔴 76/76 47/76
indices.get_template 🟢 32/32 32/32
indices.get 🟢 55/55 55/55
indices.migrate_reindex 🟢 2/2 2/2
indices.migrate_to_data_stream Missing test Missing test
indices.modify_data_stream Missing test Missing test
indices.open 🟢 20/20 20/20
indices.promote_data_stream Missing test Missing test
indices.put_alias 🔴 58/58 20/58
indices.put_data_lifecycle Missing test Missing test
indices.put_data_stream_options 🟠 Missing type Missing type
indices.put_data_stream_settings Missing test Missing test
indices.put_index_template 🟢 68/68 68/68
indices.put_mapping 🔴 124/125 128/128
indices.put_settings 🔴 56/58 58/58
indices.put_template 🔴 45/46 46/46
indices.recovery 🟢 9/9 9/9
indices.refresh 🟢 283/283 283/283
indices.reload_search_analyzers 🟢 2/2 2/2
indices.resolve_cluster 🟢 13/13 13/13
indices.resolve_index 🟢 6/6 6/6
indices.rollover 🔴 27/27 2/27
indices.segments 🔴 6/6 5/6
indices.shard_stores 🔴 5/5 4/5
indices.shrink 🟢 5/5 5/5
indices.simulate_index_template 🔴 6/7 5/7
indices.simulate_template 🔴 5/5 4/5
indices.split 🟢 11/11 11/11
indices.stats 🔴 123/123 31/122
indices.update_aliases 🔴 26/26 7/26
indices.validate_query 🟢 12/12 12/12

You can validate these APIs yourself by using the make validate target.

@masseyke masseyke requested a review from lcawl May 28, 2025 19:27
@elastic elastic deleted a comment from github-actions bot May 28, 2025
@elastic elastic deleted a comment from github-actions bot May 28, 2025
@elastic elastic deleted a comment from github-actions bot May 28, 2025
@elastic elastic deleted a comment from github-actions bot May 28, 2025
@elastic elastic deleted a comment from github-actions bot May 28, 2025
@elastic elastic deleted a comment from github-actions bot May 28, 2025
@elastic elastic deleted a comment from github-actions bot May 28, 2025
@elastic elastic deleted a comment from github-actions bot May 28, 2025
export class DataStreamSettings {
/** The name of the data stream. */
name: string
settings: IndexSettings
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

settings: IndexSettings

The example includes effective_settings. If that's valid, shouldn't it appear here too?

@@ -305,13 +305,15 @@ indices-exists,https://www.elastic.co/docs/api/doc/elasticsearch/operation/opera
indices-flush,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-flush
indices-forcemerge,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge
indices-get-alias,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-alias
indices-get-data-stream-settings,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-get-data-stream-settings
Copy link
Contributor

@lcawl lcawl May 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I test a preview where this API is ga, the URL looks like this, since I think it's derived from the rest_spec_name (or rather the operationID in OpenAPI)

Suggested change
indices-get-data-stream-settings,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-get-data-stream-settings
indices-get-data-stream-settings,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-data-stream-settings

indices-get-field-mapping,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-mapping
indices-get-index,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get
indices-get-mapping,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-mapping
indices-get-settings,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings
indices-get-template,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-index-template
indices-get-template-v1,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-template
indices-open-close,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-open
indices-put-data-stream-settings,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-put-data-stream-settings
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I test this in a preview, the URL is:

Suggested change
indices-put-data-stream-settings,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-put-data-stream-settings
indices-put-data-stream-settings,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-data-stream-settings

}
/** Settings to be applied to the data stream.
* @codegen_name settings */
body: IndexSettings
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

body: IndexSettings

This currently seems to result in a kind of recursive situation in the OpenAPI output:

  "requestBody": {
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/indices._types.IndexSettings"
              },
...
      "indices._types.IndexSettings": {
        "type": "object",
        "properties": {
          "index": {
            "$ref": "#/components/schemas/indices._types.IndexSettings"
          },
          "mode": {
            "type": "string"
          },
...

We don't have that level of detail in our old docs (e.g. https://www.elastic.co/guide/en/elasticsearch/reference/8.18/indices-update-settings.html or https://www.elastic.co/guide/en/elasticsearch/reference/8.18/index-modules.html#index-modules-settings), so I want to call it out to ensure it's accurate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip-backport This pull request should not be backported specification
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants