Skip to content

Hypercore API ref #3699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
7bc034a
chore: space to validate initial build.
billy-the-fish Jan 9, 2025
b7e966e
chore: add first hypercore files.
billy-the-fish Jan 9, 2025
3899a6e
chore: add api ref files and alter_table to the build.
billy-the-fish Jan 9, 2025
bd9946c
chore: add remove_columnstore_policy to the build.
billy-the-fish Jan 9, 2025
5ad76e2
chore: add convert_to_columnstore to the build.
billy-the-fish Jan 9, 2025
993cf07
chore: add convert_to_rowstore to the build.
billy-the-fish Jan 9, 2025
d1fde35
chore: updates for build.
billy-the-fish Jan 9, 2025
f727241
chore: updates for build.
billy-the-fish Jan 9, 2025
67c638f
chore: remove links from steps in a list.
billy-the-fish Jan 9, 2025
39c8f8c
chore: remove links from steps in a list.
billy-the-fish Jan 9, 2025
193f0d6
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 9, 2025
af34323
chore: hopefully this will work now.
billy-the-fish Jan 9, 2025
a80f250
chore: add deprecation for compression.
billy-the-fish Jan 9, 2025
c03cae9
chore: simplify workflow, keep code for main docs.
billy-the-fish Jan 9, 2025
e58a7c9
chore: updates on review. .
billy-the-fish Jan 9, 2025
338bac1
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 10, 2025
1409dfd
chore: updates on review.
billy-the-fish Jan 10, 2025
d8008e0
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 13, 2025
a4bf7f1
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 14, 2025
de1c436
feat: update structure for hypercore
billy-the-fish Jan 6, 2025
834a406
chore: add replacedby link. Has to be hardcoded so will not work unti…
billy-the-fish Jan 15, 2025
e969ec0
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 15, 2025
fe6d956
Apply suggestions from code review
billy-the-fish Jan 15, 2025
8c87b3c
chore: update on review.
billy-the-fish Jan 15, 2025
25a2bcc
chore: add enable_segmentwise_recompression.
billy-the-fish Jan 16, 2025
c849517
chore: add enable_segmentwise_recompression.
billy-the-fish Jan 16, 2025
be89feb
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 16, 2025
93ebdde
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 17, 2025
e4a4a57
chore: update deprecation notices.
billy-the-fish Jan 17, 2025
8025d11
chore: update deprecation notices.
billy-the-fish Jan 17, 2025
ce662f7
chore: remove stuff breaking the build.
billy-the-fish Jan 17, 2025
cc5030d
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 20, 2025
7ca04c9
chore: remove stuff breaking the build.
billy-the-fish Jan 18, 2025
629d4a8
chore: updates on review.
billy-the-fish Jan 23, 2025
7e0eae4
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 23, 2025
da173a7
Add API description for merge_chunks() (#3738)
erimatnor Jan 23, 2025
277031e
Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…
billy-the-fish Jan 27, 2025
41564b4
chore: update return table.
billy-the-fish Jan 28, 2025
80b0e3d
chore: change from deprecated to Old API.
billy-the-fish Jan 29, 2025
e5135b4
chore: change from deprecated to Old API.
billy-the-fish Jan 29, 2025
5deb4f8
chore: update on review.
billy-the-fish Jan 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions _partials/_deprecated_2_18_0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<Tag variant="hollow">Old API from [TimescaleDB v2.18.0](https://github.com/timescale/timescaledb/releases/tag/2.18.0)</Tag>
65 changes: 65 additions & 0 deletions _partials/_hypercore_policy_workflow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
1. **Enable columnstore**

* [Use `ALTER TABLE` for a hypertable][alter_table_hypercore]
```sql
ALTER TABLE stocks_real_time SET (timescaledb.enable_columnstore = true, timescaledb.segmentby = 'symbol');
```
* [Use ALTER MATERIALIZED VIEW for a continuous aggregate][compression_continuous-aggregate]
```sql
ALTER MATERIALIZED VIEW stock_candlestick_daily set (timescaledb.enable_columnstore = true, timescaledb.segmentby = 'symbol' );
```

1. **Add a policy to move chunks to the columnstore at a specific time interval**

For example, 60 days after the data was added to the table:
``` sql
CALL add_columnstore_policy('older_stock_prices', after => INTERVAL '60d');
```
See [add_columnstore_policy][add_columnstore_policy].

1. **View the policies that you set or the policies that already exist**

``` sql
SELECT * FROM timescaledb_information.jobs
WHERE proc_name='policy_compression';
```
See [timescaledb_information.jobs][informational-views].

1. **Pause a columnstore policy**

``` sql
SELECT * FROM timescaledb_information.jobs where proc_name = 'policy_compression' AND relname = 'stocks_real_time'

-- Select the JOB_ID from the results

SELECT alter_job(JOB_ID, scheduled => false);
```
See [alter_job][alter_job].

1. **Restart a columnstore policy**
``` sql
SELECT alter_job(JOB_ID, scheduled => true);
```
See [alter_job][alter_job].

1. **Remove a columnstore policy**
``` sql
CALL remove_columnstore_policy('older_stock_prices');
```
See [remove_columnstore_policy][remove_columnstore_policy].
1. **Disable columnstore**

If your table has chunks in the columnstore, you have to
[convert the chunks back to the rowstore][convert_to_rowstore] before you disable the columnstore.
``` sql
ALTER TABLE stocks_real_time SET (timescaledb.enable_columnstore = false);
```
See [alter_table_hypercore][alter_table_hypercore].


[alter_table_hypercore]: /api/:currentVersion:/hypercore/alter_table/
[alter_job]: /api/:currentVersion:/actions/alter_job/
[add_columnstore_policy]: /api/:currentVersion:/hypercore/add_columnstore_policy/
[informational-views]: /api/:currentVersion:/informational-views/jobs/
[remove_columnstore_policy]: /api/:currentVersion:/hypercore/remove_columnstore_policy/
[convert_to_rowstore]: /api/:currentVersion:/hypercore/convert_to_rowstore/
3 changes: 2 additions & 1 deletion _partials/_multi-node-deprecation.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
<Highlight type="warning">

[Multi-node support is deprecated][multi-node-deprecation].
[Multi-node support is sunsetted][multi-node-deprecation].

TimescaleDB v2.13 is the last release that includes multi-node support for PostgreSQL
versions 13, 14, and 15.

</Highlight>

[multi-node-deprecation]: https://github.com/timescale/timescaledb/blob/main/docs/MultiNodeDeprecation.md
1 change: 1 addition & 0 deletions _partials/_since_2_18_0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<Tag variant="hollow">Since [TimescaleDB v2.18.0](https://github.com/timescale/timescaledb/releases/tag/2.18.0)</Tag>
77 changes: 0 additions & 77 deletions api/alter_table_compression.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@ api:
license: community
type: function
---
import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# add_compression_policy() <Tag type="community" content="community" />

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/add_columnstore_policy/">add_columnstore_policy()</a>.

Allows you to set a policy by which the system compresses a chunk
automatically in the background after it reaches a given age.

Expand Down Expand Up @@ -89,3 +92,4 @@ SELECT add_compression_policy('cpu_weekly', INTERVAL '8 weeks');
[compression_continuous-aggregate]: /api/:currentVersion:/continuous-aggregates/alter_materialized_view/
[set_integer_now_func]: /api/:currentVersion:/hypertable/set_integer_now_func
[informational-views]: /api/:currentVersion:/informational-views/jobs/

82 changes: 82 additions & 0 deletions api/compression/alter_table_compression.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
api_name: ALTER TABLE (Compression)
excerpt: Change compression settings on a compressed hypertable
topics: [compression]
keywords: [compression]
tags: [settings, hypertables, alter, change]
api:
license: community
type: command
---
import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# ALTER TABLE (Compression) <Tag type="community" content="community" />

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/alter_table/">ALTER TABLE (Hypercore)</a>.

'ALTER TABLE' statement is used to turn on compression and set compression
options.

By itself, this `ALTER` statement alone does not compress a hypertable. To do so, either create a
compression policy using the [add_compression_policy][add_compression_policy] function or manually
compress a specific hypertable chunk using the [compress_chunk][compress_chunk] function.

The syntax is:

``` sql
ALTER TABLE <table_name> SET (timescaledb.compress,
timescaledb.compress_orderby = '<column_name> [ASC | DESC] [ NULLS { FIRST | LAST } ] [, ...]',
timescaledb.compress_segmentby = '<column_name> [, ...]',
timescaledb.compress_chunk_time_interval='interval'
);
```

## Required arguments

|Name|Type|Description|
|-|-|-|
|`timescaledb.compress`|BOOLEAN|Enable or disable compression|

## Optional arguments

|Name|Type| Description |
|-|-|--|
|`timescaledb.compress_orderby`|TEXT| Order used by compression, specified in the same way as the ORDER BY clause in a SELECT query. The default is the descending order of the hypertable's time column. |
|`timescaledb.compress_segmentby`|TEXT| Column list on which to key the compressed segments. An identifier representing the source of the data such as `device_id` or `tags_id` is usually a good candidate. The default is no `segment by` columns. |
|`timescaledb.enable_segmentwise_recompression`|TEXT| Set to `OFF` to disable segmentwise recompression on compressed chunks. This can be beneficial for some user workloads where segmentwise recompression is slow, and full recompression is more performant. The default is `ON`. |
|`timescaledb.compress_chunk_time_interval`|TEXT| EXPERIMENTAL: Set compressed chunk time interval used to roll chunks into. This parameter compresses every chunk, and then irreversibly merges it into a previous adjacent chunk if possible, to reduce the total number of chunks in the hypertable. Note that chunks will not be split up during decompression. It should be set to a multiple of the current chunk interval. This option can be changed independently of other compression settings and does not require the `timescaledb.compress` argument. |


## Parameters

|Name|Type|Description|
|-|-|-|
|`table_name`|TEXT|Hypertable that supports compression|
|`column_name`|TEXT|Column used to order by or segment by|
|`interval`|TEXT|Time interval used to roll compressed chunks into|

## Sample usage

Configure a hypertable that ingests device data to use compression. Here, if the hypertable
is often queried about a specific device or set of devices, the compression should be
segmented using the `device_id` for greater performance.

```sql
ALTER TABLE metrics SET (timescaledb.compress, timescaledb.compress_orderby = 'time DESC', timescaledb.compress_segmentby = 'device_id');
```

You can also specify compressed chunk interval without changing other
compression settings:

```sql
ALTER TABLE metrics SET (timescaledb.compress_chunk_time_interval = '24 hours');
```

To disable the previously set option, set the interval to 0:

```sql
ALTER TABLE metrics SET (timescaledb.compress_chunk_time_interval = '0');
```

[add_compression_policy]: /api/:currentVersion:/compression/add_compression_policy/
[compress_chunk]: /api/:currentVersion:/compression/compress_chunk/
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@ api:
license: community
type: function
---
import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# chunk_compression_stats() <Tag type="community">Community</Tag>

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/chunk_columnstore_stats/">chunk_columnstore_stats()</a>.

Get chunk-specific statistics related to hypertable compression.
All sizes are in bytes.

Expand Down
5 changes: 5 additions & 0 deletions api/compress_chunk.md → api/compression/compress_chunk.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,12 @@ api:
type: function
---

import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# compress_chunk() <Tag type="community">Community</Tag>

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/convert_to_columnstore/">convert_to_columnstore()</a>.

The `compress_chunk` function is used to compress (or recompress, if necessary)
a specific chunk. This is most often used instead of the
[`add_compression_policy`][add_compression_policy] function, when a user
Expand All @@ -27,6 +31,7 @@ You can get a list of chunks belonging to a hypertable using the
[`show_chunks` function](/api/latest/hypertable/show_chunks/).
</Highlight>


### Required arguments

|Name|Type|Description|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,12 @@ api:
license: community
type: function
---
import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# decompress_chunk() <Tag type="community">Community</Tag>

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/convert_to_cconvert_to_rowstore/">convert_to_rowstore()</a>.

If you need to modify or add a lot of data to a chunk that has already been
compressed, you should decompress the chunk first. This is especially
useful for backfilling old data.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@ api:
license: community
type: function
---
import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# hypertable_compression_stats() <Tag type="community">Community</Tag>

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/hypertable_columnstore_stats/">hypertable_columnstore_stats()</a>.

Get statistics related to hypertable compression. All sizes are in bytes.

For more information about using hypertables, including chunk size partitioning,
Expand Down
9 changes: 8 additions & 1 deletion api/compression.md → api/compression/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,13 @@ excerpt: Compress your hypertable
keywords: [compression]
tags: [hypertables]
---
import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# Compression <Tag type="community">Community</Tag>
# Compression (Old API, use Hypercore) <Tag type="community">Community</Tag>

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/">Hypercore</a>.

Compression functionality is included in Hypercore.

Before you set up compression, you need to
[configure the hypertable for compression][configure-compression] and then
Expand All @@ -30,6 +35,7 @@ Compressed chunks have the following limitations:
after constraint creation.
* [Timescale SkipScan][skipscan] does not currently work on compressed chunks.


## Restrictions

In general, compressing a hypertable imposes some limitations on the types
Expand Down Expand Up @@ -59,3 +65,4 @@ You can also use advanced insert statements like `ON CONFLICT` and `RETURNING`.
[compress_chunk]: /api/:currentVersion:/compression/compress_chunk/
[configure-compression]: /api/:currentVersion:/compression/alter_table_compression/
[skipscan]: /use-timescale/:currentVersion:/query-data/skipscan/
[hypercore]: /api/:currentVersion:/hypercore/
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,12 @@ api:
type: function
---

import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# recompress_chunk() <Tag type="community" content="Community" />

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/convert_to_columnstore/">convert_to_columnstore()</a>.

Recompresses a compressed chunk that had more data inserted after compression.

```sql
Expand Down Expand Up @@ -41,6 +45,7 @@ the procedure with `CALL`. Don't use a `SELECT` statement.
chunk for the first time, use [`compress_chunk`](/api/latest/compression/compress_chunk/).
</Highlight>


## Required arguments

|Name|Type|Description|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@ api:
license: community
type: function
---
import Deprecated2180 from "versionContent/_partials/_deprecated_2_18_0.mdx";

# remove_compression_policy() <Tag type="community" content="community" />

<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/remove_columnstore_policy/">remove_columnstore_policy()</a>.

If you need to remove the compression policy. To restart policy-based
compression you need to add the policy again. To view the policies that
already exist, see [informational views][informational-views].
Expand Down
4 changes: 2 additions & 2 deletions api/distributed-hypertables.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Distributed hypertables
title: Distributed hypertables ( Sunsetted v2.14.x )
excerpt: Create and manage distributed hypertables
keywords: [distributed hypertables]
---
Expand All @@ -8,7 +8,7 @@ import MultiNodeDeprecation from "versionContent/_partials/_multi-node-deprecati

<MultiNodeDeprecation />

# Distributed Hypertables <Tag type="community">Community</Tag>
# Distributed hypertables ( Sunsetted v2.14.x) <Tag type="community">Community</Tag>

Distributed hypertables are an extension of regular hypertables, available when
using a [multi-node installation][getting-started-multi-node] of TimescaleDB.
Expand Down
Loading