Skip to content

Use "merger" for fetch&sort tuples on router #38

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Mar 24, 2021

Conversation

olegrok
Copy link
Contributor

@olegrok olegrok commented Oct 2, 2020

Before this patch we used hand-crafted heap to sort tuples on
router. This patch removes it and changes to "merger" module
written in C.

"Merger" is available since Tarantool 2.2. Later "tuple-merger"
was introduced it's a version that is compatible with some
Tarantool 1.10 versions. In fact support several Tarantool
versions is quite complex task. Some msgpack helpers is not
avaliable in Tarantool 1.10 to solve this problem small
compatibility lawer is implemented.
Also "tuple-keydef" doesn't support collation_id options and it's
not so for built-in "key_def" module. We need to normalize index
parts and change "collation_id" to "collation" when create
"key_def" instance.

For versions where merger is not supported/available yet
(before 1.10.8 and before 2.3.3) we use previous approach with
lua-implemented heap.

Closes #33

@olegrok olegrok force-pushed the olegrok/33-merger-integraion branch 3 times, most recently from 66fbe72 to f79d7d1 Compare October 5, 2020 14:23
@olegrok olegrok mentioned this pull request Oct 5, 2020
3 tasks
@olegrok olegrok force-pushed the olegrok/33-merger-integraion branch 3 times, most recently from 9cdf7e5 to bdad3f4 Compare October 6, 2020 13:27
@olegrok olegrok force-pushed the olegrok/33-merger-integraion branch 3 times, most recently from 4d258f1 to 324e518 Compare October 26, 2020 12:16
@olegrok olegrok force-pushed the olegrok/33-merger-integraion branch 13 times, most recently from 8204e0f to 52eea03 Compare November 2, 2020 08:21
@olegrok
Copy link
Contributor Author

olegrok commented Nov 11, 2020

https://github.com/tarantool/crud/tree/perf-test

Before patch:

SELECT
Fibers count - 200
Connection count - 10
Timeout  - 30.000000
Requests - 102824
Errors - 0
RPS - 3427.466667

After patch:

SELECT
Fibers count - 200
Connection count - 10
Timeout  - 30.000000
Requests - 168745
Errors - 0
RPS - 5624.833333
Ok

@olegrok olegrok changed the title select: first dirty merger integraion attempt Use "merger" for fetch&sort tuples on router Nov 11, 2020
@olegrok olegrok requested a review from dokshina November 11, 2020 09:36
@olegrok olegrok force-pushed the olegrok/33-merger-integraion branch from 4a674dc to 6031809 Compare November 11, 2020 10:34

function select_module.pairs(space_name, user_conditions, opts)
checks('string', '?table', {
after = '?table|cdata',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's better to use ?table|tuple instead of *|cdata

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same for all places where tuple expected.

@AnaNek AnaNek force-pushed the olegrok/33-merger-integraion branch from 2efb78d to 93b5684 Compare March 19, 2021 09:14
@AnaNek AnaNek force-pushed the olegrok/33-merger-integraion branch from 83fff32 to 0714f46 Compare March 24, 2021 08:55
Copy link
Contributor

@dokshina dokshina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great patch!

@AnaNek AnaNek merged commit cf48e25 into master Mar 24, 2021
@AnaNek AnaNek deleted the olegrok/33-merger-integraion branch March 24, 2021 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[4pt] Investigate merger/key_def embedding into current implementation
3 participants