Skip to content

Calls to routers and storages when crud-router and crud-storage roles initialization is not finished yet #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
savinov opened this issue Oct 21, 2021 · 5 comments · Fixed by #299
Closed

Calls to routers and storages when crud-router and crud-storage roles initialization is not finished yet #229

savinov opened this issue Oct 21, 2021 · 5 comments · Fixed by #299
Assignees
@psergee
Labels
bug Something isn't working

Comments

@savinov
Copy link

savinov commented Oct 21, 2021

There are exceptions trown in cartridge-java driver due to crud-router and crud-storage roles initialization issues.

How to repoduce:

  • run some load test to call crud.select (with prefer_replica=true and balance=true) and crud.upsert
  • stop/start a few routers
  • stop/start a few replicas (not leaders) in replicasets

Result for select:

org.springframework.dao.DataRetrievalFailureException: InnerErrorMessage:
code: 33
message: Procedure '_crud.select_on_storage' is not defined; nested exception is io.tarantool.driver.exceptions.TarantoolInternalException: InnerErrorMessage:
code: 33
message: Procedure '_crud.select_on_storage' is not defined

Result for upsert:

Caused by: io.tarantool.driver.exceptions.TarantoolInternalException: InnerErrorMessage:
code: 32
message: ...example.lua:100: variable 'crud' is not declared
@Totktonada
Copy link
Member

The example.lua code that calls a crud function works on a router, correct? Whether this code is in a role, which depends on the 'crud-router' role? (Or, whether it is guaranteed that the call to crud is always performed after crud.init_router()?)

The documentation does not shed light on the order of roles initialization, but I guess that dependent roles are initialized first. I filed tarantool/doc#2371 regarding this several days ago.

@savinov
Copy link
Author

savinov commented Oct 21, 2021

Correct, example.lua works on a router. The code is in a role which depends on crud-router.
Somehow crud variable and _crud.select_on_storage function are not defined at the call moment.

@Totktonada Totktonada added bug Something isn't working customer teamE labels Oct 21, 2021
@savinov
Copy link
Author

savinov commented Dec 2, 2021

SDK version: Tarantool Enterprise 2.7.3-0-gdddf926c3-r428
Cartridge version: 2.7.2
CRUD version: 0.9.0

@Totktonada Totktonada mentioned this issue Dec 2, 2021
Closed
@Totktonada
Copy link
Member

The problem with storages should be generally resolved after tarantool/vshard#298, however we can implement some simple retrying strategy in crud even without this feature.

The problem with routers should be resolved by tarantool/cartridge-java#159 and, sadly, there is nothing that we can fix on the crud side.

@Totktonada Totktonada mentioned this issue Jan 22, 2022
Open
@psergee psergee mentioned this issue May 30, 2022
Merged
psergee added a commit that referenced this issue May 31, 2022
@psergee
Make select error more informative
d20b55b 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

See also #229
psergee added a commit that referenced this issue May 31, 2022
@psergee
Merge branch 'psergee/gh-229-crud-storage-init-status' of github.com:…
8cd1562 
…tarantool/crud into psergee/gh-229-crud-storage-init-status
psergee added a commit that referenced this issue Jun 2, 2022
@psergee
Provide an API to get storage intialization state
18af0da 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can wait for storages to be
initalized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jun 2, 2022
@psergee
Provide an API to get storages initialization state
0bbcf76 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get an information about storages
state: initialized or not. So, a user can wait for storages to be
initalized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jun 2, 2022
@psergee
Provide an API to get storages initialization state
f32ad8e 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
@psergee psergee mentioned this issue Jun 2, 2022
Merged
psergee added a commit that referenced this issue Jun 2, 2022
@psergee
Make select error more informative
a7d44a2 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

Part of #229
psergee added a commit that referenced this issue Jun 2, 2022
@psergee
Make select error more informative
355393e 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

See also #229
psergee added a commit that referenced this issue Jun 3, 2022
@psergee
Make select error more informative
9fbb70b 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

See also #229
psergee added a commit that referenced this issue Jun 6, 2022
@psergee
Make select error more informative
1d587f4 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

Part of #229
psergee added a commit that referenced this issue Jun 9, 2022
@psergee
Make select error more informative
8f3cea3 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

Part of #229
psergee added a commit that referenced this issue Jun 15, 2022
@psergee
Make select error more informative
82c5ad2 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

Part of #229
psergee added a commit that referenced this issue Jun 15, 2022
@psergee
Make select error more informative
6d57151 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

Part of #229
DifferentialOrange pushed a commit that referenced this issue Jun 15, 2022
@psergee @DifferentialOrange
Make select error more informative
182d46b 
Report "crud is not initialized" error if storage functions
are not defined.
Before this patch, select error description does not give
any clues about possible causes of undefined storage functions.

Part of #229
DifferentialOrange added a commit that referenced this issue Jun 15, 2022
@DifferentialOrange
Release 0.11.3
de6dfa3 
Overview

    This is a bugfix release. Several cases of pagination using select
    with after was fixed or improved. Warning has been added to
    potentially long select and count calls. Storage select errors
    were reworked to be consistent with other calls errors.

Breaking changes

    There are no breaking changes in the release.

New features

    * Optimize `crud.select()` without conditions and with
    `after` (PR #295).

    * A critical log entry containing the current stack traceback is
    created upon potentially long `select` and `count` calls —
    an user can explicitly request a full scan through by passing
    `fullscan=true` to `select` or `count` options table argument
    in which case a log entry will not be created (#276).

Bugfixes

    * Make select error description more informative when merger
    built-in module or tuple-merger external module is used
    in case of disabled/uninit storage (#229).

    * `crud.select()` if a condition is '<=' and it's
    value < `after` (PR #295).

    Previous releases:

    tarantool> crud.select('developers',
             > {{'<=', 'id', 3}}, {first = 10, after = rows[5]}).rows
    ---
    - - [2, 401, 'Sergey', 'Allred', 21]
      - [1, 477, 'Alexey', 'Adams', 20]
    ...

    After this release:

    tarantool> crud.select('developers',
             > {{'<=', 'id', 3}}, {first = 10, after = rows[5]}).rows
    ---
    - - [3, 2804, 'Pavel', 'Adams', 27]
      - [2, 401, 'Sergey', 'Allred', 21]
      - [1, 477, 'Alexey', 'Adams', 20]
    ...

    * `crud.select()` filtration by a first condition if the condition
    is '>' or '>=' and it's value > `after` (PR #295).

    Previous releases:

    tarantool> crud.select('developers',
             > {{'>=', 'id', 5}}, {first = 10, after = rows[2]}).rows
    ---
    - - [3, 2804, 'Pavel', 'Adams', 27]
      - [4, 1161, 'Mikhail', 'Liston', 51]
      - [5, 1172, 'Dmitry', 'Jacobi', 16]
      - [6, 1064, 'Alexey', 'Sidorov', 31]
    ...

    After this release:

    tarantool> crud.select('developers',
             > {{'>=', 'id', 5}}, {first = 10, after = rows[2]}).rows
    ---
      - [5, 1172, 'Dmitry', 'Jacobi', 16]
      - [6, 1064, 'Alexey', 'Sidorov', 31]
    ...

    * `crud.select()` results order with negative `first` (PR #295).
    * `crud.select()` if a condition is '=' or '==' with
    negative `first` (PR #295).

    Suppose we have a non-unique secondary index by the field `age`
    field and a space:

    tarantool> crud.select('developers', nil, {first = 10})
    ---
    - metadata: [{'name': 'id', 'type': 'unsigned'},
        {'name': 'bucket_id', 'type': 'unsigned'},
        {'name': 'name', 'type': 'string'},
        {'name': 'surname', 'type': 'string'},
        {'name': 'age', 'type': 'number'}]
      rows:
      - [1, 477, 'Alexey', 'Adams', 20]
      - [2, 401, 'Sergey', 'Allred', 27]
      - [3, 2804, 'Pavel', 'Adams', 27]
      - [4, 1161, 'Mikhail', 'Liston', 27]
      - [5, 1172, 'Dmitry', 'Jacobi', 27]
      - [6, 1064, 'Alexey', 'Sidorov', 31]
    - null
    ...

    Previous releases:

    tarantool> crud.select('developers',
             > {{'=', 'age', 27}}, {first = -10, after = rows[4]}).rows
    ---
    - []
    ...

    After this release:

    tarantool> crud.select('developers',
             > {{'=', 'age', 27}}, {first = -10, after = rows[4]}).rows
    ---
    - - [2, 401, 'Sergey', 'Allred', 27]
      - [3, 2804, 'Pavel', 'Adams', 27]
    ...
@DifferentialOrange DifferentialOrange mentioned this issue Jun 15, 2022
Merged
DifferentialOrange added a commit that referenced this issue Jun 15, 2022
@DifferentialOrange
Release 0.11.3
8e00652 
Overview

    This is a bugfix release. Several cases of pagination using select
    with after was fixed or improved. Warning has been added to
    potentially long select and count calls. Storage select errors
    were reworked to be consistent with other calls errors.

Breaking changes

    There are no breaking changes in the release.

New features

    * Optimize `crud.select()` without conditions and with
    `after` (PR #295).

    * A critical log entry containing the current stack traceback is
    created upon potentially long `select` and `count` calls —
    an user can explicitly request a full scan through by passing
    `fullscan=true` to `select` or `count` options table argument
    in which case a log entry will not be created (#276).

Bugfixes

    * Make select error description more informative when merger
    built-in module or tuple-merger external module is used
    in case of disabled/uninit storage (#229).

    * `crud.select()` if a condition is '<=' and it's
    value < `after` (PR #295).

    Previous releases:

    tarantool> crud.select('developers',
             > {{'<=', 'id', 3}}, {first = 10, after = rows[5]}).rows
    ---
    - - [2, 401, 'Sergey', 'Allred', 21]
      - [1, 477, 'Alexey', 'Adams', 20]
    ...

    After this release:

    tarantool> crud.select('developers',
             > {{'<=', 'id', 3}}, {first = 10, after = rows[5]}).rows
    ---
    - - [3, 2804, 'Pavel', 'Adams', 27]
      - [2, 401, 'Sergey', 'Allred', 21]
      - [1, 477, 'Alexey', 'Adams', 20]
    ...

    * `crud.select()` filtration by a first condition if the condition
    is '>' or '>=' and it's value > `after` (PR #295).

    Previous releases:

    tarantool> crud.select('developers',
             > {{'>=', 'id', 5}}, {first = 10, after = rows[2]}).rows
    ---
    - - [3, 2804, 'Pavel', 'Adams', 27]
      - [4, 1161, 'Mikhail', 'Liston', 51]
      - [5, 1172, 'Dmitry', 'Jacobi', 16]
      - [6, 1064, 'Alexey', 'Sidorov', 31]
    ...

    After this release:

    tarantool> crud.select('developers',
             > {{'>=', 'id', 5}}, {first = 10, after = rows[2]}).rows
    ---
      - [5, 1172, 'Dmitry', 'Jacobi', 16]
      - [6, 1064, 'Alexey', 'Sidorov', 31]
    ...

    * `crud.select()` results order with negative `first` (PR #295).
    * `crud.select()` if a condition is '=' or '==' with
    negative `first` (PR #295).

    Suppose we have a non-unique secondary index by the field `age`
    field and a space:

    tarantool> crud.select('developers', nil, {first = 10})
    ---
    - metadata: [{'name': 'id', 'type': 'unsigned'},
        {'name': 'bucket_id', 'type': 'unsigned'},
        {'name': 'name', 'type': 'string'},
        {'name': 'surname', 'type': 'string'},
        {'name': 'age', 'type': 'number'}]
      rows:
      - [1, 477, 'Alexey', 'Adams', 20]
      - [2, 401, 'Sergey', 'Allred', 27]
      - [3, 2804, 'Pavel', 'Adams', 27]
      - [4, 1161, 'Mikhail', 'Liston', 27]
      - [5, 1172, 'Dmitry', 'Jacobi', 27]
      - [6, 1064, 'Alexey', 'Sidorov', 31]
    - null
    ...

    Previous releases:

    tarantool> crud.select('developers',
             > {{'=', 'age', 27}}, {first = -10, after = rows[4]}).rows
    ---
    - []
    ...

    After this release:

    tarantool> crud.select('developers',
             > {{'=', 'age', 27}}, {first = -10, after = rows[4]}).rows
    ---
    - - [2, 401, 'Sergey', 'Allred', 27]
      - [3, 2804, 'Pavel', 'Adams', 27]
    ...
psergee added a commit that referenced this issue Jun 22, 2022
@psergee
Provide an API to get storages initialization state
3e2b158 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
@R-omk
Copy link

R-omk commented Jun 22, 2022

Same problem solved in vshard tarantool/vshard#123

psergee added a commit that referenced this issue Jun 27, 2022
@psergee
Provide an API to get storages initialization state
58633d9 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jun 28, 2022
@psergee
Provide an API to get storages initialization state
3035210 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 1, 2022
@psergee
Provide an API to get storages initialization state
428b629 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 8, 2022
@psergee
Provide an API to get storages initialization state
04f1aba 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 11, 2022
@psergee
Provide an API to get storages initialization state
f3aa7c9 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 13, 2022
@psergee
Provide an API to get storages initialization state
fcefce9 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 14, 2022
@psergee
Provide an API to get storages initialization state
41f42cc 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 21, 2022
@psergee
Provide an API to get storages initialization state
dcda205 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 21, 2022
@psergee
Provide an API to get storages initialization state
25c6594 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 22, 2022
@psergee
Provide an API to get storages initialization state
e9c58bf 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 22, 2022
@psergee
Provide an API to get storages initialization state
647dba9 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 25, 2022
@psergee
Provide an API to get storages initialization state
f4792ea 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
psergee added a commit that referenced this issue Jul 26, 2022
@psergee
Provide an API to get storages initialization state
9a730ca 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
DifferentialOrange pushed a commit that referenced this issue Jul 27, 2022
@psergee @DifferentialOrange
Provide an API to get storages initialization state
d053623 
There is an issue with using CRUD functionality if not all storages are
up. New function is added to get the information about storages
state: initialized or not. So, a user can poll state and wait for
storages to be initialized before making CRUD calls.

Resolves #229
@LeonidVas LeonidVas added the 3sp label Aug 3, 2022
DifferentialOrange added a commit that referenced this issue Aug 29, 2022
@DifferentialOrange
Release 0.13.0
994b050 
Overview

    The main feature of this release is the full support of vshard
    sharding functions (see ddl 1.6.2 for corresponding ddl release).

Breaking changes

    There are no breaking changes in the release.

New features

    * `crud.storage_info` function to get storages status (#229).

Bugfixes

    * Fix specifying `vshard` sharding funcs (#314).
DifferentialOrange added a commit that referenced this issue Aug 29, 2022
@DifferentialOrange
Release 0.13.0
68ceffc 
Overview

    The main feature of this release is the full support of vshard
    sharding functions (see ddl 1.6.2 for corresponding `ddl` release).

Breaking changes

    There are no breaking changes in the release.

New features

    * `crud.storage_info` function to get storages status (#229).

Bugfixes

    * Fix specifying `vshard` sharding funcs (#314).
@DifferentialOrange DifferentialOrange mentioned this issue Aug 29, 2022
Merged
DifferentialOrange added a commit that referenced this issue Aug 29, 2022
@DifferentialOrange
Release 0.13.0
029bc01 
Overview

    The main feature of this release is the full support of vshard
    sharding functions (see ddl 1.6.2 for corresponding `ddl` release).

Breaking changes

    There are no breaking changes in the release.

New features

    * `crud.storage_info` function to get storages status (#229).

Bugfixes

    * Fix specifying `vshard` sharding funcs (#314).
DifferentialOrange added a commit that referenced this issue Aug 29, 2022
@DifferentialOrange
Release 0.13.0
e4df508 
Overview

    The main feature of this release is the full support of vshard
    sharding functions (see ddl 1.6.2 for corresponding `ddl` release).

Breaking changes

    There are no breaking changes in the release.

New features

    * `crud.storage_info` function to get storages status (#229).

Bugfixes

    * Fix specifying `vshard` sharding funcs (#314).
DifferentialOrange added a commit that referenced this issue Oct 18, 2022
@DifferentialOrange
api: rework NonInitialized error message
56be360 
Rework `NonInitialized` error message to be more helpful for
troubleshooting.

Follows #229, closes #326
@DifferentialOrange DifferentialOrange mentioned this issue Oct 18, 2022
Merged
2 tasks
DifferentialOrange added a commit that referenced this issue Oct 18, 2022
@DifferentialOrange
api: rework NonInitialized error message
5f7e418 
Rework `NonInitialized` error message to be more helpful for
troubleshooting.

Follows #229, closes #326
@Totktonada Totktonada mentioned this issue Nov 22, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@psergee psergee

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Provide an API to get storages initialization state tarantool/crud
5 participants
@Totktonada @savinov @R-omk @LeonidVas @psergee