Skip to content

Introduce AI.DAGEXECUTE command, AI.DAGRUN is now deprecated. #723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
May 10, 2021

Conversation

alonre24
Copy link
Collaborator

@alonre24 alonre24 commented May 6, 2021

Add the new command according to the new design, no actual change in semantics.
The main difference between AI.DAGRUN and AI.DAGEXECUTE is:

  1. AI.DAGEXECUTE may contain only (non-deprecated) *EXECUTE ops, while AI.DAGRUN may contain only *RUN (deprecated) ops.
  2. AI.DAGEXECUTE has KEYS section (like AI.SCRIPTEXECUTE). At least one out of LOAD, PERSIST, KEYS sections must appear in the command. KEYS section shouldn't appear in inner AI.SCRIPTEXCUTE commands of the DAG.
  3. AI.DAGEXECUTE_RO cannot contain an inner AI.SCRIPTEXECUTE op, as it may perform write operation from within the script.

@alonre24 alonre24 requested review from lantiga and DvirDukhan May 6, 2021 14:23
@codecov
Copy link

codecov bot commented May 6, 2021

Codecov Report

Merging #723 (2e8ba43) into master (f7b870e) will increase coverage by 0.36%.
The diff coverage is 84.18%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #723      +/-   ##
==========================================
+ Coverage   78.93%   79.30%   +0.36%     
==========================================
  Files          48       48              
  Lines        7343     7474     +131     
==========================================
+ Hits         5796     5927     +131     
  Misses       1547     1547              
Impacted Files Coverage Δ
src/execution/utils.c 67.85% <0.00%> (ø)
...c/serialization/RDB/decoder/current/v1/decode_v1.c 64.63% <0.00%> (ø)
.../serialization/RDB/decoder/previous/v0/decode_v0.c 61.39% <0.00%> (ø)
src/redisai.c 85.42% <60.71%> (-0.41%) ⬇️
src/execution/parsing/deprecated.c 80.67% <87.32%> (+2.74%) ⬆️
src/execution/parsing/parse_utils.c 92.30% <88.23%> (-7.70%) ⬇️
src/execution/parsing/dag_parser.c 97.43% <93.84%> (+2.11%) ⬆️
src/execution/DAG/dag.c 89.28% <94.73%> (-0.07%) ⬇️
src/execution/DAG/dag_builder.c 100.00% <100.00%> (ø)
src/execution/command_parser.c 92.15% <100.00%> (+0.49%) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc94901...2e8ba43. Read the comment docs.

…sing + add tests.

Fix "validateKeysInShard" to work on a single shard in enterprise.
Copy link
Collaborator

@DvirDukhan DvirDukhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good
mainly cosmetics changes
please split tests_dag.py into several test files so it will be easier to track the logic

@alonre24 alonre24 self-assigned this May 9, 2021
env.assertEqual(ret, b'OK')

command = "AI.DAGEXECUTE LOAD 2 persisted_tensor_1{1} persisted_tensor_2:{1}" \
" PERSIST 1 volatile_tensor_persisted |> " \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should/might fail on cluster

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! found a bug - fixing

@alonre24 alonre24 added ci-test and removed ci-test labels May 10, 2021
@alonre24 alonre24 added ci-test and removed ci-test labels May 10, 2021
@alonre24 alonre24 added ci-test and removed ci-test labels May 10, 2021
@alonre24 alonre24 added ci-test and removed ci-test labels May 10, 2021
@alonre24 alonre24 added ci-test and removed ci-test labels May 10, 2021
@alonre24 alonre24 added ci-test and removed ci-test labels May 10, 2021
@DvirDukhan DvirDukhan merged commit ece8896 into master May 10, 2021
@DvirDukhan DvirDukhan deleted the DAG_execute_command branch May 10, 2021 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants