Skip to content

Enable saving run stats from low-level API #904

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Apr 7, 2022
Merged

Conversation

alonre24
Copy link
Collaborator

@alonre24 alonre24 commented Mar 15, 2022

  • Refactor RAI_RunStats object and API
  • Save stats entry in model/script struct
  • Move stats update to background worker (at DAG post-process) from main thread (in reply callback)
  • Add tests for stats from LLAPI. gears LLAPI, comprehensive synchronisation tests.

This also include some refactor of the run_test_multiproc util, that now allows every sub-process that represent a client to have a unique id (to enable some communications between the parent and children)

…pt struct + move stats update to background worker (post process) from main thread
@codecov
Copy link

codecov bot commented Mar 15, 2022

Codecov Report

Merging #904 (6923e86) into master (2dd4ef1) will increase coverage by 0.06%.
The diff coverage is 96.42%.

@@            Coverage Diff             @@
##           master     #904      +/-   ##
==========================================
+ Coverage   81.30%   81.36%   +0.06%     
==========================================
  Files          54       54              
  Lines        8201     8206       +5     
==========================================
+ Hits         6668     6677       +9     
+ Misses       1533     1529       -4     
Impacted Files Coverage Δ
src/execution/DAG/dag_builder.c 100.00% <ø> (ø)
src/execution/DAG/dag_op.c 100.00% <ø> (ø)
src/execution/parsing/script_commands_parser.c 78.57% <ø> (-0.12%) ⬇️
.../serialization/RDB/decoder/previous/v2/decode_v2.c 46.37% <50.00%> (+0.45%) ⬆️
tests/module/LLAPI.c 75.89% <80.00%> (-0.02%) ⬇️
src/execution/background_workers.c 94.63% <94.73%> (+0.01%) ⬆️
src/backends/tensorflow.c 72.43% <100.00%> (+0.06%) ⬆️
src/execution/DAG/dag.c 91.50% <100.00%> (-0.28%) ⬇️
src/execution/execution_contexts/execution_ctx.c 100.00% <100.00%> (ø)
src/execution/execution_contexts/modelRun_ctx.c 76.76% <100.00%> (ø)
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1e393a...6923e86. Read the comment docs.

@alonre24 alonre24 requested review from DvirDukhan and GuyAv46 March 16, 2022 17:40
@alonre24 alonre24 marked this pull request as ready for review March 16, 2022 17:45
@alonre24 alonre24 added ci-test and removed ci-test labels Mar 26, 2022
@alonre24 alonre24 added ci-test and removed ci-test labels Mar 27, 2022
@alonre24 alonre24 requested a review from DvirDukhan April 5, 2022 15:17
@@ -15,6 +15,10 @@ int RAI_InitBackendTF(int (*get_api_fn)(const char *, void *)) {
get_api_fn("RedisModule_Realloc", ((void **)&RedisModule_Realloc));
get_api_fn("RedisModule_Strdup", ((void **)&RedisModule_Strdup));

// Set min logging level to 3 (out of 5) - this is workaround since if TF is writing extensively
// log messages that to stderr, it may cause the system to be stuck.
RedisModule_Assert(putenv("TF_CPP_MIN_LOG_LEVEL=3") == 0);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not 5?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw that it's working with 3 as well, so I left it like that (not really matters...)

@alonre24 alonre24 added ci-test and removed ci-test labels Apr 6, 2022
@alonre24 alonre24 added ci-test and removed ci-test labels Apr 6, 2022
@alonre24 alonre24 added ci-test and removed ci-test labels Apr 6, 2022
Copy link
Collaborator

@DvirDukhan DvirDukhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well Done!

@alonre24 alonre24 added ci-test and removed ci-test labels Apr 6, 2022
@alonre24 alonre24 merged commit cd490b2 into master Apr 7, 2022
@alonre24 alonre24 deleted the save_stats_in_LLAPI branch April 7, 2022 05:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants