Run model from script #787
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… multiple devices). Tests pass (new feature hasn't tested yet)
…w device is introduced and use rwlock to synchronise.
…'t overflows than the tensor type in TENSORSET.
…aces that are affected)
…into ONNX_kill_switch
…om backend len (supported only for onnx now) in INFO MODULES command.
- Add a state flag to every entry in the onnx run sessions array and update it atomically, to avoid situations where main threads and bg thread both access the runOptions field. - Refactor the info_modules section, and change AI.INFO command so it must receive a module/script key. The other info will be accessible as part of the info modules command.
… instance). test info command with AI fields
Codecov Report
@@ Coverage Diff @@
## master #787 +/- ##
==========================================
+ Coverage 74.11% 80.40% +6.28%
==========================================
Files 39 52 +13
Lines 6081 7883 +1802
==========================================
+ Hits 4507 6338 +1831
+ Misses 1574 1545 -29
Continue to review full report at Codecov.
|
- Add the option the include the backends api from multiple sources in c++ project.
…disAI into run_model_from_script
DvirDukhan
reviewed
Jun 21, 2021
DvirDukhan
approved these changes
Jun 22, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR extends the external commands that torch script already supports through RedisAI (executing unblocking Redis commands), to support the execution of RedisAI models as well by adding a new
redisAI.execute_modelcommand to torch script. This command receives a model name that is stored in RedisAI, a list of the input tensors and the outputs number, and returns the output tensors.This is tested by running models from torch script in all 3 backends.
This PR includes a unified arrangement of the
device_idfield indl_tensor- this should be -1 whenever tensor is created for default CPU, otherwise torch interpreters it as if it should be sent to a different device (fordevice_id=0, the device is considered as CPU:0 insread of CPU).Todo: figure out how (or if) to pass error message from torch extension back to RedisAI, to have more informative failing (currently a generic error message is returned)