Skip to content

Commit 7833e0e

Browse files
lantigalantiga
committed
Revamp GET methods (breaking change for clients)
1 parent 26e352b commit 7833e0e

File tree

7 files changed

+501
-314
lines changed

7 files changed

+501
-314
lines changed

‎docs/commands.md

Lines changed: 161 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -41,20 +41,76 @@ AI.TENSORSET foo FLOAT 2 2 VALUES 1 2 3 4
4141
Get a tensor.
4242

4343
```sql
44-
AI.TENSORGET tensor_key [BLOB | VALUES | META]
44+
AI.TENSORGET tensor_key [META] [BLOB | VALUES]
4545
```
4646

4747
* tensor_key - Key for the tensor
48+
* META - Return tensor meta data (data type and shape)
4849
* BLOB - Return tensor content as a binary buffer
4950
* VALUES - Return tensor content as a list of values
50-
* META - Only return tensor meta data (datat type and shape)
5151

5252
### TENSORGET Example
5353

54-
Get binary data for tensor at `foo`. Meta data is also returned.
54+
Get binary data for tensor at `foo` as a buffer.
5555

5656
```sql
5757
AI.TENSORGET foo BLOB
58+
59+
> ...
60+
```
61+
62+
Get tensor values for tensor at `foo` as a list of numbers.
63+
64+
```sql
65+
AI.TENSORGET foo VALUES
66+
67+
> 1) 1.1234
68+
> 2) 2.5135
69+
> 3) 5.5425
70+
> 4) 4.1524
71+
```
72+
73+
Get meta data about tensor at `foo`
74+
75+
```sql
76+
AI.TENSORGET foo META
77+
78+
> 1) "dtype"
79+
> 2) "FLOAT"
80+
> 3) "shape"
81+
> 4) 1) 4
82+
> 4) 2) 5
83+
```
84+
85+
Get meta data as well as binary data for tensor at `foo`
86+
87+
```sql
88+
AI.TENSORGET foo META BLOB
89+
90+
> 1) "dtype"
91+
> 2) "FLOAT"
92+
> 3) "shape"
93+
> 4) 1) 4
94+
> 4) 2) 5
95+
> 5) "blob"
96+
> 6) ...
97+
```
98+
99+
Get binary data for tensor at `foo`. Meta data is also returned.
100+
101+
```sql
102+
AI.TENSORGET foo META VALUES
103+
104+
> 1) "dtype"
105+
> 2) "FLOAT"
106+
> 3) "shape"
107+
> 4) 1) 4
108+
> 4) 2) 5
109+
> 5) "values"
110+
> 6) 1) 1.1234
111+
> 6) 2) 2.5135
112+
> 6) 3) 5.5425
113+
> 6) 4) 4.1524
58114
```
59115

60116
!!! warning "Overhead of `AI.TENSORGET` with the optional arg VALUES"
@@ -115,15 +171,46 @@ AI.MODELSET resnet18 TF CPU BATCHSIZE 10 MINBATCHSIZE 6 INPUTS in1 OUTPUTS linea
115171
Get model metadata and optionally its binary blob.
116172

117173
```sql
118-
AI.MODELGET model_key [META | BLOB]
174+
AI.MODELGET model_key [META] [BLOB]
119175
```
120176

121177
* model_key - Key for the model
122-
* META - Only return information on backend, device and tag
123-
* BLOB - Return information on backend, device and tag, as well as a binary blob containing the serialized model
178+
* META - Return information on backend, device and tag
179+
* BLOB - Return binary blob containing the serialized model
180+
181+
If META is specified, command returns a list of key-value strings, namely `backend <backend> device <device> tag <tag> [blob <blob>]`.
182+
183+
### AI.MODELGET Example
184+
185+
```sql
186+
AI.MODELGET mnist_net META
187+
188+
> 1) "backend"
189+
> 2) "TORCH"
190+
> 3) "device"
191+
> 4) "CPU"
192+
> 5) "tag"
193+
> 6) "mnist"
194+
```
124195

125-
The command returns a list of key-value strings, namely `BACKEND backend DEVICE device TAG tag [BLOB blob]`.
196+
```sql
197+
AI.MODELGET mnist_net BLOB
126198

199+
> ...
200+
```
201+
202+
```sql
203+
AI.MODELGET mnist_net META BLOB
204+
205+
> 1) "backend"
206+
> 2) "TORCH"
207+
> 3) "device"
208+
> 4) "CPU"
209+
> 5) "tag"
210+
> 6) "mnist"
211+
> 7) "blob"
212+
> 8) ...
213+
```
127214

128215
## AI.MODELDEL
129216

@@ -165,23 +252,23 @@ AI.MODELRUN resnet18 INPUTS image12 OUTPUTS label12
165252
The execution of models will generate intermediate tensors that are not allocated by the Redis allocator, but by whatever allocator is used in the backends (which may act on main memory or GPU memory, depending on the device), thus not being limited by maxmemory settings on Redis.
166253
---
167254

168-
## AI._MODELLIST
255+
## AI._MODELSCAN
169256

170-
NOTE: `_MODELLIST` is EXPERIMENTAL and might be removed in future versions.
257+
NOTE: `_MODELSCAN` is EXPERIMENTAL and might be removed in future versions.
171258

172259
List all models. Returns a list of (key, tag) pairs.
173260

174261
```sql
175-
AI._MODELLIST
262+
AI._MODELSCAN
176263
```
177264

178-
### _MODELLIST Example
265+
### _MODELSCAN Example
179266

180267
```sql
181268
AI.MODELSET model1 TORCH GPU TAG resnet18:v1 < foo.pt
182269
AI.MODELSET model2 TORCH GPU TAG resnet18:v2 < bar.pt
183270

184-
AI._MODELLIST
271+
AI._MODELSCAN
185272

186273
> 1) 1) "model1"
187274
> 1) 2) "resnet18:v1"
@@ -227,12 +314,43 @@ AI.SCRIPTSET addscript GPU TAG myscript:v0.1 < addtwo.txt
227314
Get script metadata and source.
228315

229316
```sql
230-
AI.SCRIPTGET script_key
317+
AI.SCRIPTGET script_key [META] [SOURCE]
231318
```
232319

233320
* script_key - key for the script
321+
* META - Return information on backend, device and tag
322+
* SOURCE - Return string containing the source code for the script
323+
324+
The command returns a list of key-value strings, namely `device <device> tag <tag> [source <source>]`.
325+
326+
### SCRIPTGET Example
327+
328+
```sql
329+
AI.SCRIPTGET addtwo META
330+
331+
> 1) "device"
332+
> 2) "CPU"
333+
> 3) "tag"
334+
> 4) "v1.0"
335+
```
336+
337+
```sql
338+
AI.SCRIPTGET addtwo SOURCE
339+
340+
> ...
341+
```
342+
343+
```sql
344+
AI.SCRIPTGET addtwo META SOURCE
345+
346+
> 3) "device"
347+
> 4) "CPU"
348+
> 5) "tag"
349+
> 6) "v1.0"
350+
> 7) "source"
351+
> 8) ...
352+
```
234353

235-
The command returns a list of key-value strings, namely `DEVICE device TAG tag [SOURCE source]`.
236354

237355
## AI.SCRIPTDEL
238356

@@ -247,12 +365,6 @@ AI.SCRIPTDEL script_key
247365
Currently, the command is fully equivalent to calling `DEL` on `script_key`.
248366

249367

250-
### SCRIPTGET Example
251-
252-
```sql
253-
AI.SCRIPTGET addscript
254-
```
255-
256368
## AI.SCRIPTRUN
257369

258370
Run a script.
@@ -279,23 +391,23 @@ AI.SCRIPTRUN addscript addtwo INPUTS a b OUTPUTS c
279391
The execution of models will generate intermediate tensors that are not allocated by the Redis allocator, but by whatever allocator is used in the backends (which may act on main memory or GPU memory, depending on the device), thus not being limited by maxmemory settings on Redis.
280392
---
281393

282-
## AI._SCRIPTLIST
394+
## AI._SCRIPTSCAN
283395

284-
NOTE: `_SCRIPTLIST` is EXPERIMENTAL and might be removed in future versions.
396+
NOTE: `_SCRIPTSCAN` is EXPERIMENTAL and might be removed in future versions.
285397

286398
List all scripts. Returns a list of (key, tag) pairs.
287399

288400
```sql
289-
AI._SCRIPTLIST
401+
AI._SCRIPTSCAN
290402
```
291403

292-
### _SCRIPTLIST Example
404+
### _SCRIPTSCAN Example
293405

294406
```sql
295407
AI.SCRIPTSET script1 GPU TAG addtwo:v0.1 < addtwo.txt
296408
AI.SCRIPTSET script2 GPU TAG addtwo:v0.2 < addtwo.txt
297409

298-
AI._SCRIPTLIST
410+
AI._SCRIPTSCAN
299411

300412
> 1) 1) "script1"
301413
> 1) 2) "addtwo:v0.1"
@@ -312,14 +424,14 @@ At each `MODELRUN` or `SCRIPTRUN`, RedisAI will collect statistcs specific for e
312424
specific for the node (hence nodes in a cluster will have to be queried individually for their info).
313425
The following information is collected:
314426

315-
- `KEY`: the key being run
316-
- `TYPE`: either `MODEL` or `SCRIPT`
317-
- `BACKEND`: the type of backend (always `TORCH` for `SCRIPT`)
318-
- `DEVICE`: the device where the run has been executed
319-
- `DURATION`: cumulative duration in microseconds
320-
- `SAMPLES`: cumulative number of samples obtained from the 0-th (batch) dimension (for `MODEL` only)
321-
- `CALLS`: number of calls
322-
- `ERRORS`: number of errors generated after the run has been submitted (i.e. excluding errors generated during parsing of the command)
427+
- `key`: the key being run
428+
- `type`: either `MODEL` or `SCRIPT`
429+
- `backend`: the type of backend (always `TORCH` for `SCRIPT`)
430+
- `device`: the device where the run has been executed
431+
- `duration`: cumulative duration in microseconds
432+
- `samples`: cumulative number of samples obtained from the 0-th (batch) dimension (for `MODEL` only)
433+
- `calls`: number of calls
434+
- `errors`: number of errors generated after the run has been submitted (i.e. excluding errors generated during parsing of the command)
323435

324436
```sql
325437
AI.INFO <model_or_script_key>
@@ -338,21 +450,21 @@ The command can be called on a key until that key is removed using `MODELDEL` or
338450
```sql
339451
AI.INFO amodel
340452

341-
> 1) KEY
453+
> 1) key
342454
> 2) "amodel"
343-
> 3) TYPE
455+
> 3) type
344456
> 4) MODEL
345-
> 5) BACKEND
457+
> 5) backend
346458
> 6) TORCH
347-
> 7) DEVICE
459+
> 7) device
348460
> 8) CPU
349-
> 9) DURATION
461+
> 9) duration
350462
> 10) (integer) 6511
351-
> 11) SAMPLES
463+
> 11) samples
352464
> 12) (integer) 2
353-
> 13) CALLS
465+
> 13) calls
354466
> 14) (integer) 1
355-
> 15) ERRORS
467+
> 15) errors
356468
> 16) (integer) 0
357469
```
358470

@@ -363,20 +475,20 @@ AI.INFO amodel RESETSTAT
363475

364476
AI.INFO amodel
365477

366-
> 1) KEY
478+
> 1) key
367479
> 2) "amodel"
368-
> 3) TYPE
480+
> 3) type
369481
> 4) MODEL
370-
> 5) BACKEND
482+
> 5) backend
371483
> 6) TORCH
372-
> 7) DEVICE
484+
> 7) device
373485
> 8) CPU
374-
> 9) DURATION
486+
> 9) duration
375487
> 10) (integer) 0
376-
> 11) SAMPLES
488+
> 11) samples
377489
> 12) (integer) 0
378-
> 13) CALLS
490+
> 13) calls
379491
> 14) (integer) 0
380-
> 15) ERRORS
492+
> 15) errors
381493
> 16) (integer) 0
382494
```

0 commit comments

Comments
 (0)