MODELRUN results missing output values

[pohjao]

I'm using this ONNX YOLOv3 model including the pre- and post processing functions:
https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/yolov3

If this this model is used with RedisAI it consistently returns only one detection. When I configure the same model using seperate ONNX Runtime instance, multiple detections are returned. I also tried using YOLO TensorFlow model with RedisAI, but the results are still invalid. I also tried with different input images, with no luck.

From the outputs below I can see that RedisAI returns only the first detection. Is the problem a RedisAI bug or my is my implementation faulty?

I'm running latest redisai/redisai docker image. Both ONNX Runtime instances are version 1.2.0

AI.INFO returns 0 errors.

Test image

Preprocessing code

import onnxruntime
import numpy as np
from PIL import Image
from redisai import Client as RaiClient
import redisai

def letterbox_image(image, size):
    iw, ih = image.size
    w, h = size
    scale = min(w/iw, h/ih)
    nw = int(iw*scale)
    nh = int(ih*scale)

    image = image.resize((nw,nh), Image.BICUBIC)
    new_image = Image.new('RGB', size, (128,128,128))
    new_image.paste(image, ((w-nw)//2, (h-nh)//2))
    return new_image

def preprocess(numpy_img):
    img = Image.fromarray(numpy_img)
    model_image_size = (416, 416)
    boxed_image = letterbox_image(img, tuple(reversed(model_image_size)))
    image_data = np.array(boxed_image, dtype='float32')
    image_data /= 255.
    image_data = np.transpose(image_data, [2, 0, 1])
    image_data = np.expand_dims(image_data, 0)
    return image_data

RedisAI implementation

yolo_model = "./yolov3-10.onnx"

with open(yolo_model, 'rb') as f:
    model = f.read()

con = redisai.Client(host='localhost', port=6380)
con.modelset('model', "ONNX", 'cpu', model)

pil_image = Image.open("./dog.jpg")
numpy_img = np.array(pil_image)
numpy_img = preprocess(numpy_img)

side_size = 416
image_size = np.array([side_size, side_size], dtype=np.float32) \
    .reshape(1, 2)

con.tensorset(
    'input_1',
    numpy_img,
    dtype="float32",
    shape=(1, 3, side_size, side_size)
    )

con.tensorset(
    'image_shape',
    image_size,
    dtype="float32",
    shape=(1, 2)
    )

con.modelrun(
    'model',
    inputs=["input_1", "image_shape"],
    outputs=['boxes', 'scores', 'classes']
    )

#classes should contain 3 detections
classes = con.tensorget('classes')
print(classes)

RedisAI outputs

[[ 0 1 253]]

ONNX Runtime

yolo_model = ("./yolov3-10.onnx")
sess = onnxruntime.InferenceSession(yolo_model)

pil_image = Image.open("./dog.jpg")
numpy_img = np.array(pil_image)
numpy_img = preprocess(numpy_img)

side_size = 416
image_size = np.array([side_size, side_size], dtype=np.float32) \
    .reshape(1, 2)
result = sess.run(
    [],
    {
        "input_1": numpy_img,
        "image_shape": image_size
    }
    ) 

classes = result[2]
print(classes)

ONNX Runtime outputs

[[ 0 1 253] [ 0 7 1111] [ 0 16 322]]

[lantiga]

HI @pohjao, it might be an issue with the ONNXRuntime backend. YOLO on TF and TORCH works correctly, but we haven't tested it on ONNXRuntime. It might be that ONNXRuntime returns a tensor list and we don't handle that correctly.
I'm going to look into this shortly, thanks for the report

[lantiga]

Hi @pohjao, I understand now what's going on.

There's an issue with how the non-maximum suppression code is implemented in the model.

Usually in DL models the batch dimension (the 0-th dimension) is preserved from inputs to outputs: if I call a model with a [1, 3, 512, 512] input (batch size of 1), then all outputs will have a value of 1 for the 0-th dimension. In the case of this YOLO model, the non-maximum suppression output (classes) doesn't honor this.

If you run the onnxruntime version with the same input image and print shapes, the input shape is (1, 3, 416, 416) and output shapes are [(1, 10647, 4), (1, 80, 10647), (3, 3)]. The classes output does not preserve batch size (which BTW also means that the model won't work in batched mode).

The reason why you don't see two bounding boxes is that RedisAI interprets the (3, 3) output as a batch of three vectors, so it picks the first one to reply to the request.

Now, since this happened with an official ONNX zoo model we need to find a solution on the RedisAI end. I'll submit a patch shortly.

[lantiga]

@pohjao I just submitted a PR fixing this issue

[pohjao]

Thank you! I checked out the related branch and the issue is fixed indeed.

[lantiga]

Hi @pohjao, we just merged the fix in master