Skip to content

predict np.vstack problem when batch_size if no factor of test dataset size #288

@leftluoyi

Description

@leftluoyi

Have a trained network. The test set includes 28000 rows. If I set the batch_size to 256, i.e.,

prediction = tl.utils.predict(sess, network, test_X, X, y_op, batch_size=256)

the following exception is thrown:

Traceback (most recent call last):
  File "D:/Project/Python/kaggle/mnist/run.py", line 136, in <module>
    prediction = tl.utils.predict(sess, network, test_X, X, y_op, batch_size=256)
  File "D:\Anaconda3\lib\site-packages\tensorlayer\utils.py", line 296, in predict
    result = np.vstack((result, result_a))
  File "D:\Anaconda3\lib\site-packages\numpy\core\shape_base.py", line 234, in vstack
    return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
ValueError: all the input array dimensions except for the concatenation axis must match exactly

If set the batch_size to a factor of 28000, like 400. the problem goes away.

Looking into the utils.py, the exception is caused by the fact that tl.utils.predict tries to vstack the results from each batch. In my case, predict tries to divide the 28000 results into groups of 256, which left 96 extra results. When stacking the 96 results which has shape [1,96] with the other results with shape `[109,256]', the issue happens.

I think this issue this is important because some time the number test set is prime number.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions