All tensors initially have the dimensions of the MNIST dataset regardless if they actually have those dimension.