Definition of vecdot is ambiguous in the complex case.

[lezcano]

The function linalg.vecdot is defined as:

Computes the (vector) dot product of two arrays.

It is not clear what this means for complex vectors, as it could be \sum x_i y_i or it could be (as defined for a complex dot product) \sum \conj(x_i) y_i.

I am aware that the API does not cover complex tensors yet, but many of the libraries that are currently implementing this operation do support complex tensors, so it would be great to disambiguate the meaning of this operation now, so that libraries that implement it can safely implement the intended operation for complex tensors.

[kgryte]

@lezcano In pytorch/pytorch#70542, you've implemented the complex dot product as the \sum x_i y_i. Any particular reason for this choice?

As documented in #137, NumPy, in its main namespace, has both non-conjugating APIs and a conjugating API (vdot).

In the OP of #137, I advocated for complex conjugation in vecdot rather than having a separate API, which has the nicety of ensuring that the inner product of a complex vector with itself is real and positive definite.

For external reference, both Julia and MATLAB conjugate. BLAS has two routines: one conjugating (cdotc) and the other non-conjugating (cdotu).

[lezcano]

I just had to choose one and I went for implementing the equivalent of matmul for vectors (as you cannot do this with matmul).

Now, according to the specification, the correct way of implementing this would be as a complex inner product, i.e. x^H y. What the operation that I just implemented means in the complex case is the application of a batch of complex 1-forms on a batch of complex vectors.

In my opinion, given the name, the correct implementation in the complex case should be x^H y. What do you think?

[kgryte]

I'm of the same opinion: x^H y. Perhaps others have different opinions?

[asmeurer]

I just had to choose one and I went for implementing the equivalent of matmul for vectors (as you cannot do this with matmul).

matmul/@ supports 1D array inputs. For higher dimensional arrays you can use mT to align the dimensions for matmul.

[lezcano]

You can indeed replicate the this behaviour resizing the inputs adding a dimension of size 1 in both inputs (see pytorch/pytorch#18027) but this seems fairly clunky. There is no easy way to do this at the moment with the API.

[kgryte]

This issue was discussed during the most recent consortium meeting (1-6-2021). Consensus there was that the definition of the vector dot product for complex number input should conjugate (i.e., x^H y; similar to NumPy's vdot).

Implementors can begin their complex number vecdot implementations assuming that this will be added to the specification along with more general complex number guidance.