Serialize concurrent execute and executemany calls

[ods]

Use asyncio.Lock() to serialize concurrent calls to execute and executemany
instead of raising exception.

[1st1]

I'm -1 on this.

asyncpg is low-level and its goal to be as fast as possible. Adding a Lock adds extra overhead and frankly using one connection in multiple concurrent tasks is an antipattern leading to lock contention.

The current solution is to use a lock around asyncpg calls. That also makes it clear that the code uses the same connection from concurrent tasks.

[ods]

Adding a Lock adds extra overhead

What do you mean? Without concurrent tasks the overhead of asyncio.Lock is virtually the same as with _Atomic it replaces: it's just a check for locked flag and loop through empty sequence.

[1st1]

What do you mean? Without concurrent tasks the overhead of asyncio.Lock is virtually the same as with _Atomic it replaces: it's just a check for locked flag and loop through empty sequence.

It also creates a collections.deque() object.

And my point that it's better to use an explicit async with Lock() in high-level code still stands.

[ods]

Adding a Lock adds extra overhead

I've measured the difference for the extreme (worst) case with no query at all:

• Call of empty function: 0.2 usec
• Current _Atomic: 0.8 usec
• async with and modified _Atomic: 1.1 usec (if it were used it would be simpler to redefine context manager in user code)
• asyncio.Lock: 3.0 usec

Code used for benchmark: https://gist.github.com/ods/4fd92af20d7020f0a92d99ab12407887

[vitaly-burovoy]

Also -1 on this.

The Atomic is an assertion that the connection is not going to be used while another query is running.
The high-level code is responsible for managing connection and queuing queries.
If you want to just "send and not mind", use pools, not connections directly.

For your proposal the worst case is sending thousand long queries to the same connection and wait for a long time while they (the last one) return values instead of reusing free connections returned to a pool.