Skip to content

Fixed #14, implemented prepared statement #219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 15, 2018
Merged

Fixed #14, implemented prepared statement #219

merged 2 commits into from
May 15, 2018

Conversation

fantix
Copy link
Member

@fantix fantix commented May 4, 2018

Anyone feels like reviewing this one? It's a bit hacky, but works at least. It is really a pain that SQLAlchemy does not support prepared statement at all.

@coveralls
Copy link

Pull Request Test Coverage Report for Build 818

  • 127 of 133 (95.49%) changed or added relevant lines in 5 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.1%) to 97.96%

Changes Missing Coverage Covered Lines Changed/Added Lines %
gino/dialects/base.py 72 78 92.31%
Totals Coverage Status
Change from base Build 816: -0.1%
Covered Lines: 3458
Relevant Lines: 3530

💛 - Coveralls

@coveralls
Copy link

coveralls commented May 4, 2018

Pull Request Test Coverage Report for Build 833

  • 135 of 140 (96.43%) changed or added relevant lines in 5 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.07%) to 97.993%

Changes Missing Coverage Covered Lines Changed/Added Lines %
gino/dialects/base.py 75 80 93.75%
Totals Coverage Status
Change from base Build 831: -0.07%
Covered Lines: 3466
Relevant Lines: 3537

💛 - Coveralls

Copy link
Member

@wwwjfy wwwjfy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bravo

params = []
for val in ctx.parameters[0]:
if asyncio.iscoroutine(val):
val = await val
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be taken care at call sites?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Urh yeah you're probably right about it. It was a necessary await for _ResultProxy.execute() because it is a workaround for the schema generator. But no one should have used prepared statement yet, so I'll remove this one.

self.clause, *multiparams, **params).context
if ctx.executemany:
raise ValueError('PreparedStatement does not support multiple '
'parameters.')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is there such limitation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because asyncpg doesn't have it on prepared statement. User who wants to use executemany() with prepared statement has to fall back to sequential all() calls, or not to use prepared statement to save RTT.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx for the info. I looked up, and it seems to be due to PostgreSQL.
(I saw some "workaround" https://stackoverflow.com/a/14903130/238634 but it'd make it too complex)

Copy link
Member Author

@fantix fantix May 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm it's more complicated than I thought.

First of all, current executemany() is not saving RTT, it is only saving some slow Python time: MagicStack/asyncpg#36

And yes, extended query protocol (video) does not support multiple SQL statements, but what we need is actually multiple Bind and Execute commands, ideally issued in a batch. Under the hood prepared statement is actually named statement of extended query, so it should be possible to use prepared statement with executemany(), asyncpg just didn't expose the interface.

So for the first issue, I tried to tweak asyncpg a bit to send Bind and Execute in one packet, it just worked fine. And for the second issue, it won't be hard to add executemany() in prepared statement I suppose (commit).

As for this PR, guess I'll leave as it is now, and note about upstream.

@fantix
Copy link
Member Author

fantix commented May 14, 2018

Thanks for @wwwjfy your comments! I've submitted MagicStack/asyncpg#289 for comments.

@fantix fantix merged commit 815c5c5 into master May 15, 2018
@fantix fantix deleted the t14 branch May 15, 2018 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants