Skip to content

Document N+1 mitigation techniques in the book #444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MikailBag opened this issue Oct 28, 2019 · 6 comments · Fixed by #1230
Closed

Document N+1 mitigation techniques in the book #444

MikailBag opened this issue Oct 28, 2019 · 6 comments · Fixed by #1230
Assignees
Labels
feature New feature or request k::documentation Related to project documentation
Milestone

Comments

@MikailBag
Copy link

Is your feature request related to a problem? Please describe.
Now, resolver signature looks like:
fn field(&self, /*args*/).
This can easily lead to N + 1 DB query problem. (Or even N**K if query has K levels of depth). To avoid this problem, special libraries should be used, that somehow accumulate queries and merge them into one.

Describe the solution you'd like
(Idea itself is very simple. It is possible that is was already discussed and rejected, but I didn't find such an issue). I propose to allow batched resolvers. Signature now looks smth like:
fn field(self: &[Self], ctx:&Ctx, args: ...)
Now, from N**K requests we go to K (each level will be processed with one resolver execution and one db request).
Describe alternatives you've considered
Alternative: do nothing.
However, I believe batched resolvers provide good enough performance with high flexibility.

@MikailBag MikailBag added the enhancement Improvement of existing features or bugfix label Oct 28, 2019
@tyranron
Copy link
Member

@MikailBag for N + 1 query problem the usual solution is either dataloader or lookahead.

As for proposed solution I'm unsure how well it will fit. Usually, resolvers may contain allowed-resoling-depth checks, authorization-rights check, authentication checks, etc, which all are query-scoped. Resolving multiple queries at the same time, but now with the same context, seems to be less flexible.

@davidpdrsn
Copy link
Contributor

davidpdrsn commented Oct 29, 2019

If you're interested in the lookahead approach you could consider juniper-eager-loading which handles all the boilerplate. We're using it in production at @tonsser and its been working well so far. Full disclosure: I'm the maintainer of the library.

I'm sure once async support has landed that we're gonna experiment with the dataloader approach as well.

@theduke
Copy link
Member

theduke commented Oct 29, 2019

As mentioned by others, the possible approaches here are

  • datalaoder
  • using lookahead
  • uniper-eager-loading

We should really add to the book, since it's a common stumbling block.

@theduke theduke added k::documentation Related to project documentation and removed enhancement Improvement of existing features or bugfix labels Oct 29, 2019
@theduke theduke changed the title Batched field resolvers Document N+1 mitigation techniques in the book Oct 29, 2019
@MikailBag
Copy link
Author

Thanks for the responses!
I will study your proposals (so far juniper-eager-loading looks the most promising).

Regarding book, I would like to see quite complex example, e.g.:

  • Response data are aggregated from DB and in-memory source (e.g. calculated).
  • Application doesn't directly use diesel, it uses some trait, MyAppDbProvider.
  • And without too much glue code :)

@davidpdrsn
Copy link
Contributor

@MikailBag I maintain a complete example app. You can find it here https://github.com/davidpdrsn/graphql-app-example. It does use Diesel directly however.

@jayy-lmao
Copy link
Contributor

jayy-lmao commented Jan 29, 2020

@MikailBag I'd be happy to make any requested changes to my demo repo and provide it as an example.

https://github.com/jayy-lmao/rust-graphql-docker

It currently uses dataloaders and has an emphasis on code structure. I welcome any feedback/critique.

Edit: It uses actix-web, juniper, postgres, dataloader-rs

@tyranron tyranron added this to the 0.16.0 milestone Nov 24, 2023
@tyranron tyranron added the feature New feature or request label Nov 24, 2023
@tyranron tyranron self-assigned this Dec 1, 2023
@tyranron tyranron mentioned this issue Dec 1, 2023
31 tasks
@tyranron tyranron linked a pull request Dec 1, 2023 that will close this issue
31 tasks
tyranron added a commit that referenced this issue Mar 20, 2024
- rework and update existing chapters
- mention correct case transformation for GraphQL enums (#1029)
- document N+1 mitigation techniques and look-ahead features (#234, #444)
- mention all integration crates (#867)
- fix Book links (#679, #974, #1056)
- remove old version of Book (#1168)

Additionally:
- disable `bson`, `url`, `uuid` and `schema-language` Cargo features by default in `juniper` crate

Co-authored-by: Christian Legnitto <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request k::documentation Related to project documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants