Skip to content

Fixes #19309: N+1 problem on /interfaces, /ip-addresses and /prefixes requests #19304

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Tishka17
Copy link
Contributor

@Tishka17 Tishka17 commented Apr 24, 2025

Fixes: #19309

In case of requesting /interfaces, /ip-addresses and /prefixes some of the fields are selected separately for each model. We should prefetch them.

@jnovinger jnovinger changed the title Fixes N+1 problem on /interfaces, /ip-addresses and /prefixes requests Fixes #19309: N+1 problem on /interfaces, /ip-addresses and /prefixes requests Apr 24, 2025
@jnovinger jnovinger self-requested a review April 24, 2025 17:50
@jnovinger jnovinger self-requested a review April 25, 2025 13:29
Copy link
Member

@jnovinger jnovinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Tishka17 , there's one change I'd like to see for IPAddressViewSet. Other than that, this looks great. Sorry for the lengthy review time!

Interface.objects.select_related("device"),
],
),
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry this took so long. I really wanted to understand why we needed to use the GenericPrefetch here, which meant some poking at things in my local.

So, it turns out that prefetch_related() can handle GFKs perfectly fine on its own, withouth the GenericPrefetch--which I'm guessing you already knew. I was pretty sure they did, but wanted to be sure I understood that correctly. As a matter of fact, prefetch_related() is also prefetching related VMInterface and FHRPGroups that are assigned via the GFK.

The interesting thing here, is what you did with the querysets argument, where you used select_related() on any associated interfaces to cut down any more incidental N+1 issues on that relation. So, I got to digging a bit and realized this is great and we can extend it a bit more, like so:

    queryset = IPAddress.objects.prefetch_related(
        GenericPrefetch(
            "assigned_object",
            [
                FHRPGroup.objects.all(),  # comment about why this is here
                Interface.objects.select_related("cable", "device"),
                VMInterface.objects.select_related("virtual_machine"),
            ],
        ),
    )

There are three changes here I'd like to see:

  1. Add "cable" to the list of .select_related(...) arguments for Interface.
  2. Add a VMInterface queryset with a .select_related("virtual_machine") since that also gets fetch if any IP Address is assigned to a VMInterface.
  3. Add an FHRPGroup queryset added to the list for consistency. It's actually already picked up, but I think it makes sense to list it here since the others are as well so that future code readers aren't confused about why it's missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

N+1 problem in dcim/ipam API calls
2 participants