Interface name prefixes breaks natural sorting

[tyler-8]

Environment

• Python version: 3.6.8
• NetBox version: 2.5.5

Steps to Reproduce

1. Create interfaces on a device with names like 1/A1, 1/A2, 1/A3, 1/A10, 1/A20, 1/A30, 2/A1

Expected Behavior

Interfaces are ordered this way:

• 1/A1
• 1/A2
• 1/A3
• 1/A10
• 1/A20
• 1/A30
• 2/A1

Observed Behavior

Without the prefixes, netbox currently sorts interface names correctly. However, when using the prefixes, interfaces are ordered this way:

• 1/A1
• 1/A10
• 1/A2
• 1/A20
• 1/A3
• 1/A30
• 2/A1

Notes

These prefixes are how the network OS names the interfaces when a virtual chassis is in use. The 1/ signifies the first switch, and 2/, 3/, etc. signifies the member device's logical position in the stack.

Related issue: #2165

[jeremystretch]

You can find the code used to effect natural sorting for interfaces here, and the tests used to validate it here. We're employing eight regular expressions as it is to achieve consistent ordering across different platforms. Introducing alphabetic characters within the slot/position/id hierarchy greatly complicates things, because we have to break out the numbers and cast them as integers so that they're ordered correctly.

Adding more regex to break out the alphabetic characters will further complicate an already fairly convoluted approach. We could ignore alphabetic characters in those positions, but that will obviously break if you have e.g. 1/A9 and 1/B1.

One potential workaround which was raised in the NetworkToCode chat would be to prepend the interface IDs with zeros (example: 1/A1 becomes 1/A01). This may or may not work in your particular scenario.

[tyler-8]

Adding more regex to break out the alphabetic characters will further complicate and already fairly convoluted approach.

Completely agreed, and as I went down the rabbit hole today to see how you're doing it now I'm wondering if I just deal with the sort issue for the time being. I'll do some tinkering to see if its viable.

...prepend the interface IDs with zeros (example: 1/A1 becomes 1/A01).

My main concern there is that I'd be deviating from how the device's OS names the interface, which would complicate things as well. In any case, the interfaces are there and it may just be something to deal with in the GUI and handle a custom way through the API.

[DanSheps]

One potential workaround which was raised in the NetworkToCode chat would be to prepend the interface IDs with zeros (example: 1/A1 becomes 1/A01). This may or may not work in your particular scenario.

Would it be possible to zero-pad the integer component of the slot/position/id on the backend? For example, problematically convert A1 to A01 so that they all match in length? I don't see how if you were to stick to straight regex (and the casting you are doing) so my guess is probably not, but I thought I would ask.

[jeremystretch]

@DanSheps We can't do any sort of manipulation since the format is entirely arbitrary and we don't want to risk breaking other naming schemes.

[simonmcconnell]

Cisco is already broken in V2.5.5. It sorts like:

Gi1/1/1
Te1/1/1
Gi1/1/2
Te1/1/2
Gi1/1/3
Gi1/1/4

Where one would expect:
Gi1/1/1
Gi1/1/2
Gi1/1/3
Gi1/1/4
Te1/1/1
Te1/1/2

Looks as though the alpha chars are ignored entirely.

[DanSheps]

Cisco is already broken in V2.5.5. It sorts like:

Gi1/1/1
Te1/1/1
Gi1/1/2
Te1/1/2
Gi1/1/3
Gi1/1/4

Where one would expect:
Gi1/1/1
Gi1/1/2
Gi1/1/3
Gi1/1/4
Te1/1/1
Te1/1/2

Looks as though the alpha chars are ignored entirely.

If you do "TenGigabitEthernet" and "GigabitEthernet" does it sort properly?

[YY-VV]

NetBox version: 2.5.7
Cisco interface sorting broken
It sorts like:
Fa1/0/1  
Gi1/0/1
Fa1/0/2
Gi1/0/2
Fa1/0/3
Gi1/0/3
Fa1/0/4
Gi1/0/4
Fa1/0/5
Where one would expect:
Fa1/0/1  
Fa1/0/2
Fa1/0/3
Fa1/0/4
Fa1/0/5
Gi1/0/1
Gi1/0/2
Gi1/0/3
Gi1/0/4

"TenGigabitEthernet" and "GigabitEthernet" - sorting correctly

[DanSheps]

There is definitely an issue here (confirmed with some test cases), however I don't believe there is an easy fix for your issue with Fa/Gi/Te as it matches on the chassis/slot/port first, then the name and on the newer architecture, modules are properly slotted (for example, the NM-10G-8X is x/1/[1-8])

[jeremystretch]

Closing this out as there's not much we can do about it. Cisco made the unfortunate decision to reuse position indices for different types, which results in a sort of zipper effect when the interfaces are ordered. Previously, this was handled by a field on the DeviceType which informed which type of ordering to use for interfaces (media first or slot/position first), but this was removed because it made it impossible to list interfaces outside the context of a specific device.

Edit: But see the discussion in #3097 around improving natural ordering in general.

[candlerb]

Here is a simple natural ordering algorithm:

• split name into alternating non-numeric (string) and numeric parts. The first part is always non-numeric, but it may be empty string.
• sort left to right, comparing strings in string order, and numbers in number order.

Examples:

• Gi1/0/1 -> ["Gi", 1, "/", 0, "/", 1]
• vlan100 -> ["vlan", 100]
• 1/A1 -> ["", 1, "/A", 1]

>>> ["Gi", 1, "/", 0, "/", 24] < ["Te", 1, "/", 0, "/", 1]
True

Could you give a concrete example of where this sort would not give the expected result?

[jeremystretch]

Try it with the test data.

[candlerb]

OK, I have tried it with the test data. https://gist.github.com/candlerb/7456256993276fb44d1860d9e0927695

As it stands, two tests fail:

• test_interface_ordering_junos wants "xe" to sort before "ae" (why?) If I move all the xe to the end, it's happy.
• test_interface_ordering_ios wants to give priority over the second field (the line card number) rather than the interface type (GigabitEthernet or TenGigabitEthernet)

OK, so that's the fundamental issue here. Certainly on Dell devices I can have Gi1/0/1 and Te1/0/1, and I want the Gi and Te to go into different blocks (meaning: sort on Gi/Te first, then number). But it seems other people want all the "0"s before the "1"s.

I thought that Netbox allowed two different ordering methods to be selected, but I'm having trouble finding it right now (it's not on "device type" anyway). If it does still exist, then the natural ordering algorithm I've given can easily have an option to swap the first two entries, or to move the first entry to the end.

[candlerb]

This one implements the option:
https://gist.github.com/candlerb/a8a7c8c6f3e3215baa83c54ba8c4512a

[jeremystretch]

From above:

Previously, this was handled by a field on the DeviceType which informed which type of ordering to use for interfaces (media first or slot/position first), but this was removed because it made it impossible to list interfaces outside the context of a specific device.

I've spent quite a bit of time on this problem over the years, and frankly what we have now is sufficient: It's simply not feasible to entertain every arbitrary ordering logic. We could maybe add some option to the web UI to toggle type/position sorting on the device view (perhaps piggybacking on #3090), but I think the solution we have now is reasonable as far as the API is concerned.