xsk: Intel driver improvements #2442

kernel-patches-bot · 2022-01-21T12:12:36Z

Pull request for series with
subject: xsk: Intel driver improvements
version: 3
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=607245

kernel-patches-bot · 2022-01-21T12:12:37Z

Master branch: 820e6e2
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=607245
version: 3

Remove the likely before napi_complete_done as this is the unlikely case when busy-poll is used. Removing this has a positive performance impact for busy-poll and no negative impact to the regular case. Signed-off-by: Maciej Fijalkowski <[email protected]>

Currently, if ice_clean_rx_irq_zc() processed the whole ring and next_to_use != 0, then ice_alloc_rx_buf_zc() would not refill the whole ring even if the XSK buffer pool would have enough free entries (either from fill ring or the internal recycle mechanism) - it is because ring wrap is not handled. Improve the logic in ice_alloc_rx_buf_zc() to address the problem above. Do not clamp the count of buffers that is passed to xsk_buff_alloc_batch() in case when next_to_use + buffer count >= rx_ring->count, but rather split it and have two calls to the mentioned function - one for the part up until the wrap and one for the part after the wrap. Signed-off-by: Magnus Karlsson <[email protected]> Signed-off-by: Maciej Fijalkowski <[email protected]>

XDP_TX workloads use a concept of Tx threshold that indicates the interval of setting RS bit on descriptors which in turn tells the HW to generate an interrupt to signal the completion of Tx on HW side. It is currently based on a constant value of 32 which might not work out well for various sizes of ring combined with for example batch size that can be set via SO_BUSY_POLL_BUDGET. Internal tests based on AF_XDP showed that most convenient setup of mentioned threshold is when it is equal to quarter of a ring length. Introduce @tx_thresh field in ice_tx_ring struct that will store the value of threshold and use it in favor of ICE_TX_THRESH. Signed-off-by: Maciej Fijalkowski <[email protected]>

Move desc_array from the driver to the pool. The reason behind this is that we can then reuse this array as a temporary storage for descriptors in all zero-copy drivers that use the batched interface. This will make it easier to add batching to more drivers. i40e is the only driver that has a batched Tx zero-copy implementation, so no need to touch any other driver. Signed-off-by: Magnus Karlsson <[email protected]>

Commit 9610bd9 ("ice: optimize XDP_TX workloads") introduced @next_dd and @next_rs to ice_tx_ring struct. Currently, their state is not restored in ice_clean_tx_ring(), which was not causing any troubles as the XDP rings are gone after we're done with XDP prog on interface. For upcoming usage of mentioned fields in AF_XDP, this might expose us to a potential dead Tx side. Scenario would look like following (based on xdpsock): - two xdpsock instances are spawned in Tx mode - one of them is killed - XDP prog is kept on interface due to the other xdpsock still running * this means that XDP rings stayed in place - xdpsock is launched again on same queue id that was terminated on - @next_dd and @next_rs setting is bogus, therefore transmit side is broken To protect us from the above, restore the default @next_rs and @next_dd values when cleaning the Tx ring. Signed-off-by: Maciej Fijalkowski <[email protected]>

Apply the logic that was done for regular XDP from commit 9610bd9 ("ice: optimize XDP_TX workloads") to the ZC side of the driver. On top of that, introduce batching to Tx that is inspired by i40e's implementation with adjustments to the cleaning logic - take into the account NAPI budget in ice_clean_xdp_irq_zc(). Separating the stats structs onto separate cache lines seemed to improve the performance. Signed-off-by: Maciej Fijalkowski <[email protected]>

One of the things that commit 5574ff7 ("i40e: optimize AF_XDP Tx completion path") introduced was the @xdp_tx_active field. Its usage from i40e can be adjusted to ice driver and give us positive performance results. If the descriptor that @next_dd to points has been sent by HW (its DD bit is set), then we are sure that there are ICE_TX_THRESH count of descriptors ready to be cleaned. If @xdp_tx_active is 0 which means that related xdp_ring is not used for XDP_{TX, REDIRECT} workloads, then we know how many XSK entries should placed to completion queue, IOW walking through the ring can be skipped. Signed-off-by: Maciej Fijalkowski <[email protected]>

kernel-patches-bot · 2022-01-21T22:26:06Z

Master branch: a9921ce
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=607245
version: 3

kernel-patches-bot · 2022-01-22T02:19:17Z

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=607245 expired. Closing PR.

…arations) Fix checkpatch code style warnings: WARNING: Missing a blank line after declarations kernel-patches#761: FILE: net/core/pktgen.c:761: + char c; + if (get_user(c, &user_buffer[i])) WARNING: Missing a blank line after declarations kernel-patches#780: FILE: net/core/pktgen.c:780: + char c; + if (get_user(c, &user_buffer[i])) WARNING: Missing a blank line after declarations kernel-patches#806: FILE: net/core/pktgen.c:806: + char c; + if (get_user(c, &user_buffer[i])) WARNING: Missing a blank line after declarations kernel-patches#823: FILE: net/core/pktgen.c:823: + char c; + if (get_user(c, &user_buffer[i])) WARNING: Missing a blank line after declarations kernel-patches#1968: FILE: net/core/pktgen.c:1968: + char f[32]; + memset(f, 0, 32); WARNING: Missing a blank line after declarations kernel-patches#2410: FILE: net/core/pktgen.c:2410: + struct pktgen_net *pn = net_generic(dev_net(pkt_dev->odev), pg_net_id); + if (!x) { WARNING: Missing a blank line after declarations kernel-patches#2442: FILE: net/core/pktgen.c:2442: + __u16 t; + if (pkt_dev->flags & F_QUEUE_MAP_RND) { WARNING: Missing a blank line after declarations kernel-patches#2523: FILE: net/core/pktgen.c:2523: + unsigned int i; + for (i = 0; i < pkt_dev->nr_labels; i++) WARNING: Missing a blank line after declarations kernel-patches#2567: FILE: net/core/pktgen.c:2567: + __u32 t; + if (pkt_dev->flags & F_IPSRC_RND) WARNING: Missing a blank line after declarations kernel-patches#2587: FILE: net/core/pktgen.c:2587: + __be32 s; + if (pkt_dev->flags & F_IPDST_RND) { WARNING: Missing a blank line after declarations kernel-patches#2634: FILE: net/core/pktgen.c:2634: + __u32 t; + if (pkt_dev->flags & F_TXSIZE_RND) { WARNING: Missing a blank line after declarations kernel-patches#2736: FILE: net/core/pktgen.c:2736: + int i; + for (i = 0; i < pkt_dev->cflows; i++) { WARNING: Missing a blank line after declarations kernel-patches#2738: FILE: net/core/pktgen.c:2738: + struct xfrm_state *x = pkt_dev->flows[i].x; + if (x) { WARNING: Missing a blank line after declarations kernel-patches#2752: FILE: net/core/pktgen.c:2752: + int nhead = 0; + if (x) { WARNING: Missing a blank line after declarations kernel-patches#2795: FILE: net/core/pktgen.c:2795: + unsigned int i; + for (i = 0; i < pkt_dev->nr_labels; i++) WARNING: Missing a blank line after declarations kernel-patches#3480: FILE: net/core/pktgen.c:3480: + ktime_t idle_start = ktime_get(); + schedule(); Signed-off-by: Peter Seiderer <[email protected]> Signed-off-by: NipaLocal <nipa@local>

…arations) Fix checkpatch code style warnings: WARNING: Missing a blank line after declarations kernel-patches#761: FILE: net/core/pktgen.c:761: + char c; + if (get_user(c, &user_buffer[i])) WARNING: Missing a blank line after declarations kernel-patches#780: FILE: net/core/pktgen.c:780: + char c; + if (get_user(c, &user_buffer[i])) WARNING: Missing a blank line after declarations kernel-patches#806: FILE: net/core/pktgen.c:806: + char c; + if (get_user(c, &user_buffer[i])) WARNING: Missing a blank line after declarations kernel-patches#823: FILE: net/core/pktgen.c:823: + char c; + if (get_user(c, &user_buffer[i])) WARNING: Missing a blank line after declarations kernel-patches#1968: FILE: net/core/pktgen.c:1968: + char f[32]; + memset(f, 0, 32); WARNING: Missing a blank line after declarations kernel-patches#2410: FILE: net/core/pktgen.c:2410: + struct pktgen_net *pn = net_generic(dev_net(pkt_dev->odev), pg_net_id); + if (!x) { WARNING: Missing a blank line after declarations kernel-patches#2442: FILE: net/core/pktgen.c:2442: + __u16 t; + if (pkt_dev->flags & F_QUEUE_MAP_RND) { WARNING: Missing a blank line after declarations kernel-patches#2523: FILE: net/core/pktgen.c:2523: + unsigned int i; + for (i = 0; i < pkt_dev->nr_labels; i++) WARNING: Missing a blank line after declarations kernel-patches#2567: FILE: net/core/pktgen.c:2567: + __u32 t; + if (pkt_dev->flags & F_IPSRC_RND) WARNING: Missing a blank line after declarations kernel-patches#2587: FILE: net/core/pktgen.c:2587: + __be32 s; + if (pkt_dev->flags & F_IPDST_RND) { WARNING: Missing a blank line after declarations kernel-patches#2634: FILE: net/core/pktgen.c:2634: + __u32 t; + if (pkt_dev->flags & F_TXSIZE_RND) { WARNING: Missing a blank line after declarations kernel-patches#2736: FILE: net/core/pktgen.c:2736: + int i; + for (i = 0; i < pkt_dev->cflows; i++) { WARNING: Missing a blank line after declarations kernel-patches#2738: FILE: net/core/pktgen.c:2738: + struct xfrm_state *x = pkt_dev->flows[i].x; + if (x) { WARNING: Missing a blank line after declarations kernel-patches#2752: FILE: net/core/pktgen.c:2752: + int nhead = 0; + if (x) { WARNING: Missing a blank line after declarations kernel-patches#2795: FILE: net/core/pktgen.c:2795: + unsigned int i; + for (i = 0; i < pkt_dev->nr_labels; i++) WARNING: Missing a blank line after declarations kernel-patches#3480: FILE: net/core/pktgen.c:3480: + ktime_t idle_start = ktime_get(); + schedule(); Signed-off-by: Peter Seiderer <[email protected]> Reviewed-by: Toke Høiland-Jørgensen <[email protected]> Signed-off-by: Jakub Kicinski <[email protected]>

kernel-patches-bot added bpf-next new V3 labels Jan 21, 2022

Nobody and others added 8 commits January 21, 2022 14:26

adding ci files

642f1f4

kernel-patches-bot force-pushed the series/607245=>bpf-next branch from 39edd6a to 02a6994 Compare January 21, 2022 22:26

kernel-patches-bot added changes-requested and removed new labels Jan 22, 2022

kernel-patches-bot closed this Jan 22, 2022

kernel-patches-bot deleted the series/607245=>bpf-next branch January 24, 2022 10:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

xsk: Intel driver improvements #2442

xsk: Intel driver improvements #2442

Uh oh!

kernel-patches-bot commented Jan 21, 2022

Uh oh!

kernel-patches-bot commented Jan 21, 2022

Uh oh!

kernel-patches-bot commented Jan 21, 2022

Uh oh!

kernel-patches-bot commented Jan 22, 2022

Uh oh!

Uh oh!

xsk: Intel driver improvements #2442

xsk: Intel driver improvements #2442

Uh oh!

Conversation

kernel-patches-bot commented Jan 21, 2022

Uh oh!

kernel-patches-bot commented Jan 21, 2022

Uh oh!

kernel-patches-bot commented Jan 21, 2022

Uh oh!

kernel-patches-bot commented Jan 22, 2022

Uh oh!

Uh oh!