Various BPF helper improvements #122

kernel-patches-bot · 2020-09-28T14:44:32Z

Pull request for series with
subject: Various BPF helper improvements
version: 2
url: https://patchwork.ozlabs.org/project/netdev/list/?series=204609

kernel-patches-bot · 2020-09-28T14:44:33Z

Master branch: ba5f4cf
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

Pull request is NOT updated. Failed to apply https://patchwork.ozlabs.org/project/netdev/list/?series=204609, error message:
Cmd('git') failed due to: exit code(128)
cmdline: git am -3
stderr: 'fatal: previous rebase directory .git/rebase-apply still exists but mbox given.'

kernel-patches-bot · 2020-09-28T16:15:35Z

Master branch: ba5f4cf
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

kernel-patches-bot · 2020-09-28T19:25:46Z

Master branch: 1fd17c8
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

kernel-patches-bot · 2020-09-28T19:57:16Z

Master branch: 09d8ad1
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

kernel-patches-bot · 2020-09-28T21:37:15Z

Master branch: efa90b5
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

kernel-patches-bot · 2020-09-28T23:56:52Z

Master branch: 84a20d8
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

kernel-patches-bot · 2020-09-29T00:27:07Z

Master branch: b000def
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

Similarly to 5a52ae4 ("bpf: Allow to retrieve cgroup v1 classid from v2 hooks"), add a helper to retrieve cgroup v1 classid solely based on the skb->sk, so it can be used as key as part of BPF map lookups out of tc from host ns, in particular given the skb->sk is retained these days when crossing net ns thanks to 9c4c325 ("skbuff: preserve sock reference when scrubbing the skb."). This is similar to bpf_skb_cgroup_id() which implements the same for v2. Kubernetes ecosystem is still operating on v1 however, hence net_cls needs to be used there until this can be dropped in with the v2 helper of bpf_skb_cgroup_id(). Signed-off-by: Daniel Borkmann <[email protected]>

With its use in BPF, the cookie generator can be called very frequently in particular when used out of cgroup v2 hooks (e.g. connect / sendmsg) and attached to the root cgroup, for example, when used in v1/v2 mixed environments. In particular, when there's a high churn on sockets in the system there can be many parallel requests to the bpf_get_socket_cookie() and bpf_get_netns_cookie() helpers which then cause contention on the atomic counter. As similarly done in f991bd2 ("fs: introduce a per-cpu last_ino allocator"), add a small helper library that both can use for the 64 bit counters. Given this can be called from different contexts, we also need to deal with potential nested calls even though in practice they are considered extremely rare. One idea as suggested by Eric Dumazet was to use a reverse counter for this stiuation since we don't expect 64 bit overflows anyways; that way, we can avoid bigger gaps in the 64 bit counter space compared to just batch-wise increase. Even on machines with small number of cores (e.g. 4) the cookie generation shrinks from min/max/med/avg (ns) of 22/50/40/38.9 down to 10/35/14/17.3 when run in parallel from multiple CPUs. Signed-off-by: Daniel Borkmann <[email protected]> Cc: Eric Dumazet <[email protected]>

Add a redirect_neigh() helper as redirect() drop-in replacement for the xmit side. Main idea for the helper is to be very similar in semantics to the latter just that the skb gets injected into the neighboring subsystem in order to let the stack do the work it knows best anyway to populate the L2 addresses of the packet and then hand over to dev_queue_xmit() as redirect() does. This solves two bigger items: i) skbs don't need to go up to the stack on the host facing veth ingress side for traffic egressing the container to achieve the same for populating L2 which also has the huge advantage that ii) the skb->sk won't get orphaned in ip_rcv_core() when entering the IP routing layer on the host stack. Given that skb->sk neither gets orphaned when crossing the netns as per 9c4c325 ("skbuff: preserve sock reference when scrubbing the skb.") the helper can then push the skbs directly to the phys device where FQ scheduler can do its work and TCP stack gets proper backpressure given we hold on to skb->sk as long as skb is still residing in queues. With the helper used in BPF data path to then push the skb to the phys device, I observed a stable/consistent TCP_STREAM improvement on veth devices for traffic going container -> host -> host -> container from ~10Gbps to ~15Gbps for a single stream in my test environment. Signed-off-by: Daniel Borkmann <[email protected]> Cc: David Ahern <[email protected]>

Port of tail_call_static() helper function from Cilium's BPF code base [0] to libbpf, so others can easily consume it as well. We've been using this in production code for some time now. The main idea is that we guarantee that the kernel's BPF infrastructure and JIT (here: x86_64) can patch the JITed BPF insns with direct jumps instead of having to fall back to using expensive retpolines. By using inline asm, we guarantee that the compiler won't merge the call from different paths with potentially different content of r2/r3. We're also using Cilium's __throw_build_bug() macro (here as: __bpf_unreachable()) in different places as a neat trick to trigger compilation errors when compiler does not remove code at compilation time. This works for the BPF back end as it does not implement the __builtin_trap(). [0] cilium/cilium@f5537c2 Signed-off-by: Daniel Borkmann <[email protected]> Cc: Andrii Nakryiko <[email protected]> Acked-by: Andrii Nakryiko <[email protected]>

For those locations where we use an immediate tail call map index use the newly added bpf_tail_call_static() helper. Signed-off-by: Daniel Borkmann <[email protected]>

Add a small test that excercises the new redirect_neigh() helper for the IPv4 and IPv6 case. Signed-off-by: Daniel Borkmann <[email protected]>

kernel-patches-bot · 2020-09-29T00:37:56Z

Master branch: a871b04
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

kernel-patches-bot · 2020-09-29T01:45:50Z

Master branch: 98b972d
series: https://patchwork.ozlabs.org/project/netdev/list/?series=204609
version: 2

Pull request is NOT updated. Failed to apply https://patchwork.ozlabs.org/project/netdev/list/?series=204609
error message:

Cmd('git') failed due to: exit code(128)
  cmdline: git am -3
  stdout: 'Applying: bpf: add classid helper only based on skb->sk
Using index info to reconstruct a base tree...
M	include/uapi/linux/bpf.h
M	tools/include/uapi/linux/bpf.h
Falling back to patching base and 3-way merge...
Auto-merging tools/include/uapi/linux/bpf.h
CONFLICT (content): Merge conflict in tools/include/uapi/linux/bpf.h
Auto-merging include/uapi/linux/bpf.h
CONFLICT (content): Merge conflict in include/uapi/linux/bpf.h
Patch failed at 0001 bpf: add classid helper only based on skb->sk
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".'
  stderr: 'error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch' to see the failed patch'

conflict:

diff --cc include/uapi/linux/bpf.h
index 96ddb00b91dc,88d5d900255c..000000000000
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@@ -3595,49 -3588,14 +3595,60 @@@ union bpf_attr 
   * 	Return
   * 		0 on success, or a negative error in case of failure.
   *
++<<<<<<< HEAD
 + * long bpf_snprintf_btf(char *str, u32 str_size, struct btf_ptr *ptr, u32 btf_ptr_size, u64 flags)
 + *	Description
 + *		Use BTF to store a string representation of *ptr*->ptr in *str*,
 + *		using *ptr*->type_id.  This value should specify the type
 + *		that *ptr*->ptr points to. LLVM __builtin_btf_type_id(type, 1)
 + *		can be used to look up vmlinux BTF type ids. Traversing the
 + *		data structure using BTF, the type information and values are
 + *		stored in the first *str_size* - 1 bytes of *str*.  Safe copy of
 + *		the pointer data is carried out to avoid kernel crashes during
 + *		operation.  Smaller types can use string space on the stack;
 + *		larger programs can use map data to store the string
 + *		representation.
 + *
 + *		The string can be subsequently shared with userspace via
 + *		bpf_perf_event_output() or ring buffer interfaces.
 + *		bpf_trace_printk() is to be avoided as it places too small
 + *		a limit on string size to be useful.
 + *
 + *		*flags* is a combination of
 + *
 + *		**BTF_F_COMPACT**
 + *			no formatting around type information
 + *		**BTF_F_NONAME**
 + *			no struct/union member names/types
 + *		**BTF_F_PTR_RAW**
 + *			show raw (unobfuscated) pointer values;
 + *			equivalent to printk specifier %px.
 + *		**BTF_F_ZERO**
 + *			show zero-valued struct/union members; they
 + *			are not displayed by default
 + *
 + *	Return
 + *		The number of bytes that were written (or would have been
 + *		written if output had to be truncated due to string size),
 + *		or a negative error in cases of failure.
 + *
 + * long bpf_seq_printf_btf(struct seq_file *m, struct btf_ptr *ptr, u32 ptr_size, u64 flags)
 + *	Description
 + *		Use BTF to write to seq_write a string representation of
 + *		*ptr*->ptr, using *ptr*->type_id as per bpf_snprintf_btf().
 + *		*flags* are identical to those used for bpf_snprintf_btf.
 + *	Return
 + *		0 on success or a negative error in case of failure.
++=======
+  * u64 bpf_skb_cgroup_classid(struct sk_buff *skb)
+  * 	Description
+  * 		See **bpf_get_cgroup_classid**\ () for the main description.
+  * 		This helper differs from **bpf_get_cgroup_classid**\ () in that
+  * 		the cgroup v1 net_cls class is retrieved only from the *skb*'s
+  * 		associated socket instead of the current process.
+  * 	Return
+  * 		The id is returned or 0 in case the id could not be retrieved.
++>>>>>>> bpf: add classid helper only based on skb->sk
   */
  #define __BPF_FUNC_MAPPER(FN)		\
  	FN(unspec),			\
@@@ -3789,8 -3747,7 +3800,12 @@@
  	FN(inode_storage_delete),	\
  	FN(d_path),			\
  	FN(copy_from_user),		\
++<<<<<<< HEAD
 +	FN(snprintf_btf),		\
 +	FN(seq_printf_btf),		\
++=======
+ 	FN(skb_cgroup_classid),		\
++>>>>>>> bpf: add classid helper only based on skb->sk
  	/* */
  
  /* integer value in 'imm' field of BPF_CALL instruction selects which helper
diff --cc tools/include/uapi/linux/bpf.h
index 96ddb00b91dc,88d5d900255c..000000000000
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@@ -3595,49 -3588,14 +3595,60 @@@ union bpf_attr 
   * 	Return
   * 		0 on success, or a negative error in case of failure.
   *
++<<<<<<< HEAD
 + * long bpf_snprintf_btf(char *str, u32 str_size, struct btf_ptr *ptr, u32 btf_ptr_size, u64 flags)
 + *	Description
 + *		Use BTF to store a string representation of *ptr*->ptr in *str*,
 + *		using *ptr*->type_id.  This value should specify the type
 + *		that *ptr*->ptr points to. LLVM __builtin_btf_type_id(type, 1)
 + *		can be used to look up vmlinux BTF type ids. Traversing the
 + *		data structure using BTF, the type information and values are
 + *		stored in the first *str_size* - 1 bytes of *str*.  Safe copy of
 + *		the pointer data is carried out to avoid kernel crashes during
 + *		operation.  Smaller types can use string space on the stack;
 + *		larger programs can use map data to store the string
 + *		representation.
 + *
 + *		The string can be subsequently shared with userspace via
 + *		bpf_perf_event_output() or ring buffer interfaces.
 + *		bpf_trace_printk() is to be avoided as it places too small
 + *		a limit on string size to be useful.
 + *
 + *		*flags* is a combination of
 + *
 + *		**BTF_F_COMPACT**
 + *			no formatting around type information
 + *		**BTF_F_NONAME**
 + *			no struct/union member names/types
 + *		**BTF_F_PTR_RAW**
 + *			show raw (unobfuscated) pointer values;
 + *			equivalent to printk specifier %px.
 + *		**BTF_F_ZERO**
 + *			show zero-valued struct/union members; they
 + *			are not displayed by default
 + *
 + *	Return
 + *		The number of bytes that were written (or would have been
 + *		written if output had to be truncated due to string size),
 + *		or a negative error in cases of failure.
 + *
 + * long bpf_seq_printf_btf(struct seq_file *m, struct btf_ptr *ptr, u32 ptr_size, u64 flags)
 + *	Description
 + *		Use BTF to write to seq_write a string representation of
 + *		*ptr*->ptr, using *ptr*->type_id as per bpf_snprintf_btf().
 + *		*flags* are identical to those used for bpf_snprintf_btf.
 + *	Return
 + *		0 on success or a negative error in case of failure.
++=======
+  * u64 bpf_skb_cgroup_classid(struct sk_buff *skb)
+  * 	Description
+  * 		See **bpf_get_cgroup_classid**\ () for the main description.
+  * 		This helper differs from **bpf_get_cgroup_classid**\ () in that
+  * 		the cgroup v1 net_cls class is retrieved only from the *skb*'s
+  * 		associated socket instead of the current process.
+  * 	Return
+  * 		The id is returned or 0 in case the id could not be retrieved.
++>>>>>>> bpf: add classid helper only based on skb->sk
   */
  #define __BPF_FUNC_MAPPER(FN)		\
  	FN(unspec),			\
@@@ -3789,8 -3747,7 +3800,12 @@@
  	FN(inode_storage_delete),	\
  	FN(d_path),			\
  	FN(copy_from_user),		\
++<<<<<<< HEAD
 +	FN(snprintf_btf),		\
 +	FN(seq_printf_btf),		\
++=======
+ 	FN(skb_cgroup_classid),		\
++>>>>>>> bpf: add classid helper only based on skb->sk
  	/* */
  
  /* integer value in 'imm' field of BPF_CALL instruction selects which helper

kernel-patches-bot · 2020-09-29T06:49:45Z

At least one diff in series https://patchwork.ozlabs.org/project/netdev/list/?series=204609 expired. Closing PR.

When configuring TC-MQPRIO offload, only turn off netdev carrier and don't bring physical link down in hardware. Otherwise, when the physical link is brought up again after configuration, it gets re-trained and stalls ongoing traffic. Also, when firmware is no longer accessible or crashed, avoid sending FLOWC and waiting for reply that will never come. Fix following hung_task_timeout_secs trace seen in these cases. INFO: task tc:20807 blocked for more than 122 seconds. Tainted: G S 5.13.0-rc3+ #122 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:tc state:D stack:14768 pid:20807 ppid: 19366 flags:0x00000000 Call Trace: __schedule+0x27b/0x6a0 schedule+0x37/0xa0 schedule_preempt_disabled+0x5/0x10 __mutex_lock.isra.14+0x2a0/0x4a0 ? netlink_lookup+0x120/0x1a0 ? rtnl_fill_ifinfo+0x10f0/0x10f0 __netlink_dump_start+0x70/0x250 rtnetlink_rcv_msg+0x28b/0x380 ? rtnl_fill_ifinfo+0x10f0/0x10f0 ? rtnl_calcit.isra.42+0x120/0x120 netlink_rcv_skb+0x4b/0xf0 netlink_unicast+0x1a0/0x280 netlink_sendmsg+0x216/0x440 sock_sendmsg+0x56/0x60 __sys_sendto+0xe9/0x150 ? handle_mm_fault+0x6d/0x1b0 ? do_user_addr_fault+0x1c5/0x620 __x64_sys_sendto+0x1f/0x30 do_syscall_64+0x3c/0x80 entry_SYSCALL_64_after_hwframe+0x44/0xae RIP: 0033:0x7f7f73218321 RSP: 002b:00007ffd19626208 EFLAGS: 00000246 ORIG_RAX: 000000000000002c RAX: ffffffffffffffda RBX: 000055b7c0a8b240 RCX: 00007f7f73218321 RDX: 0000000000000028 RSI: 00007ffd19626210 RDI: 0000000000000003 RBP: 000055b7c08680ff R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000055b7c085f5f6 R13: 000055b7c085f60a R14: 00007ffd19636470 R15: 00007ffd196262a0 Fixes: b1396c2 ("cxgb4: parse and configure TC-MQPRIO offload") Signed-off-by: Rahul Lakkireddy <[email protected]> Signed-off-by: David S. Miller <[email protected]>

Currently, mlxsw allows cooling states to be set above the maximum cooling state supported by the driver: # cat /sys/class/thermal/thermal_zone2/cdev0/type mlxsw_fan # cat /sys/class/thermal/thermal_zone2/cdev0/max_state 10 # echo 18 > /sys/class/thermal/thermal_zone2/cdev0/cur_state # echo $? 0 This results in out-of-bounds memory accesses when thermal state transition statistics are enabled (CONFIG_THERMAL_STATISTICS=y), as the transition table is accessed with a too large index (state) [1]. According to the thermal maintainer, it is the responsibility of the driver to reject such operations [2]. Therefore, return an error when the state to be set exceeds the maximum cooling state supported by the driver. To avoid dead code, as suggested by the thermal maintainer [3], partially revert commit a421ce0 ("mlxsw: core: Extend cooling device with cooling levels") that tried to interpret these invalid cooling states (above the maximum) in a special way. The cooling levels array is not removed in order to prevent the fans going below 20% PWM, which would cause them to get stuck at 0% PWM. [1] BUG: KASAN: slab-out-of-bounds in thermal_cooling_device_stats_update+0x271/0x290 Read of size 4 at addr ffff8881052f7bf8 by task kworker/0:0/5 CPU: 0 PID: 5 Comm: kworker/0:0 Not tainted 5.15.0-rc3-custom-45935-gce1adf704b14 #122 Hardware name: Mellanox Technologies Ltd. "MSN2410-CB2FO"/"SA000874", BIOS 4.6.5 03/08/2016 Workqueue: events_freezable_power_ thermal_zone_device_check Call Trace: dump_stack_lvl+0x8b/0xb3 print_address_description.constprop.0+0x1f/0x140 kasan_report.cold+0x7f/0x11b thermal_cooling_device_stats_update+0x271/0x290 __thermal_cdev_update+0x15e/0x4e0 thermal_cdev_update+0x9f/0xe0 step_wise_throttle+0x770/0xee0 thermal_zone_device_update+0x3f6/0xdf0 process_one_work+0xa42/0x1770 worker_thread+0x62f/0x13e0 kthread+0x3ee/0x4e0 ret_from_fork+0x1f/0x30 Allocated by task 1: kasan_save_stack+0x1b/0x40 __kasan_kmalloc+0x7c/0x90 thermal_cooling_device_setup_sysfs+0x153/0x2c0 __thermal_cooling_device_register.part.0+0x25b/0x9c0 thermal_cooling_device_register+0xb3/0x100 mlxsw_thermal_init+0x5c5/0x7e0 __mlxsw_core_bus_device_register+0xcb3/0x19c0 mlxsw_core_bus_device_register+0x56/0xb0 mlxsw_pci_probe+0x54f/0x710 local_pci_probe+0xc6/0x170 pci_device_probe+0x2b2/0x4d0 really_probe+0x293/0xd10 __driver_probe_device+0x2af/0x440 driver_probe_device+0x51/0x1e0 __driver_attach+0x21b/0x530 bus_for_each_dev+0x14c/0x1d0 bus_add_driver+0x3ac/0x650 driver_register+0x241/0x3d0 mlxsw_sp_module_init+0xa2/0x174 do_one_initcall+0xee/0x5f0 kernel_init_freeable+0x45a/0x4de kernel_init+0x1f/0x210 ret_from_fork+0x1f/0x30 The buggy address belongs to the object at ffff8881052f7800 which belongs to the cache kmalloc-1k of size 1024 The buggy address is located 1016 bytes inside of 1024-byte region [ffff8881052f7800, ffff8881052f7c00) The buggy address belongs to the page: page:0000000052355272 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x1052f0 head:0000000052355272 order:3 compound_mapcount:0 compound_pincount:0 flags: 0x200000000010200(slab|head|node=0|zone=2) raw: 0200000000010200 ffffea0005034800 0000000300000003 ffff888100041dc0 raw: 0000000000000000 0000000000100010 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff8881052f7a80: 00 00 00 00 00 00 04 fc fc fc fc fc fc fc fc fc ffff8881052f7b00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc >ffff8881052f7b80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ^ ffff8881052f7c00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ffff8881052f7c80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [2] https://lore.kernel.org/linux-pm/[email protected]/ [3] https://lore.kernel.org/linux-pm/[email protected]/ CC: Daniel Lezcano <[email protected]> Fixes: a50c1e3 ("mlxsw: core: Implement thermal zone") Fixes: a421ce0 ("mlxsw: core: Extend cooling device with cooling levels") Signed-off-by: Ido Schimmel <[email protected]> Tested-by: Vadim Pasternak <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Jakub Kicinski <[email protected]>

Like commit 1cf3bfc ("bpf: Support 64-bit pointers to kfuncs") for s390x, add support for 64-bit pointers to kfuncs for LoongArch. Since the infrastructure is already implemented in BPF core, the only thing need to be done is to override bpf_jit_supports_far_kfunc_call(). Before this change, several test_verifier tests failed: # ./test_verifier | grep # | grep FAIL #119/p calls: invalid kfunc call: ptr_to_mem to struct with non-scalar FAIL #120/p calls: invalid kfunc call: ptr_to_mem to struct with nesting depth > 4 FAIL #121/p calls: invalid kfunc call: ptr_to_mem to struct with FAM FAIL #122/p calls: invalid kfunc call: reg->type != PTR_TO_CTX FAIL #123/p calls: invalid kfunc call: void * not allowed in func proto without mem size arg FAIL #124/p calls: trigger reg2btf_ids[reg->type] for reg->type > __BPF_REG_TYPE_MAX FAIL #125/p calls: invalid kfunc call: reg->off must be zero when passed to release kfunc FAIL #126/p calls: invalid kfunc call: don't match first member type when passed to release kfunc FAIL #127/p calls: invalid kfunc call: PTR_TO_BTF_ID with negative offset FAIL #128/p calls: invalid kfunc call: PTR_TO_BTF_ID with variable offset FAIL #129/p calls: invalid kfunc call: referenced arg needs refcounted PTR_TO_BTF_ID FAIL #130/p calls: valid kfunc call: referenced arg needs refcounted PTR_TO_BTF_ID FAIL #486/p map_kptr: ref: reference state created and released on xchg FAIL This is because the kfuncs in the loaded module are far away from __bpf_call_base: ffff800002009440 t bpf_kfunc_call_test_fail1 [bpf_testmod] 9000000002e128d8 T __bpf_call_base The offset relative to __bpf_call_base does NOT fit in s32, which breaks the assumption in BPF core. Enable bpf_jit_supports_far_kfunc_call() lifts this limit. Note that to reproduce the above result, tools/testing/selftests/bpf/config should be applied, and run the test with JIT enabled, unpriv BPF enabled. With this change, the test_verifier tests now all passed: # ./test_verifier ... Summary: 777 PASSED, 0 SKIPPED, 0 FAILED Tested-by: Tiezhu Yang <[email protected]> Signed-off-by: Hengqi Chen <[email protected]> Signed-off-by: Huacai Chen <[email protected]>

With structure layout randomization enabled for 'struct inode' we need to avoid overlapping any of the RCU-used / initialized-only-once members, e.g. i_lru or i_sb_list to not corrupt related list traversals when making use of the rcu_head. For an unlucky structure layout of 'struct inode' we may end up with the following splat when running the ftrace selftests: [<...>] list_del corruption, ffff888103ee2cb0->next (tracefs_inode_cache+0x0/0x4e0 [slab object]) is NULL (prev is tracefs_inode_cache+0x78/0x4e0 [slab object]) [<...>] ------------[ cut here ]------------ [<...>] kernel BUG at lib/list_debug.c:54! [<...>] invalid opcode: 0000 [#1] PREEMPT SMP KASAN [<...>] CPU: 3 PID: 2550 Comm: mount Tainted: G N 6.8.12-grsec+ #122 ed2f536ca62f28b087b90e3cc906a8d25b3ddc65 [<...>] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014 [<...>] RIP: 0010:[<ffffffff84656018>] __list_del_entry_valid_or_report+0x138/0x3e0 [<...>] Code: 48 b8 99 fb 65 f2 ff ff ff ff e9 03 5c d9 fc cc 48 b8 99 fb 65 f2 ff ff ff ff e9 33 5a d9 fc cc 48 b8 99 fb 65 f2 ff ff ff ff <0f> 0b 4c 89 e9 48 89 ea 48 89 ee 48 c7 c7 60 8f dd 89 31 c0 e8 2f [<...>] RSP: 0018:fffffe80416afaf0 EFLAGS: 00010283 [<...>] RAX: 0000000000000098 RBX: ffff888103ee2cb0 RCX: 0000000000000000 [<...>] RDX: ffffffff84655fe8 RSI: ffffffff89dd8b60 RDI: 0000000000000001 [<...>] RBP: ffff888103ee2cb0 R08: 0000000000000001 R09: fffffbd0082d5f25 [<...>] R10: fffffe80416af92f R11: 0000000000000001 R12: fdf99c16731d9b6d [<...>] R13: 0000000000000000 R14: ffff88819ad4b8b8 R15: 0000000000000000 [<...>] RBX: tracefs_inode_cache+0x0/0x4e0 [slab object] [<...>] RDX: __list_del_entry_valid_or_report+0x108/0x3e0 [<...>] RSI: __func__.47+0x4340/0x4400 [<...>] RBP: tracefs_inode_cache+0x0/0x4e0 [slab object] [<...>] RSP: process kstack fffffe80416afaf0+0x7af0/0x8000 [mount 2550 2550] [<...>] R09: kasan shadow of process kstack fffffe80416af928+0x7928/0x8000 [mount 2550 2550] [<...>] R10: process kstack fffffe80416af92f+0x792f/0x8000 [mount 2550 2550] [<...>] R14: tracefs_inode_cache+0x78/0x4e0 [slab object] [<...>] FS: 00006dcb380c1840(0000) GS:ffff8881e0600000(0000) knlGS:0000000000000000 [<...>] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [<...>] CR2: 000076ab72b30e84 CR3: 000000000b088004 CR4: 0000000000360ef0 shadow CR4: 0000000000360ef0 [<...>] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [<...>] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [<...>] ASID: 0003 [<...>] Stack: [<...>] ffffffff818a2315 00000000f5c856ee ffffffff896f1840 ffff888103ee2cb0 [<...>] ffff88812b6b9750 0000000079d714b6 fffffbfff1e9280b ffffffff8f49405f [<...>] 0000000000000001 0000000000000000 ffff888104457280 ffffffff8248b392 [<...>] Call Trace: [<...>] <TASK> [<...>] [<ffffffff818a2315>] ? lock_release+0x175/0x380 fffffe80416afaf0 [<...>] [<ffffffff8248b392>] list_lru_del+0x152/0x740 fffffe80416afb48 [<...>] [<ffffffff8248ba93>] list_lru_del_obj+0x113/0x280 fffffe80416afb88 [<...>] [<ffffffff8940fd19>] ? _atomic_dec_and_lock+0x119/0x200 fffffe80416afb90 [<...>] [<ffffffff8295b244>] iput_final+0x1c4/0x9a0 fffffe80416afbb8 [<...>] [<ffffffff8293a52b>] dentry_unlink_inode+0x44b/0xaa0 fffffe80416afbf8 [<...>] [<ffffffff8293fefc>] __dentry_kill+0x23c/0xf00 fffffe80416afc40 [<...>] [<ffffffff8953a85f>] ? __this_cpu_preempt_check+0x1f/0xa0 fffffe80416afc48 [<...>] [<ffffffff82949ce5>] ? shrink_dentry_list+0x1c5/0x760 fffffe80416afc70 [<...>] [<ffffffff82949b71>] ? shrink_dentry_list+0x51/0x760 fffffe80416afc78 [<...>] [<ffffffff82949da8>] shrink_dentry_list+0x288/0x760 fffffe80416afc80 [<...>] [<ffffffff8294ae75>] shrink_dcache_sb+0x155/0x420 fffffe80416afcc8 [<...>] [<ffffffff8953a7c3>] ? debug_smp_processor_id+0x23/0xa0 fffffe80416afce0 [<...>] [<ffffffff8294ad20>] ? do_one_tree+0x140/0x140 fffffe80416afcf8 [<...>] [<ffffffff82997349>] ? do_remount+0x329/0xa00 fffffe80416afd18 [<...>] [<ffffffff83ebf7a1>] ? security_sb_remount+0x81/0x1c0 fffffe80416afd38 [<...>] [<ffffffff82892096>] reconfigure_super+0x856/0x14e0 fffffe80416afd70 [<...>] [<ffffffff815d1327>] ? ns_capable_common+0xe7/0x2a0 fffffe80416afd90 [<...>] [<ffffffff82997436>] do_remount+0x416/0xa00 fffffe80416afdd0 [<...>] [<ffffffff829b2ba4>] path_mount+0x5c4/0x900 fffffe80416afe28 [<...>] [<ffffffff829b25e0>] ? finish_automount+0x13a0/0x13a0 fffffe80416afe60 [<...>] [<ffffffff82903812>] ? user_path_at_empty+0xb2/0x140 fffffe80416afe88 [<...>] [<ffffffff829b2ff5>] do_mount+0x115/0x1c0 fffffe80416afeb8 [<...>] [<ffffffff829b2ee0>] ? path_mount+0x900/0x900 fffffe80416afed8 [<...>] [<ffffffff8272461c>] ? __kasan_check_write+0x1c/0xa0 fffffe80416afee0 [<...>] [<ffffffff829b31cf>] __do_sys_mount+0x12f/0x280 fffffe80416aff30 [<...>] [<ffffffff829b36cd>] __x64_sys_mount+0xcd/0x2e0 fffffe80416aff70 [<...>] [<ffffffff819f8818>] ? syscall_trace_enter+0x218/0x380 fffffe80416aff88 [<...>] [<ffffffff8111655e>] x64_sys_call+0x5d5e/0x6720 fffffe80416affa8 [<...>] [<ffffffff8952756d>] do_syscall_64+0xcd/0x3c0 fffffe80416affb8 [<...>] [<ffffffff8100119b>] entry_SYSCALL_64_safe_stack+0x4c/0x87 fffffe80416affe8 [<...>] </TASK> [<...>] <PTREGS> [<...>] RIP: 0033:[<00006dcb382ff66a>] vm_area_struct[mount 2550 2550 file 6dcb38225000-6dcb3837e000 22 55(read|exec|mayread|mayexec)]+0x0/0xb8 [userland map] [<...>] Code: 48 8b 0d 29 18 0d 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d f6 17 0d 00 f7 d8 64 89 01 48 [<...>] RSP: 002b:0000763d68192558 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5 [<...>] RAX: ffffffffffffffda RBX: 00006dcb38433264 RCX: 00006dcb382ff66a [<...>] RDX: 000017c3e0d11210 RSI: 000017c3e0d1a5a0 RDI: 000017c3e0d1ae70 [<...>] RBP: 000017c3e0d10fb0 R08: 000017c3e0d11260 R09: 00006dcb383d1be0 [<...>] R10: 000000000020002e R11: 0000000000000246 R12: 0000000000000000 [<...>] R13: 000017c3e0d1ae70 R14: 000017c3e0d11210 R15: 000017c3e0d10fb0 [<...>] RBX: vm_area_struct[mount 2550 2550 file 6dcb38433000-6dcb38434000 5b 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] RCX: vm_area_struct[mount 2550 2550 file 6dcb38225000-6dcb3837e000 22 55(read|exec|mayread|mayexec)]+0x0/0xb8 [userland map] [<...>] RDX: vm_area_struct[mount 2550 2550 anon 17c3e0d0f000-17c3e0d31000 17c3e0d0f 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] RSI: vm_area_struct[mount 2550 2550 anon 17c3e0d0f000-17c3e0d31000 17c3e0d0f 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] RDI: vm_area_struct[mount 2550 2550 anon 17c3e0d0f000-17c3e0d31000 17c3e0d0f 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] RBP: vm_area_struct[mount 2550 2550 anon 17c3e0d0f000-17c3e0d31000 17c3e0d0f 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] RSP: vm_area_struct[mount 2550 2550 anon 763d68173000-763d68195000 7ffffffdd 100133(read|write|mayread|maywrite|growsdown|account)]+0x0/0xb8 [userland map] [<...>] R08: vm_area_struct[mount 2550 2550 anon 17c3e0d0f000-17c3e0d31000 17c3e0d0f 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] R09: vm_area_struct[mount 2550 2550 file 6dcb383d1000-6dcb383d3000 1cd 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] R13: vm_area_struct[mount 2550 2550 anon 17c3e0d0f000-17c3e0d31000 17c3e0d0f 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] R14: vm_area_struct[mount 2550 2550 anon 17c3e0d0f000-17c3e0d31000 17c3e0d0f 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] R15: vm_area_struct[mount 2550 2550 anon 17c3e0d0f000-17c3e0d31000 17c3e0d0f 100033(read|write|mayread|maywrite|account)]+0x0/0xb8 [userland map] [<...>] </PTREGS> [<...>] Modules linked in: [<...>] ---[ end trace 0000000000000000 ]--- The list debug message as well as RBX's symbolic value point out that the object in question was allocated from 'tracefs_inode_cache' and that the list's '->next' member is at offset 0. Dumping the layout of the relevant parts of 'struct tracefs_inode' gives the following: struct tracefs_inode { union { struct inode { struct list_head { struct list_head * next; /* 0 8 */ struct list_head * prev; /* 8 8 */ } i_lru; [...] } vfs_inode; struct callback_head { void (*func)(struct callback_head *); /* 0 8 */ struct callback_head * next; /* 8 8 */ } rcu; }; [...] }; Above shows that 'vfs_inode.i_lru' overlaps with 'rcu' which will destroy the 'i_lru' list as soon as the 'rcu' member gets used, e.g. in call_rcu() or later when calling the RCU callback. This will disturb concurrent list traversals as well as object reuse which assumes these list heads will keep their integrity. For reproduction, the following diff manually overlays 'i_lru' with 'rcu' as, otherwise, one would require some good portion of luck for gambling an unlucky RANDSTRUCT seed: --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -629,6 +629,7 @@ struct inode { umode_t i_mode; unsigned short i_opflags; kuid_t i_uid; + struct list_head i_lru; /* inode LRU list */ kgid_t i_gid; unsigned int i_flags; @@ -690,7 +691,6 @@ struct inode { u16 i_wb_frn_avg_time; u16 i_wb_frn_history; #endif - struct list_head i_lru; /* inode LRU list */ struct list_head i_sb_list; struct list_head i_wb_list; /* backing dev writeback list */ union { The tracefs inode does not need to supply its own RCU delayed destruction of its inode. The inode code itself offers both a "destroy_inode()" callback that gets called when the last reference of the inode is released, and the "free_inode()" which is called after a RCU synchronization period from the "destroy_inode()". The tracefs code can unlink the inode from its list in the destroy_inode() callback, and the simply free it from the free_inode() callback. This should provide the same protection. Link: https://lore.kernel.org/all/[email protected]/ Cc: [email protected] Cc: Masami Hiramatsu <[email protected]> Cc: Mathieu Desnoyers <[email protected]> Cc: Ajay Kaher <[email protected]> Cc: Ilkka =?utf-8?b?TmF1bGFww6TDpA==?= <[email protected]> Link: https://lore.kernel.org/[email protected] Fixes: baa23a8 ("tracefs: Reset permissions on remount if permissions are options") Reported-by: Mathias Krause <[email protected]> Reported-by: Brad Spengler <[email protected]> Suggested-by: Al Viro <[email protected]> Signed-off-by: Steven Rostedt (Google) <[email protected]>

kernel-patches-bot added bpf-next merge-conflict new V2 labels Sep 28, 2020

kernel-patches-bot force-pushed the series/203994=>bpf-next branch from 82df0f6 to 61c377d Compare September 28, 2020 16:15

kernel-patches-bot removed the merge-conflict label Sep 28, 2020

kernel-patches-bot force-pushed the series/203994=>bpf-next branch from 61c377d to 87e9a2b Compare September 28, 2020 19:25

kernel-patches-bot force-pushed the series/203994=>bpf-next branch from 87e9a2b to 7341c24 Compare September 28, 2020 19:57

kernel-patches-bot force-pushed the series/203994=>bpf-next branch from 7341c24 to 7e5125f Compare September 28, 2020 21:37

kernel-patches-bot force-pushed the series/203994=>bpf-next branch from 7e5125f to a04c767 Compare September 28, 2020 23:56

kernel-patches-bot force-pushed the series/203994=>bpf-next branch from a04c767 to 074efc3 Compare September 29, 2020 00:27

kernel-patches-bot and others added 7 commits September 28, 2020 17:37

adding ci files

b7e3d6b

bpf, selftests: use bpf_tail_call_static where appropriate

54a47d1

For those locations where we use an immediate tail call map index use the newly added bpf_tail_call_static() helper. Signed-off-by: Daniel Borkmann <[email protected]>

bpf, selftests: add redirect_neigh selftest

9466981

Add a small test that excercises the new redirect_neigh() helper for the IPv4 and IPv6 case. Signed-off-by: Daniel Borkmann <[email protected]>

kernel-patches-bot force-pushed the series/203994=>bpf-next branch from 074efc3 to 9466981 Compare September 29, 2020 00:37

kernel-patches-bot added the merge-conflict label Sep 29, 2020

kernel-patches-bot added changes-requested and removed merge-conflict new labels Sep 29, 2020

kernel-patches-bot closed this Sep 29, 2020

kernel-patches-bot deleted the series/203994=>bpf-next branch September 30, 2020 16:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Various BPF helper improvements #122

Various BPF helper improvements #122

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 29, 2020

Uh oh!

kernel-patches-bot commented Sep 29, 2020

Uh oh!

kernel-patches-bot commented Sep 29, 2020

Uh oh!

kernel-patches-bot commented Sep 29, 2020

Uh oh!

Uh oh!

Various BPF helper improvements #122

Various BPF helper improvements #122

Uh oh!

Conversation

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 28, 2020

Uh oh!

kernel-patches-bot commented Sep 29, 2020

Uh oh!

kernel-patches-bot commented Sep 29, 2020

Uh oh!

kernel-patches-bot commented Sep 29, 2020

Uh oh!

kernel-patches-bot commented Sep 29, 2020

Uh oh!

Uh oh!