rpi-update 9a25ccebfd9e2ff9116af665e782b3dba1d8a201 breaks WiFi on 3A+

[DougieLawson]

Describe the bug
WiFi breaks

[   25.845144] Unable to handle kernel paging request at virtual address dead000000000122
[   25.859068] Mem abort info:
[   25.867989]   ESR = 0x96000044
[   25.875694]   EC = 0x25: DABT (current EL), IL = 32 bits
[   25.885705]   SET = 0, FnV = 0
[   25.893330]   EA = 0, S1PTW = 0
[   25.901011] Data abort info:
[   25.908237]   ISV = 0, ISS = 0x00000044
[   25.916542]   CM = 0, WnR = 1
[   25.923816] [dead000000000122] address between user and kernel address ranges
[   25.935598] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[   25.945710] Modules linked in: 8021q garp stp llc mcp320x industrialio ch341 usbserial sg brcmfmac(-) brcmutil cfg80211 rfkill raspberrypi_hwmon i2c_bcm2835 bcm2835_codec(C) bcm2835_v4l2(C) bcm2835_isp(C) bcm2835_mmal_vchiq(C) v4l2_mem2mem videobuf2_vmalloc videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common spi_bcm2835 videodev mc vc_sm_cma(C) uio_pdrv_genirq uio spidev i2c_dev drm fuse drm_panel_orientation_quirks backlight ip_tables x_tables ipv6
[   26.003458] CPU: 3 PID: 534 Comm: rmmod Tainted: G         C        5.10.76-v8+ #1477
[   26.016790] Hardware name: Raspberry Pi 3 Model A Plus Rev 1.0 (DT)
[   26.028598] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO BTYPE=--)
[   26.040262] pc : __cfg80211_unregister_wdev+0x60/0x160 [cfg80211]
[   26.052128] lr : __cfg80211_unregister_wdev+0x50/0x160 [cfg80211]
[   26.063847] sp : ffffffc011f93b50
[   26.072557] x29: ffffffc011f93b50 x28: ffffff8007605ac0
[   26.083405] x27: 0000000000000000 x26: 0000000000000000
[   26.094286] x25: ffffff8006b6c7c8 x24: 0000000000000000
[   26.105047] x23: 0000000000000001 x22: 0000000000000001
[   26.115915] x21: ffffff8006b6c000 x20: ffffff8006b6c300
[   26.126759] x19: ffffff8006b8e008 x18: 0000000000000001
[   26.137579] x17: 0000000000000000 x16: 0000000000000000
[   26.148389] x15: ffffffffffffffff x14: 002e000800007e62
[   26.159076] x13: 6eeb27ba0006000a x12: 0000000000000002
[   26.169835] x11: 0099000c0000000a x10: 0005000800000000
[   26.180587] x9 : ffffffc0109a79cc x8 : 0000000000000000
[   26.191204] x7 : ffffffc011410e88 x6 : ffffffc011f93950
[   26.201912] x5 : 0000000000000000 x4 : ffffff8007605ac0
[   26.212581] x3 : 0000000000000001 x2 : dead000000000122
[   26.223131] x1 : ffffff8002b76018 x0 : dead000000000122
[   26.233767] Call trace:
[   26.241435]  __cfg80211_unregister_wdev+0x60/0x160 [cfg80211]
[   26.252620]  cfg80211_unregister_wdev+0x24/0x38 [cfg80211]
[   26.263516]  brcmf_p2p_ifp_removed+0x8c/0xb0 [brcmfmac]
[   26.273988]  brcmf_remove_interface+0x18c/0x1e0 [brcmfmac]
[   26.284851]  brcmf_detach+0x88/0x110 [brcmfmac]
[   26.294718]  brcmf_sdio_remove+0x70/0x148 [brcmfmac]
[   26.304947]  brcmf_sdiod_remove+0x2c/0x98 [brcmfmac]
[   26.315206]  brcmf_ops_sdio_remove+0xc4/0x110 [brcmfmac]
[   26.325702]  sdio_bus_remove+0x44/0x150
[   26.334767]  device_release_driver_internal+0x118/0x1e0
[   26.345279]  driver_detach+0x58/0xe8
[   26.354016]  bus_remove_driver+0x60/0xb8
[   26.363117]  driver_unregister+0x38/0x68
[   26.372121]  sdio_unregister_driver+0x2c/0x38
[   26.381707]  brcmf_sdio_exit+0x3c/0x48 [brcmfmac]
[   26.391551]  brcmf_core_exit+0x10/0x688 [brcmfmac]
[   26.401585]  brcmfmac_module_exit+0x10/0x38 [brcmfmac]
[   26.411935]  __arm64_sys_delete_module+0x1a4/0x238
[   26.421841]  el0_svc_common.constprop.3+0xc0/0x1f8
[   26.431805]  do_el0_svc_compat+0x24/0x50
[   26.440678]  el0_svc_compat+0x20/0x30
[   26.449297]  el0_sync_compat_handler+0xe4/0x110
[   26.458740]  el0_sync_compat+0x184/0x1c0
[   26.467623] Code: a9410261 f9000420 d2802442 f2fbd5a2 (f9000001)
[   26.478765] ---[ end trace a39796721f47c1b5 ]---

With 5.10.76-v8+ #1475 SMP PREEMPT Mon Nov 1 15:51:53

To reproduce
rpi-update 9a25ccebfd9e2ff9116af665e782b3dba1d8a201

Expected behaviour
WiFi auto connects to my network.

Actual behaviour
Kernel oops as above.

System
Don't know what raspinfo command is.

• Which model of Raspberry Pi? RPi 3A+
• Which OS and version? Bullseye (early version installed with RonR's script)
• Which firmware version? System reverted to current apt-get level of raspberrypi-bootloader & raspberrypi-kernel (to get it working)
• Which kernel version? 5.10.76-v8+

I get the same error with BRANCH=next 93e98b366f5eae1292819ea2b6c971ce4b2b1269

[mati75]

I confirm.

[pelwell]

Confirm on 3A+ with 64-bit kernel?

[pelwell]

I haven't managed to reproduce this. @DougieLawson In a Forum post you wrote:

Booted OK on one 3A+
Didn't boot on the other

Can you think of a difference between the configuration of those two 3A+s? It's interesting that the brcmfmac module is being exited when the crash occurs, not something that would normally happen.

[DougieLawson]

The 3A+ that worked has a TV HAT. The one that doesn't work has nothing special attached.

Both have been apt-get dist-upgraded to Bullseye (using Ron R's script) and were fully upgraded about a week ago. Fundamentally I can't tell what might be different between each system.

I've got the power management switch set in /etc/rc.local with

iwconfig wlxb827eb6e627e power off

in the one that failed.

[pelwell]

I haven't been able to reproduce - a 3A+ that looked promising I now think to be faulty. I did see a potential problem that might have caused stale pointers to be used if the brcmfmac driver was ever probed twice, but only Zero 2 W should have been affected (there's a patch for it in rpi-5.10.y and rpi-5.15.y all the same).

The kernel log seems to point towards the brcmfmac device being removed from a list twice - dead000000000122 is the constant LIST_POISON2 that is used to invalidate the prev pointer in a list item when it is removed - but I have no explanation of what might be triggering that.

[DougieLawson]

This can close. The system where this failure occurred is now running 5.15.0 af9810ecac78d1e5bc9e55a20d6dcb1302eb5429 with no troubles.

Thank you.