Kernel panic caused by bcm2835_spi_interupt #5048

wbb12 · 2022-05-26T08:01:33Z

Describe the bug

When I use the Raspberry Pi 4B with raspberry pi OS 64 bits, the bcm2835_spi_interupt always causes the kernel to panic. Two SPIs are used on the Pi, spi0 for mcp2515 and spi6 for ADS1263. In the beginning, two SPI devices work correctly, but the kernel panic always occurs after several minutes or several hours. Actually, I installed the ubuntu server 22.04 64bits on another Pi and it shows the same problem.
However, if only one SPI device is connected to the Pi, it will not cause the kernel panic. And if the Pi runs 32 bits OS with the two SPIs, the kernel panic will not occurs either.

Steps to reproduce the behaviour

This bug is related to the spi hardware. It may be difficult to reproduce it.
mcp2515 is a RS485 CAN HAT from waveshare.net. By editing the config.txt to enable the mcp2515 driver.
dtparam=spi=on dtoverlay=mcp2515-can0,oscillator=12000000,interrupt=25,spimaxfrequency=2000000
Python can package is used to read the can messages.

ADS1263 is High-Precision AD HAT from waveshare.net. To use both the two SPI devices in one single Pi, the AD HAT is connected to the SPI1 pins. however, SPI1 does not support mode 1, and SPI6 whose pins are identical to SPI1 is enabled.
Python spidev and RPI.gpio package are used to read the AD data.

When the python script starts running, both devices are working properly. I am sure the hardware is all right.

Device (s)

Raspberry Pi 4 Mod. B

System

$ cat /etc/rpi-issue
Raspberry Pi reference 2022-04-04
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 27a8050c3c06e567c794620394a8c2d74262a516, stage2

$ vcgencmd version
Mar 24 2022 13:19:26
Copyright (c) 2012 Broadcom
version e5a963efa66a1974127860b42e913d2374139ff5 (clean) (release) (start)

$uname -a
Linux rpi-test2 5.15.32-v8+ #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022 aarch64 GNU/Linux

another ubuntu server 64bit OS:
$ vcgencmd version
Nov 18 2021 16:16:49
Copyright (c) 2012 Broadcom
version d9b293558b4cef6aabedcc53c178e7604de90788 (clean) (release) (start)

$uname -a
Linux rpi-test1 5.15.0-1008-raspi #8-Ubuntu SMP PREEMPT Thu May 19 11:58:51 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

Logs

These are the messages on the monitor.

Additional context

No response

The text was updated successfully, but these errors were encountered:

boe-pi · 2022-06-03T09:33:35Z

We had nearly the same problem using 2 CANFD-HATS.
One on SPI0 one on SPI6.
The thing is the controllers share one interrupt and the interrupt service routine does not care about this.
After inserting following lines at the begin of the interrupt service routine bcm2835_spi_interrupt every thing works fine.

if ( !(cs & BCM2835_SPI_CS_INTR) )
return IRQ_NONE;

static irqreturn_t bcm2835_spi_interrupt(int irq, void *dev_id)
{
struct bcm2835_spi *bs = dev_id;
u32 cs = bcm2835_rd(bs, BCM2835_SPI_CS);

if ( !(cs & BCM2835_SPI_CS_INTR) )
	return IRQ_NONE;

Hope this helps and this can be verified.

pelwell · 2022-06-03T11:25:07Z

Thanks - that looks very plausible.

wbb12 · 2022-06-06T06:11:56Z

I add the following lines to spi-bcm2835.c and compile the spi-bcm2835 driver module. After replacing the files of spi-bcm2835aux.ko and spi-bcm2835.ko, the kernel panic no longer occurs.
Thank you @boe-pi for your suggestion.

if ( !(cs & BCM2835_SPI_CS_INTR) ) 
	return IRQ_NONE;

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: #5048 Signed-off-by: Phil Elwell <[email protected]>

pelwell · 2022-06-06T16:39:58Z

This is now merged into rpi-5.15.y - see 46feb7d. It will be added to the newer branches soon.

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: #5048 Signed-off-by: Phil Elwell <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: raspberrypi#5048 Signed-off-by: Phil Elwell <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: #5048 Signed-off-by: Phil Elwell <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: raspberrypi#5048 Signed-off-by: Phil Elwell <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: #5048 Signed-off-by: Phil Elwell <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: raspberrypi#5048 Signed-off-by: Phil Elwell <[email protected]> (cherry picked from commit 46feb7d) Signed-off-by: Marc Kleine-Budde <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Signed-off-by: Martin Sperl <[email protected]> Link: raspberrypi/linux#5048 Suggested-by: https://github.com/boe-pi Co-developed-by: Phil Elwell <[email protected]> Signed-off-by: Phil Elwell <[email protected]> Cc: Mark Brown <[email protected]> Cc: Lukas Wunner <[email protected]> Signed-off-by: Marc Kleine-Budde <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Mark Brown <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: #5048 Signed-off-by: Phil Elwell <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: raspberrypi#5048 Signed-off-by: Phil Elwell <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: #5048 Signed-off-by: Phil Elwell <[email protected]>

BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: raspberrypi#5048 Signed-off-by: Phil Elwell <[email protected]>

commit 46feb7d7241b3f568910a054c1c39a41ab515477 from https://github.com/raspberrypi/linux.git rpi-5.15.y BCM2711 shares an interrupt betweem 5 SPI interfaces (0, 3, 4, 5 & 6). Another interrupt is shared between SPI1, SPI2 and UART1, which also affects BCM2835/6/7. Acting on an interrupt intended for another interface ought to be harmless (although potentially inefficient), but it can cause this driver to crash - presumably because some critical state is not ready. Add a test to the spi-bcm2835 interrupt service routine that interrupts are enabled on this interface to avoid the crash and improve efficiency. Suggested by GitHub user boe-pi. See: raspberrypi/linux#5048 Signed-off-by: Phil Elwell <[email protected]> Signed-off-by: Meng Li <[email protected]>