SPI speed calculated wrong on RPi3

[TuxCoder]

Problem

The SPI speed on a RPi3 is currently miscalculated is is 0.6 from the set speed.

This comes from the way how the speed is calculated, over an divider from the core clock.
Currently the SPI speed is fix calculated from 250Mhz coreclock.
This clock changed from RPi2 to RPi3 to 400MHz. -> 250/400 = 5/8 ~ 0.66

As the the core clock can be set in /boot/config.txt ist should be dynamic.

Set SPI speed to 3.2MHz result in a clock signal of 2Mhz.
As a workaround I set the core clock to 250MHz.
I measured everything with an logic analysiere before and after.
Now the spi clock is as set at 3.2MHz.

Testet

Platform is based on https://github.com/drtyhlpr/rpi23-gen-image
Debian Stretch 32bit arm
kernels:

• current main branch
• 4.12.0 rc7

Workaround

add core_freq=250 into /boot/config.txt/

[TuxCoder]

When I set the speed to the 5.12MHz I get my requestet 3.2MHz so it is not a hardware restriction.

Go into the code.

It uses the the clk from the spi master to divided by the requested one.
https://github.com/raspberrypi/linux/blob/rpi-4.9.y/drivers/spi/spi-bcm2835aux.c#L341

This spi master clk is referenced in the dts as BCM2835_CLOCK_VPU which looks right:
https://github.com/raspberrypi/linux/blob/rpi-4.9.y/arch/arm/boot/dts/bcm283x.dtsi#L167

My System says currently.
/sys/kernel/debug/clk/vpu/clk_rate = 400000000
Same for /sys/kernel/debug/clk/aux_spi1/clk_rate

I currently looking into the register from the datasheet, and look what actually is set.
https://www.raspberrypi.org/app/uploads/2012/02/BCM2835-ARM-Peripherals.pdf

[TuxCoder]

Oh there are two spi controllers, I have the bug with spi0.

driver:
https://github.com/raspberrypi/linux/blob/rpi-4.9.y/drivers/spi/spi-bcm2835.c#L554
dts:
https://github.com/raspberrypi/linux/blob/rpi-4.9.y/arch/arm/boot/dts/bcm283x.dtsi#L167

https://www.raspberrypi.org/app/uploads/2012/02/BCM2835-ARM-Peripherals.pdf page 152

[TuxCoder]

After reading through the documentation I'm not quite sure what the problem is.

The code looks good, and every thing gets calculated like in the datasheet.

Thinking: If the measured speed is less than the set one, it calculates the divider for the higher clock speed but the hardware works with a slower one. So probably the false clock is assigned to spi0.

In the schematic on page 152 there is the clock divider from the apb interface which runs an different clock speed. But page 156 says core clock:

Clock Divider
SCLK = Core Clock / CDIV
If CDIV is set to 0, the divisor is 65536. The
divisor must be a power of 2. Odd numbers
rounded down. The maximum SPI clock rate is
of the APB clock.

Now I'm out can't find more.

[pelwell]

This is a known issue - the SPI block derives the SPI clock by dividing the system/core clock, which is the same as the VPU processor clock. As you've described, on a default Pi3 the VPU clock switches from 250MHz to 400Mhz when the ARM goes into Turbo mode, but in order to stop a broken ARM OS from allowing the processor to overheat the clocks are still under the control of the VPU.

This source clock variability affects a number of peripherals - UART1, I2C, SPI and SDHOST. The UART clock cannot be allowed to vary because both ends have to agree on a frequency in order to communicate successfully, so when UART1 is the primary (console) UART we require users to set enable_uart=1, which has the side effect of setting core_freq=250.

The SPI and I2C protocols include explicit clock signals, so provided the frequencies are never too large it should be safe for the clock speed to vary (although there will be bandwidth and latency implications). For this reason we get the Linux drivers to configure the divisors as if for the Turbo frequency (400MHz), so that in non-Turbo mode the resulting clock is slower than expected rather than faster.

The SD card interface also has an explicit clock signal, but a 40% drop in performance at 250MHz is too noticeable to tolerate, so there is code on both sides of the kernel driver/firmware boundary that cooperates to keep derived SD clock as close to the preferred value as possible when entering and leaving Turbo mode.

To summarise, this behaviour is expected and more-or-less by design. Changing the clock handling to be more like the SD clock would require code changes on both sides, including to otherwise-unmodified upstream drivers. The workaround you found is the preferred option, as demonstrated by the UART1 support.

[TuxCoder]

Thank you very much, this explain a lot.
I abuse the SPI bus a bit so the clock should be nearly accurate.

Will work with the core_freq=250 workaround.

[rec]

This might be a "known issue" to you and "expected and more-or-less by design" but it wasn't known to or expected by us and consumed quite a lot of time unnecessarily.

I'm curious as to how you expect developers who need a value for the SPI speed to write correct code, given that there is an API that apparently gives this value and no evidence that there is any problem in using it?

[pelwell]

We are constrained by the fact that the peripherals in question are hard-wired to the core clock, and that we need to be able to reduce the core clock without notifying the ARM.

It's unfortunate that your searching didn't uncover anything of use (#974, http://pi-plates.com/the-problem-with-overclocking-and-the-spi-bus/, ...) but as @TuxCoder acknowledged, using the SPI interface as intended works isn't affected by the changing clock frequency.

[JamesH65]

@pelwell Should we update the SPI page to add some warnings on this?

https://www.raspberrypi.org/documentation/hardware/raspberrypi/spi/README.md

[pelwell]

That whole page could do with updating - we've switched to the upstream SPI driver now (bcm2835_spi) - and the equivalent page for I2C. Which doesn't exist.

[cariad-robert-abel]

Are the comments about the firmware/kernel boundary still valid with the recent changes? Is there a plan to tackle this issue and introduce code to keep track of core clock changes?

[pelwell]

The comments are the same, and the situation is unlikely to change. The problem is that the firmware reserves the right to change the clock speeds without notice, for the safety of the chip. It will not wait until the current SPI transaction has completed, because that could be an arbitrarily long time in the future.

The only way to avoid this problem is to prevent the core clock from changing. The safe solution is to fix it to the minimum, which is what core_freq=250 does - the ARM clock will still change on demand. force_turbo=1 also works, but is actively discouraged on Pi3 and Pi3+ because it also keeps the ARM clocks elevated and gets hot. There is now a more adjustable compromise fix which is to choose an intermediate core frequency and explicitly set both the maximum and minimum to that value, for example:

core_freq=350
core_freq_min=350

But for most applications, core_freq=250will have no noticeable impact on performance.

[wilriker]

I just want to add that on Pi4B this is even worse because the defaults are

core_freq=500
core_freq_min=200

and the SPI frequency is still calculated against 250MHz core clock. This gives an enormous range for actual SPI frequency unless both values above are fixed to the same value.

[jal-frezie]

Getting same issue on a Pi Zero. I also abuse the SPI interfaces to regularly sample a signal that isn't generated in response to the clock output, so this is very much worth knowing!

[wilriker]

Hi @pelwell,

Above you stated:

There is now a more adjustable compromise fix which is to choose an intermediate core frequency and explicitly set both the maximum and minimum to that value, for example:

core_freq=350
core_freq_min=350

What I can't figure out (and have no means to measure) is if in the case with your example where the core clock is fixed to something higher than 250Mhz will then the divisor also be applied to this elevated core clock, i.e. max SPI frequency would then be 350Mhz/2? Or will it still be applied to 250Mhz?

[pelwell]

The divisor will be calculated with respect to the actual core frequency, so the SPI bus speed will be as close as possible (given the restrictions of integer divisors) to the requested frequency.

[jaxxzer]

This source clock variability affects a number of peripherals - UART1, I2C, SPI and SDHOST.

@pelwell Can you explain how to determine exactly which peripheral clocks are derived from the core clock and which have independent clocks? Is UART1 the only UART affected by this? Are each of the SPI clocks derived from the core clock? Is there some document I can access that will contain this information? (I'm using a Raspberry Pi 4)