Firmware update breaks 2B with USB/SATA HD root filesystem

[DougieLawson]

The latest two commits fail to boot when the rootfs is on a 160GB USB/SATA HD.

9e69b79f484fe005d7eca698962f3ee27fa00aaf broke my 2B
ff93337300068d3cddd7f823bc77c4eb954357c9 boots OK if the root is on /dev/mmcblk0pX
8ddfa1f9eba8a207724644b22255597757dbcb75 books OK with root on /dev/disk/by-uuid/296e7e22-8bb1-4970-8dfd-c24fd4f03346 and is what I'm running right now with the USB/SATA rootfs.

cmdline.txt

dwc_otg.lpm_enable=0 console=tty1 root=UUID=296e7e22-8bb1-4970-8dfd-c24fd4f03346 rootfstype=ext4 elevator=deadline rootwait rootdelay=5

I've tried setting rootdelay to 15, it makes no difference the boot waits forever for the root filesystem to come up ready.

config.txt

dtparam=spi=on
dtparam=i2c_arm=on

max_usb_current=1
initramfs initrd.sda

Current uname -a

Linux apollo 3.18.7-v7+ #757 SMP PREEMPT Sat Feb 21 14:54:05 GMT 2015 armv7l GNU/Linux

/etc/fstab

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
#/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
UUID=296e7e22-8bb1-4970-8dfd-c24fd4f03346 /     ext4    defaults,noatime       0 1
# a swapfile is not a swap partition, so no using swapon|off from here on, use  dphys-swapfile swap[on|off]  for that
10.1.1.230:/i-data/b43a16cf/nfs/home   /home  nfs defaults 0 0
10.1.1.230:/i-data/b43a16cf/nfs/shared /shared nfs defaults 0 0
10.1.1.230:/i-data/b43a16cf/nfs/server /srv nfs defaults 0 0

[popcornmix]

Can you narrow it down to a firmware (start.elf) or kernel change?

[DougieLawson]

I'll give that a go tonight.

[DougieLawson]

I've run the tests. It doesn't matter which kernel7 I try it hangs waiting for the rootfs. So it looks like a firmware bug.

[popcornmix]

Are you saying with latest rpi-update, if you downgrade start.elf/fixup.dat to 8ddfa1f then it works?

[DougieLawson]

Yes. I've now downgraded everything to 8ddfa1f and it's running OK.

[popcornmix]

No, not everything at 8ddfa1f.
I want to know if latest rpi-update with just start.elf and fixup.dat reverted to 8ddfa1f works.

[DougieLawson]

I did the test with the latest kernel and old start.elf and the system booted.
I've reverted everything because there's a bunch of "mismatch" messages turning up in syslog.

That's going to happen when it reboots at 02:00, because I'm currently running a smartctl test against my HD right now.

[ 27.598355] ipv6: disagrees about version of symbol module_layout [ 27.692610] nfsd: disagrees about version of symbol module_layout

[DougieLawson]

I've re-run the tests this morning.
My 2B is now running
Linux apollo 3.18.7-v7+ #759 SMP PREEMPT Thu Feb 26 15:53:40 GMT 2015 armv7l GNU/Linux
But with

-rwxr-xr-x 1 root root    2354 Feb 25 23:48 fixup_cd.dat

-rwxr-xr-x 1 root root    6144 Feb 25 23:48 fixup.dat

-rwxr-xr-x 1 root root    9200 Feb 25 23:48 fixup_x.dat

-rwxr-xr-x 1 root root  555352 Feb 25 23:48 start_cd.elf

-rwxr-xr-x 1 root root 2646744 Feb 25 23:48 start.elf

-rwxr-xr-x 1 root root 3603560 Feb 25 23:48 start_x.elf 

from 8ddfa1f

[popcornmix]

So are you saying with the latest kernel, and gpu firmware from 8ddfa1f that mounting by uuid works?
And any newer versions of gpu firmware, mounting by uuid fails?

Can you report output of

vcgencmd get_config int
vcgencmd get_config str
vcgencmd version
cat /proc/cmdline

for the working and non working cases?

[DougieLawson]

Working case:

vcgencmd get_config int

arm_freq=900
sdram_freq=450
over_voltage_avs=0x1b774
disable_overscan=1
ramfsaddr=-1
disable_l2cache=1
program_serial_random=1
config_hdmi_boost=2
disable_commandline_tags=2
emmc_pll_core=1
hdmi_force_cec_address=65535
ignore_lcd=1
framebuffer_ignore_alpha=1
framebuffer_swap=1
max_usb_current=1
arm_control=0xa5800010
temp_limit=85
force_pwm_open=1
pause_burst_frames=1
avoid_fix_ts=1

vcgencmd get_config str

ramfsfile=initrd.sda
device_tree=-

vcgencmd version

Feb 21 2015 18:40:51
Copyright (c) 2012 Broadcom
version 35719cfaa6b22a5743a7de87ddbae98b1ff37d9b (clean) (release)

cat /proc/cmdline

dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1366 bcm2708_fb.fbheight=768 bcm2709.boardrev=0xa01041 bcm2709.serial=0x5a67d427 smsc95xx.macaddr=B8:27:EB:67:D4:27 bcm2708_fb.fbswap=1 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000 dwc_otg.lpm_enable=0 console=tty1 root=UUID=296e7e22-8bb1-4970-8dfd-c24fd4f03346 rootfstype=ext4 elevator=deadline rootwait rootdelay=5

Broken case

(note I've had to change the cmdline to use a rootfs on the SDCard because otherwise I can't login):

vcgencmd get_config int

arm_control=0xa5800010
arm_freq=900
avoid_fix_ts=1
config_hdmi_boost=2
disable_commandline_tags=2
disable_l2cache=1
disable_overscan=1
emmc_pll_core=1
force_pwm_open=1
framebuffer_ignore_alpha=1
framebuffer_swap=1
hdmi_force_cec_address=65535
ignore_lcd=1
max_usb_current=1
over_voltage_avs=0x1b774
pause_burst_frames=1
program_serial_random=1
sdram_freq=450
temp_limit=85

vcgencmd get_config str

device_tree=-

vcgencmd version

Feb 26 2015 16:02:45
Copyright (c) 2012 Broadcom
version e987975e117c6c63876e726aa1282b4705ab2849 (clean) (release)

cat /proc/cmdline

dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1366 bcm2708_fb.fbheight=768 bcm2709.boardrev=0xa01041 bcm2709.serial=0x5a67d427 smsc95xx.macaddr=B8:27:EB:67:D4:27 bcm2708_fb.fbswap=1 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

[popcornmix]

Thanks. Looks like ramfsfile=initrd.sda is missing, which could explain your symptoms.
I'll investigate and should have a fix soon.

[popcornmix]

Can you try this test firmware:
https://dl.dropboxusercontent.com/u/3669512/temp/firmware_ramfsfile.zip

[DougieLawson]

Thanks. That booted OK.

vcgencmd get_config int

arm_control=0xa5800010
arm_freq=900
avoid_fix_ts=1
config_hdmi_boost=2
disable_commandline_tags=2
disable_l2cache=1
disable_overscan=1
emmc_pll_core=1
force_pwm_open=1
framebuffer_ignore_alpha=1
framebuffer_swap=1
hdmi_force_cec_address=65535
ignore_lcd=1
max_usb_current=1
over_voltage_avs=0x1b774
pause_burst_frames=1
program_serial_random=1
ramfsaddr=-1
sdram_freq=450
temp_limit=85

vcgencmd get_config str

device_tree=-
ramfsfile=initrd.sda

vcgencmd version

Feb 27 2015 13:45:40
Copyright (c) 2012 Broadcom
version ce90d5ce49c21fcc290b019aab735b6299699a33 (clean) (release)

cat /proc/cmdline

dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1366 bcm2708_fb.fbheight=768 bcm2709.boardrev=0xa01041 bcm2709.serial=0x5a67d427 smsc95xx.macaddr=B8:27:EB:67:D4:27 bcm2708_fb.fbswap=1 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000 dwc_otg.lpm_enable=0 console=tty1 root=UUID=296e7e22-8bb1-4970-8dfd-c24fd4f03346 rootfstype=ext4 elevator=deadline rootwait rootdelay=5

[popcornmix]

Can you confirm latest rpi-update firmware fixes the issue.

[DougieLawson]

Thank you very much. That's working just fine right now.

uname -a

Linux apollo 3.18.8-v7+ #761 SMP PREEMPT Fri Feb 27 15:48:09 GMT 2015 armv7l GNU/Linux

vcgencmd get_config int

arm_control=0xa5800010
arm_freq=900
avoid_fix_ts=1
config_hdmi_boost=2
disable_commandline_tags=2
disable_l2cache=1
disable_overscan=1
emmc_pll_core=1
force_pwm_open=1
framebuffer_ignore_alpha=1
framebuffer_swap=1
hdmi_force_cec_address=65535
ignore_lcd=1
max_usb_current=1
over_voltage_avs=0x1b774
pause_burst_frames=1
program_serial_random=1
ramfsaddr=-1
sdram_freq=450
temp_limit=85

vcgencmd get_config str

device_tree=-
ramfsfile=initrd.sda

vcgencmd version

Feb 27 2015 15:31:31
Copyright (c) 2012 Broadcom
version ce90d5ce49c21fcc290b019aab735b6299699a33 (clean) (release)

cat /proc/cmdline

dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1366 bcm2708_fb.fbheight=768 bcm2709.boardrev=0xa01041 bcm2709.serial=0x5a67d427 smsc95xx.macaddr=B8:27:EB:67:D4:27 bcm2708_fb.fbswap=1 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000 dwc_otg.lpm_enable=0 console=tty1 root=UUID=296e7e22-8bb1-4970-8dfd-c24fd4f03346 rootfstype=ext4 elevator=deadline rootwait rootdelay=5

[popcornmix]

Good to hear.