-
Notifications
You must be signed in to change notification settings - Fork 5.2k
apt-get upgrade on Raspian Jessie LITE on RPi3 could cause Kernel Panic after reboot? #1416
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
If you can, could you try adding |
OK there is more of the error, I hope it helps you understand because for me it's stil mysterious :/ here it is said:
So i unplugged the RPi, unplugged usb mouse and keyboard and plugged the RPi again and end up with the same kernel panic. (And I think unplugging all usb devices you want to use in order to make your RPi boot is not a solution.) |
Yes, that absence of librt.so.1 is the immediate cause of the crash. Without it init dies, which is fatal. In order to examine the filesystem you either need an Pi image which loads itself into a RAM disk or a host computer which understands the ext4 filing system (a Linux PC certainly would). I did find this application which claims to be free for personal use: https://www.paragon-software.com/home/extfs-windows/ . The website appears genuine, but I haven't tried the product and can't vouch for it. |
I don't get it... I see here that librt.so.1 is part of libc6 package that upgrade when i do |
Perhaps the files are intact but for some reason the read failed. How are you powering your Pi3? It needs a good power supply - 2.5A or greater. Do you see the red power LED flicker during boot? |
I use a 5v 3.1A power supply, the red LED blinked once then the green one flicked and then red stays on and green turn off |
When you turn on the power the red LED should come on and stay on. If it ever goes off then you have insufficient power - the voltage is dropping below the safe level. I can think of a few possible explanations:
|
Mmmmh... These are more difficult to check but i'm goning to try. |
I had a similar issue this afternoon, but was an "Invalid Elf Header" for libpam.so.0 which is a symlink to libpam.so.0.83.1... This was on a B+ running standard jessie. Fsck and power supply checks got me nowhere. Just putting this comment here since it's an odd coincidence that I also just bumped into such a similar issue within the last few hours. |
I've bought the raspberry power supply, red LED stays on without blinking, but still have that kernel panic after @PeterPablo if you are asking me to do that :
I can't do it because I really don't understand what it's all about :/ But I can try to do what popcornmix said later in the comments in #413 and see if it helps. Thanks! |
Nothing worked, everytime I use apt-get upgrade i got the kernel panic. |
@PeterPablo sorry, I was in a rush and didn't think to save the original file or get a checksum! Hopefully if anyone else comes across this issue, they'll do so. |
Finally I bought a new microSDCard and I was able to use |
@Davidiio - I'd like to take a look at that card. If you send me your name and address - phil at raspberry pi dot org - I will send you a Sandisk 32GB in return for your non-functioning Kingston. |
I have been looking at this for the last few days, using @Davidiio's card. Reproducing the problem, which manifests as file corruption, is easy but pinpointing where it goes wrong is proving difficult. But maybe today will be the day... |
I have the exact same problem with the same SD card. Hopefully you guys are able to sort it out so i dont have to buy a new one. |
A quick update: @Davidiio's card looks like a fake Kingston card. It seems to implement the ERASE command very slowly (0.34s for 1 sector), and my suspicion is that it is erasing more than requested. By hacking the SD driver to not advertise the ERASE capability the card becomes faster at ext4 operations involving deleting data, and the apt-get upgrade doesn't crash. I'd like to understand the failure mode more, then find a way to recognise cards with this fault and disable ERASE support for them. |
@Davidiio can you say where you bought the Kingston card from? |
I got it in a small shop in Brussels where they sell computer parts and other accessories, they usualy sell good stuff and give good advices that's why I went there and the card was in it's packaging that looks perfectly "normal" (I sent it to pelwell with the card.) |
From what I have read, Kingston themselves have not always been good at recognising (or at least acknowledging) fakes. Without another to compare it with, the packaging looked convincing, and the card does at least appear to genuinely be a 15-16GB card. We have a contact at Kingston I've passed the details of the card to, and I'll comment again if I hear back from them. |
I'm having the same issue. Fresh NOOBS installation. apt-get update. Reboot for good measure. apt-get upgrade (or dist-upgrade). Reboot. Panic. Kingston 16 Gb SDC10/16GB 94938-E06.A00LF EDIT: Pi-3 |
@akobelan That looks like exactly the same sort of card as @Davidiio had, which I've been analysing. The card seems to have a fairly major bug in the controller that can cause corruption in neighbouring data. I had suspected that the card was a fake, but a Kingston rep thinks it is genuine but of a new type. I have a kernel patch that should work around it by adding an SD card quirk to say that erase is broken and shouldn't be used, but I've been reluctant to apply it without confirmation that this isn't just a defect in a single card. For this test, please download and flash Jessie Lite instead, then, before putting the card in your Pi, copy this updated kernel7.img over the existing one. You can then boot the Pi, expand the filesystem, apt-get update and apt-get upgrade as usual. This won't overwrite the kernel so the fix will still be in place. Let me know how you get on. |
I'll do as you suggest this afternoon and AK Sent from my abacus
|
Hi Phil, I did as you suggested. The short version, is that the Pi3 booted perfectly. Switched locales to en_CA (UTF-8). Previously, this would fail with a kernel panic. On Fri, May 13, 2016 at 5:55 AM, Phil Elwell [email protected]
|
@pelwell Can you explain what the impact of your changes would be on the day-to-day use of the card? Do I continue using it? Do I look for an alternative? |
The patch is to disable the ERASE functionality, which is mainly a performance enhancement. If we disabled ERASE for all SD cards people probably wouldn't notice. I'd be a bit concerned that if this major error exists, there may be other issues as well, but as long as you don't trust mission-critical data to it you should be OK. I've just pushed the patch to our 4.4 tree, and I'll see about pushing it upstream. Unfortunately this is too late for the latest Raspbian builds, so I wouldn't upgrade without first asking here for a patched kernel. The alternative is that you perform the upgrade then without rebooting follow it with an rpi-update to grab a fixed kernel, although again we haven't built one of those yet. |
New rpi-update firmware has been pushed including this patch, should you want to upgrade to a 4.4 kernel. |
@Berryfier Please post the output of:
|
@pelwell Thank you for giving your time to this. Inputting the command line on my pi with Jessie Lite & the replaced Kernel7.img on the Kingston card, and after errors have been observed, the output is |
What about |
Or just:
|
No sure if it helps, but a cat of some files in /sys/class/mmc_host/mmc0/mmc0:0003 shows |
Yes, that is exactly the information I was looking for. Those details match one of the quirk match rules I added:
These rules appeared in yesterday's rpi-update firmware releases, so if you continue to see a problem after updating then there is another issue. If you want to try the new kernel I suggest you install the recent Jessie Lite release, boot it then rpi-update straight away. Since this is a kernel version bump it shouldn't cause many (any?) deletions on the ext4 partition, so you should get away with it. |
From a clean Jessie Lite you run:
Mine rebooted without complaints, so I hope it will also work for you. |
Thank you for your suggestions. |
I have the same problem with the latest Rasbian Jessie (25-05-2016) and the Kingston sd card SDC10G2-16GB N0579-003.A00LF. I already have the latest version if I try rpi-update. here is the grep output: grep: /sys/class/mmc_host/mmc0/mmc0:0003/block: Is a directory |
The supplier of my card (MyMemory) replaced the problem Kingston card without quibble. The replacement is a Kingston SDC 10G2/16GB 31608-005.A00LF, and has etched print on its rear '16G1602'. The replacement card gives no trouble. |
There was an error in the quirks above - where it says add_quirk_mmc it should have said add_quirk. A patch for that will be in the next release, but until then you can add |
@pelwell Thank you for the quick reply, unfortunately it does not work, I still get the panic. Tried it 2 times from fresh imaging of the sd card, to no avail. If you have other suggestions I am willing to try, meanwhile I will look at getting a replacement card. |
@Driffster have you solved your issue with a New sd card? |
Yes. I reinstalled it on an Kingston GB SDCA/GB N0465-002.A00LF, which is their more expensive version (I wanted another brand but that is all they had at the store I went.. ) I had no issue updating with that card and the Raspberry works fine since. |
@Driffster then please close this issue ;-) Thanks. |
@Ruffio |
This is absolutely repeatable and not an anomaly specific to what any one person is trying. I've had the exact same issue with the exact same steps using a 16GB Kingston sd card. Although I won't go as far as to say my 16GB sd cards are exactly the same as the ones above. However, what I do know this is: On Sunday I did the install and upgrade steps using an 8GB kingston sd card with no problems. Shipped it out. On Monday I repeated same steps with 16GB Kingston card because boss bought a whole bunch of them on sale for $5 each. Ran into the above problems. After much trying of different things I found this thread and had an ahah moment. So I scrounged around the office and found a kingston 32GB card. Did the same install and upgrade steps as before and voila no problem. I knew from the very second step it would work because when I resized the file system the Pi 3 didn't hang for a few seconds after resizing the root file system like it did with the 16GB sd card. So the only question that remains is: Is this a bad card design by Kingston or is it an inherent problem with Raspian/linux/RPi 3? Actually it goes much deeper than that. I've found that with even good power supplies the sd card on the raspberry pi 3 can get corrupted. I now have to use great power supplies. Soon we will be moving to compute modules with emmc memory. |
Did you try it with mmc_block.card_quirks=0x80000000 ? We think that it should already be enabled since Phil is trying to trap your SDCard manufacturer id / card id but it's possible there is a different ID out there. Make sure you add the line to config.txt after imaging the card otherwise it can easily get corrupted again before you even do the rpi-update Gordon |
But its a non-issue for me with respect to this particular 16GB card. Like others have said just avoid this card. Putting in a fix to make a $5 card work is not acceptable UNLESS the problem is not really with the card. In that case do the fix AND buy a new card. Is there any indication at all that this is a system problem and and not a card problem? |
No it is definitely a card problem, but when people go out and buy an SD card and install Raspbian on it and plug it into a Raspberry Pi and it doesn't work they invariably point the finger at Raspberry Pi rather than the card. I'm just trying to ascertain whether the card you have is not triggering the card quirk for some reason. Or if it is a different fault again. Gordon |
Okay thats cool I'm all about tracking stuff down definitively. I have to sleep sometime though.... I'll try it tomorrow (today) or the day after. Thanks for noticing the problem to begin with. And yeah I know what you mean about users. After the power supply fiasco (also bought by the boss) I went back to the store where they sold the Pis and power supplies and told them the "power supplies" they were selling were really USB chargers and they looked at me like I was stupid. |
Hi everybody, i know this thread is old, but I cant find a newer one and not sure if its worse to open a new one. I tried to install a fresh raspbian jessie light image to mi B2+, replaced the mentioned kernel, rpi-update it, reboot was ok. Then try "apt update" and "apt upgrade" without success. Upgrade just takes a long time at 93% removing som "kernalhack" but it finished without errors, However if i reboot the PI, it wont boot at all, the screen flicker a short while but the colored boot picture is missing and no shell would come up. If i reboot the PI then by removing power the colored boot screen is showing forever, still no shell. Maybe you can help me, maybe my card is really broken, i dont know. This is the end of the upgrade output:
This are my card details (4gb Sandisk is printed on it, worked since 2014 without problems until i try to upgrade)
Maybe you can just build me another kernel with fits to my OEMID and i can give it a try. regards |
I'm not aware of any problems with SanDisk cards, but as it says above you can enable the quirk that avoids ERASEs for any card using a kernel command-line parameter. Add the following to /boot/cmdline.txt, keeping all of the text in a single line:
|
@Davidiio is this still an issue? |
In the case that disabling ERASE commands still doesn't work, I'd suggest buying another $5 SD card. Generally we recommend the use of our official accessories (which include 8 and 16GB uSD cards). |
Thanks for posting this important Raspberry "heads-up" as I was tempted to try exactly this! No way. I think it's clearly better to start fresh and format a clean SD. It's more stable & saves the day! Yep. |
I had the same issue, i found that the card had become corrupted. This model differs but take note, it is also a bad model: SDCS/16GB B0658-003.A00LFTS do not trust kingston |
And also i found that the drivers were from 2006 |
I was recently facing this issue on my Pi, event after formatting the SD card. |
I don't know if it is the right place to report that problem but I haven't found any similar issue on forums where I searched except for this one #914 but it doesn't mention the jessie lite version.
I tried to install Raspbian Jessie Lite 2016-03-18 Kernel 4.1 on my Raspberry Pi 3B, after first boot I enter
sudo raspi-config
and expand filesystem then reboot and entersudo apt-get update
andsudo apt-get upgrade
and after that, it always fails to boot again and stop on :if I unplug the RPi and plug it again I end up with same error but different numbers on the left.
I tried to reinstall several times, also used
sudo apt-get install rpi-update
andsudo rpi-update
, it always goes well until I doapt-get upgrade
ordist-upgrade
and I think hardware is ok because the non-lite jessie worked just fine and so did the NOOBS-installed version.The text was updated successfully, but these errors were encountered: