Kernel panic after Systemupdate (pacman -Syu)

[linarmux]

Hi @all,

first a short introduction (because it's my first post here):
I'm new to linux but in the near future i want to switch from windows to linux. I bought some Raspberry Pi's to get used to (arch) linux before i switch completly.

Now i have a problem after a systemupdate via pacman -Syu both Rasberry Pi's are booting witch kernel panic (see screenshots). ( Yeah i was stupid enough to update both simultaneous *grrrr*! )

a) Raspberry Pi arch linux nginx + owncloud + maria db


b) Raspberry Pi arch linux transmission + samba + vsftpd


Now i need some help to get a working system again, because i really got used to both services (owncloud, transmission)
After boot keyboard doesnt response and i have no network connectivity!

1) What can i do to get a booting system again or to revert all updates ?
2) Most importent Question, because i always thought it's safe and reasonable to keep your system up2date. What can i do to prevent something like this to happen again? I'm really concerned how to handle updates now, what happens when i use linux as my main and only Desktopsystem? Shouldn't i never update again ? What's the "safe-way" to update your system ?

Thanks in advance!

Edit: Image link fixed

[WarheadsSE]

First: you link the thumbnails, and not the full size image. Thanks for the intent, but eerps

http://www.imageshost.de/img/jpg/8404f9 ... 416d4e.jpg
"no init found"

What are the contents of your config files on the boot partition, and when was your last prior update?

(serial consoles tend to work well, 3.3v TTL serial cable)

[linarmux]

Thanks for the fast response!

@First: Sorry my bad ... attachment upload didn't seem to work and i was short on time thats why i rushed the imagehoster upload. It's fixed now!

What config files are importent or may cause the "no init found" error?

There are two "readable" files on the boot partion:

config.txt
$this->bbcode_second_pass_code('', '
# uncomment if you get no picture on HDMI for a default "safe" mode
#hdmi_safe=1

# uncomment this if your display has a black border of unused pixels visible
# and your display can output without overscan
#disable_overscan=1

# uncomment the following to adjust overscan. Use positive numbers if console
# goes off screen, and negative if there is too much border
#overscan_left=16
#overscan_right=16
#overscan_top=16
#overscan_bottom=16

# uncomment to force a console size. By default it will be display's size minus
# overscan.
#framebuffer_width=1280
#framebuffer_height=720

# uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1

# uncomment to force a specific HDMI mode (this will force VGA)
#hdmi_group=1
#hdmi_mode=1

# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
#hdmi_drive=2

# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
#config_hdmi_boost=4

# uncomment for composite PAL
#sdtv_mode=2

#uncomment to overclock the arm. 700 MHz is the default.
#arm_freq=800

# for more options see http://elinux.org/RPi_config.txt

## Some over clocking settings, govenor already set to ondemand

##None
#arm_freq=700
#core_freq=250
#sdram_freq=400
#over_voltage=0

##Modest
#arm_freq=800
#core_freq=300
#sdram_freq=400
#over_voltage=0

##Medium
arm_freq=900
core_freq=333
sdram_freq=450
over_voltage=2

##High
#arm_freq=950
#core_freq=450
#sdram_freq=450
#over_voltage=6

##Turbo
#arm_freq=1000
#core_freq=500
#sdram_freq=500
#over_voltage=6

gpu_mem_512=316
gpu_mem_256=128
cma_lwm=16
cma_hwm=16
cma_offline_start=16
')

and

cmdline.txt
$this->bbcode_second_pass_code('', '
ipv6.disable=1
avoid_safe_mode=1
selinux=0
plymouth.enable=0
smsc95xx.turbo_mode=N
dwc_otg.lpm_enable=0
console=ttyAMA0,115200
kgdboc=ttyAMA0,115200
console=tty1
root=/dev/mmcblk0p5
rootfstype=ext4
elevator=noop rootwait
')

I really don't know when my last update before that crash was, anyway to find this out ?

(sorry i'dont have a serial cable not even a serial port anymore ... and to be totaly honest i really dunno even if i had such cable if it would be very helpful for me! (remeber linux n00b) But good to know that there are options in case i need direct access)

[pepedog]

cmdline.txt should all be on one line, not multiple
However config.txt is multiple lines
sync before reboot

[linarmux]

Whoops *that's my fault i edited the cmdline.txt for better readability. The original file is indeed one line.

original cmdline.txt (without manual linebreak):
$this->bbcode_second_pass_code('', '
ipv6.disable=1 avoid_safe_mode=1 selinux=0 plymouth.enable=0 smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p5 rootfstype=ext4 elevator=noop rootwait
')

But i did'nt sync before reboot because i lost ssh connectivity during update and wasn't aware (at that time) that you should sync after updates.

Unfortunately now it's to late!

[pepedog]

You can manually grab the packages for kernel and firmware
http://us.mirror.archlinuxarm.org/armv6h/
Extract with
tar xf pkgname
And move them about, then sync and umount

[linarmux]

Thanks for replay!

Should i replace the files on the Bootpartition:
$this->bbcode_second_pass_code('', '
29.04.2014 13:20 <DIR> System Volume Information
24.12.2013 01:46 17.824 bootcode.bin
14.11.2013 15:49 221 cmdline.txt
13.08.2013 21:11 195 cmdline.txte
26.12.2013 23:42 1.668 config.txt
24.12.2013 01:46 18.693 COPYING.linux
30.04.2014 11:29 0 dir.txt
24.12.2013 01:46 5.778 fixup.dat
24.12.2013 01:46 2.062 fixup_cd.dat
24.12.2013 01:46 8.826 fixup_x.dat
24.12.2013 02:09 8.655.424 kernel.img
24.12.2013 01:46 9.791.520 kernel_emergency.img
24.12.2013 01:46 1.447 LICENCE.broadcom
24.12.2013 01:46 2.513.844 start.elf
24.12.2013 01:46 480.056 start_cd.elf
24.12.2013 01:46 3.494.980 start_x.elf
')

Kernel.img should be the kernel, right? (strange: modified 24.12.2013?) Which file is the firmware? Isn't it enough to simply overwrite these files?

Is there a way to find out the old and working Kernel and Firmware version i used before update? Is it wise to always exclude Kernel and Firmware updates when doing a full system update (pacman -Syu) ?

I still don't understand what i need to do to prevent something like this to happen again?!?

[dazboa]

Hello linarmux,
I had the same problem as you when I updated to the latest kernel the other day.
I'm not quite sure if the error was the same as yours as I run my pi headless and the journalctl method of
going through the logs is horrifying, to say the least:)

After 4 attempts at reboot nothing was connecting so I put the sdcard in my
main machine. I renamed the kernel.img to *.bak and slapped it back in the pi to let the emergency kernel boot.

once it booted I downgraded the kernel to the last version and hey-ho off we go.

I see in your boot partition the dates as indeed different from mines
for, kernel.img - ver.3.14.2-1, I'm showing 27/04/14 19:56
and start*.elf are all showing 16/04/14
are both your boot dirs showing the same?

on a side note, with no relevance except curiosity, I see you have a directory called "system volume information," I thought you got that only a ntfs formatted drive!? maybe you converted the partition?

When I saw your post today I went through the process of updating to the latest kernel again,
to check what the error actually was, but it booted first time without fail, weird!

So now I'm really curious...

The only thing that changed, when I had updated originally, was my fstab. I had a reference to an
external drive that was switched off, this drive is what stores my transmission torrents and also samba uses it over the network -- to display my my shiny new torrents;) I commented this line out and had no problem.
I should add that I had not had a problem rebooting the pi before, with the drive off.

It turns out this was the problem for me it fails under these conditions:
with the correct fstab entry in place and the external drive plugged into the usb port but
the external drive is powered off. The raspberry pi will fail to boot. I will find a hdmi cable and check the exact error later.

Probably not what the problem is for you but if you need any other info I will happy to supply any.

Best Regards
Darren

[linarmux]

Hello dazboa,

thanks for your detailed replay and all your effort, really much appreciated! Both of my Pi (owncloud & transmission) are running headless too (only connected them to my TV to see the error messages).

After a full system update on my transmission pi i made a reboot without the sync command because at that time a wasn't aware of the sync command and that it's recommended ( or mandatory? ) to sync after update.

Did you sync after kernel upgrade and still got the kernel panic error?

On my second Pi (owncloud) i lost telnet connectivity (screen freeze) during system update (package installation) after a while (to wait for the maybe ongoing package installation) i powered off the Pi.

Both Pi booted at first with all LED's on but they had no network connectivity (ping) and because they are both headless i couldn't see the error messages. After serveral reboots (poweroff) only the red power led was on and i got the error messages i posted.

Thanks to your post i renamed the kernel.img and used the emergency kernel to boot and now the owncloud Pi is booting with keyboard support (and maybe network conectivity too). Unfortunately i don't have the time right now to investigate there any further because it seems to be some kind of recovery console but it's definitely a step in the right direction! Hopefully i can look into it later!

You're right the "system volume information" was created by windows (looking at the matching timestamp) because i only have a VM running with Linux and at first the card reader was connected to the windows host.

On my transmission PI the Kernel.img, start*.elf etc. are all showing 16/04/14 and i don't have a kernel_emergency.img in the boot partition - hence a differnet approach is needed here.

Will post any new results when i have a more time (little stressed right now but i wanted to response to your post).

Thanks again you already helped me a lot!

Best Regards
Thomas

[sdjf]

I would say that sync after updating is essential. Think about it. the sync command ensures that all writes waiting in memory, are written to your system before you power off. If you do not sync, there is likely to be stuff that never got written that is essential.

as far as Linux as the main system, the rule is you make backups before updates! If anything goes wrong with an update, you can roll your system back to the backup from just before the update.

making backups occasionally is a good idea, anyhow, as you never know when something might go wrong, and important stuff can get lost or difficult to access. But that is true for all computer operating systems, not just Linux.