kernel panic with latest kirkwood 3.9 update

[replabrobin]

After updating today I'm getting a kernel panic on my sheevaplug. I think the kernel package was 3.9.1

I'm booting from an SD card. I fscked the boot & root partitions on another machine and they're both fine. I have a backup of the SD card so can restore to that.

Any advice on what might be wrong?


$this->bbcode_second_pass_quote('', 'U')-Boot 2011.12 (Mar 11 2012 - 18:59:46)
Marvell-Sheevaplug - eSATA - SD/MMC

SoC: Kirkwood 88F6281_A1
DRAM: 512 MiB
WARNING: Caches not enabled
NAND: 512 MiB
In: serial
Out: serial
Err: serial
Net: egiga0 [PRIME]
88E1116 Initialized on egiga0
Hit any key to stop autoboot: 0
SDHC found. Card desciption is:
Manufacturer: 0x02, OEM "TM"
Product name: "SD08G", revision 3.8
Serial number: 3221812274
Manufacturing date: 9/2008
CRC: 0x00, b0 = 0
mmc1 is available
Loading file "uImage" from mmc device 0:1 (xxa1)
3293000 bytes read
## Booting kernel from Legacy Image at 00800000 ...
Image Name: Linux-3.9.1-1-ARCH
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 3292936 Bytes = 3.1 MiB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK

Starting kernel ...

Uncompressing Linux... done, booting the kernel.
[ 19.883905] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 19.892231] [<c000dab8>] (unwind_backtrace+0x0/0xe0) from [<c0466560>] (panic+0x80/0x1dc)
[ 19.900459] [<c0466560>] (panic+0x80/0x1dc) from [<c0603d20>] (mount_block_root+0x260/0x2b4)
[ 19.908947] [<c0603d20>] (mount_block_root+0x260/0x2b4) from [<c0604030>] (prepare_namespace+0x15c/0x1bc)
[ 19.918564] [<c0604030>] (prepare_namespace+0x15c/0x1bc) from [<c0603970>] (kernel_init_freeable+0x1bc/0x204)
[ 19.928529] [<c0603970>] (kernel_init_freeable+0x1bc/0x204) from [<c0464884>] (kernel_init+0x8/0xe4)
[ 19.937710] [<c0464884>] (kernel_init+0x8/0xe4) from [<c0008d50>] (ret_from_fork+0x14/0x24)

[replabrobin]

I may have some kind of hardware issue. I can no longer boot from another SD card I keep for backup or from the internal nand. The latter might be because I have ugraded u-boot to allow booting the latest arch and my nand kernel is very old.

However, when I handomatically restored the previous 3.8.12-1 packages using a second machine I find that my sheevaplug does boot from the sd card and all seems well. Any ideas?

[moonman]

Hard to say. I just checked the config and the patch and all seems fine. Is there really no other messages before the error dump? I would try booting with initrd to see if it you get more messages that maybe useful. Also, what is your u-boot environment?

[replabrobin]

No that is all the messages I'm getting (copied from a screen window). Not sure what you mean by boot from initrd.

[moonman]

not from initrd, but with it. Don't know if your U-Boot is setup to check for initrd, but here's uInitrd cd http://goo.gl/GFbtn -O uInitrd place in the /boot directory. Alternatively, if you can boot from USB, then see if your SD card is detected under the same name.

Moonman, my u-boot expertise is limited, but here's what I did.

1) copy the uInitrd and 3.9.1 uImage to my working mmc /boot/uInitrd & /boot/uIm391. The mmc rootfs still has the 3.9.1 modules etc etc installed.

2) reboot to the Marvel prompt and see this after booting the 3.9.1 + uInitrd
$this->bbcode_second_pass_quote('', 'U')-Boot 2011.12 (Mar 11 2012 - 18:59:46)
Marvell-Sheevaplug - eSATA - SD/MMC

SoC: Kirkwood 88F6281_A1
DRAM: 512 MiB
WARNING: Caches not enabled
NAND: 512 MiB
In: serial
Out: serial
Err: serial
Net: egiga0 [PRIME]
88E1116 Initialized on egiga0
Hit any key to stop autoboot: 0
Marvell>> mmc init
SDHC found. Card desciption is:
Manufacturer: 0x02, OEM "TM"
Product name: "SD08G", revision 3.8
Serial number: 3225807631
Manufacturing date: 10/2008
CRC: 0x00, b0 = 0
mmc1 is available
Marvell>> ext2ls mmc 0:1
<DIR> 1024 .
<DIR> 1024 ..
<DIR> 12288 lost+found
3251040 uImage
2510970 uInitrd
3293000 uIm391
Marvell>> setenv bootargs_console console=ttyS0,115200
Marvell>> setenv bootcmd_mmc 'mmc init; ext2load mmc 0:1 0x00800000 /uIm391; ext2load mmc 0:1 0x01100000 /uInitrd'
Marvell>> setenv bootcmd 'setenv bootargs $(bootargs_console); run bootcmd_mmc; bootm 0x00800000 0x01100000'
Marvell>> boot
SDHC found. Card desciption is:
Manufacturer: 0x02, OEM "TM"
Product name: "SD08G", revision 3.8
Serial number: 3225807631
Manufacturing date: 10/2008
CRC: 0x00, b0 = 0
mmc1 is available
Loading file "/uIm391" from mmc device 0:1 (xxa1)
3293000 bytes read
Loading file "/uInitrd" from mmc device 0:1 (xxa1)
2510970 bytes read
## Booting kernel from Legacy Image at 00800000 ...
Image Name: Linux-3.9.1-1-ARCH
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 3292936 Bytes = 3.1 MiB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 01100000 ...
Image Name: initramfs
Image Type: ARM Linux RAMDisk Image (gzip compressed)
Data Size: 2510906 Bytes = 2.4 MiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK

Starting kernel ...

Uncompressing Linux... done, booting the kernel.
:: running early hook [udev]
:: running hook [udev]
:: Triggering uevents...
ERROR: device '' not found. Skipping fsck.
ERROR: Unable to find root device ''.
You are being dropped to a recovery shell
Type 'exit' to try and continue booting
sh: can't access tty; job control turned off
[rootfs /]# ls /dev/mm*
ls: /dev/mm*: No such file or directory
[rootfs /]# ls /dev
block ptyp9 tty19 tty56
bus ptypa tty2 tty57
char ptypb tty20 tty58
console ptypc tty21 tty59
core ptypd tty22 tty6
cpu_dma_latency ptype tty23 tty60
fd ptypf tty24 tty61
full ram0 tty25 tty62
input ram1 tty26 tty63
kmsg ram10 tty27 tty7
loop-control ram11 tty28 tty8
loop0 ram12 tty29 tty9
loop1 ram13 tty3 ttyS0
loop2 ram14 tty30 ttyS1
loop3 ram15 tty31 ttyp0
loop4 ram2 tty32 ttyp1
loop5 ram3 tty33 ttyp2
loop6 ram4 tty34 ttyp3
loop7 ram5 tty35 ttyp4
mem ram6 tty36 ttyp5
mtd0 ram7 tty37 ttyp6
mtd0ro ram8 tty38 ttyp7
mtd1 ram9 tty39 ttyp8
mtd1ro random tty4 ttyp9
mtd2 rtc tty40 ttypa
mtd2ro rtc0 tty41 ttypb
network_latency stderr tty42 ttypc
network_throughput stdin tty43 ttypd
null stdout tty44 ttype
port tty tty45 ttypf
psaux tty0 tty46 ubi_ctrl
ptmx tty1 tty47 urandom
ptyp0 tty10 tty48 vcs
ptyp1 tty11 tty49 vcs1
ptyp2 tty12 tty5 vcsa
ptyp3 tty13 tty50 vcsa1
ptyp4 tty14 tty51 vga_arbiter
ptyp5 tty15 tty52 watchdog
ptyp6 tty16 tty53 watchdog0
ptyp7 tty17 tty54 zero
ptyp8 tty18 tty55
[rootfs /]#

[replabrobin]

Moonman,

I see from the repository that there's a 3.9.2 update. Should I try the kernel from that with your uInitrd and see if that has the mmc drivers? Alternatively is this a case of needing to get the right modules loaded early somehow?

[moonman]

The SD driver is built into the kernel and not built as a module. I really haven't changed the config. I wish I had one of these devices to test.

[replabrobin]

Moonman, I had a look myself and didn't see anything obvious in the config, but there were quite a few changes in the archlinuxarm patch. I could try and do a kernel build with those reversed if that makes any sense?

[moonman]

You can't reverse the patch because I had to make theold one work with the new kernel. The only changes are to structs in board files. There actually aren't that many, it's just git is creating a patch for a patch so it looks like a lot. Specifically the one with just 1 applied patch.