kernel panic on Bubba3/Excito, Kirkwood 88F6281_A1

[ReedWood]

Hi all,

as of 3.15.2-1-ARCH #1 PREEMPT I get a kernel panic on boot, most probably due to L2 cache. Here is the log.

I am really busy with real live right now, but I try to be responsive in case anyone works on this issue and needs info.

Thank you in advance!

---kernel panic log---

Log of kernel panic.
Kernel: 3.15.2-1-ARCH #1 PREEMPT armv5tel GNU/Linux

L2 cache is left enabled by u-boot version on this device (but there is no L2)

System: Bubba3/Excito
SoC: Kirkwood 88F6281_A1
DRAM: 512 MiB

processor : 0
model name : Feroceon 88FR131 rev 1 (v5l)
Features : swp half thumb fastmult edsp
CPU implementer : 0x56
CPU architecture: 5TE
CPU variant : 0x2
CPU part : 0x131
CPU revision : 1

Hardware : BUBBA3 Kirkwood based miniserver
Revision : 0000
Serial : 0000000000000000


[ 30.626202] mm/pgtable-generic.c:33: bad pmd e28d8003.
[ 30.713963] random: systemd urandom read with 107 bits of entropy available
[ 30.721143] Unable to handle kernel paging request at virtual address e09bb034
[ 30.728391] pgd = df7e8000
[ 30.731082] [e09bb034] *pgd=1f421811, *pte=00000000, *ppte=00000000
[ 30.737384] Internal error: Oops: 807 [#1] PREEMPT ARM
[ 30.742494] Modules linked in:
[ 30.745540] CPU: 0 PID: 1 Comm: systemd Not tainted 3.15.2-1-ARCH #1
[ 30.751862] task: df448000 ti: df44c000 task.ti: df44c000
[ 30.757245] PC is at n_tty_open+0x38/0x124
[ 30.761329] LR is at map_vm_area+0x38/0x58
[ 30.765411] pc : [<c032fb78>] lr : [<c00dc9b4>] psr: a0000013
[ 30.765411] sp : df44dd58 ip : 00000000 fp : 00000000
[ 30.776830] r10: df513180 r9 : 00020101 r8 : 00500001
[ 30.782032] r7 : 00002240 r6 : df761000 r5 : e09bd240 r4 : e09bb000
[ 30.788529] r3 : ffff8eda r2 : c087926c r1 : c068bee8 r0 : e09bd258
[ 30.795028] Flags: NzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 30.802130] Control: 0005397f Table: 1f7e8000 DAC: 00000015
[ 30.807849] Process systemd (pid: 1, stack limit = 0xdf44c1c0)
[ 30.813654] Stack: (0xdf44dd58 to 0xdf44e000)
[ 30.817994] dd40: c032fb40 df761000
[ 30.826135] dd60: df7bbf00 df761000 00000000 c0333c8c df55f400 00000000 df7bbf00 c0334404
[ 30.834274] dd80: df5f8280 df761000 00000000 c032c814 00020101 df513180 df44deb0 df40cd20
[ 30.842413] dda0: 00000001 df5f8280 df40cd20 c032d464 00000001 00000000 c07b5399 c0879230
[ 30.850552] ddc0: c057e194 df513180 df40cd20 00000000 df44c000 00000000 df44deb0 c00f5f88
[ 30.858691] dde0: 00000000 00000000 df44deb0 df40cd20 df513180 df40cd28 df717d80 c00f5ebc
[ 30.866830] de00: 00000000 c00f04e0 00000000 df44deac df44df74 00020101 00000000 00000000
[ 30.874968] de20: 00000000 c00f08f8 df44def0 c00feb04 b6d04930 df44de64 00000004 00000000
[ 30.883107] de40: beb02d74 df40cd20 00000022 df0012a8 00000000 00000000 00000000 c0109a98
[ 30.891246] de60: 05b6719b 00000007 df41d015 df513180 c086a5e0 df40cd20 df44def0 df44df74
[ 30.899385] de80: df41d000 df44c000 00000000 00000000 beb02d74 c00ff46c df44deac 00000000
[ 30.907524] dea0: 00000000 c0103624 00000000 00000000 df40cbf0 df14cc38 beb02d74 c00eb124
[ 30.915663] dec0: 00000cc5 df44df74 00000001 df41d000 ffffff9c c00095c4 df44c000 00000000
[ 30.923801] dee0: beb02d74 c0100a34 00000041 df44df90 df40cbf0 df14cc38 05b6719b 00000007
[ 30.931941] df00: df41d015 c0780a40 00000000 df170770 df513180 00000101 00000004 0000004a
[ 30.940079] df20: 00000000 00000000 000a0101 00080101 000a0101 df40e900 00000003 c010da38
[ 30.948218] df40: 000a0101 000a0101 df41d000 ffffff9c 00000005 000a0101 df41d000 ffffff9c
[ 30.956356] df60: 00000003 c00f1284 df40cb50 00000000 00000003 00020101 00000000 00000022
[ 30.964496] df80: 00000100 00000001 00000000 b6f4ebf4 00080101 beb02550 00000005 c00095c4
[ 30.972635] dfa0: df44c000 c0009460 b6f4ebf4 00080101 b6f4ebf4 000a0101 00000000 00000000
[ 30.980773] dfc0: b6f4ebf4 00080101 beb02550 00000005 f6febe50 0004fcf5 b6f9250c beb02d74
[ 30.988913] dfe0: b6f88c48 beb02360 b6f21858 b6db5534 60000010 b6f4ebf4 00000000 00010000
[ 30.997064] [<c032fb78>] (n_tty_open) from [<c0333c8c>] (tty_ldisc_open.isra.2+0x48/0x8c)
[ 31.005212] [<c0333c8c>] (tty_ldisc_open.isra.2) from [<c0334404>] (tty_ldisc_setup+0x18/0x58)
[ 31.013789] [<c0334404>] (tty_ldisc_setup) from [<c032c814>] (tty_init_dev+0xa4/0x1a0)
[ 31.021668] [<c032c814>] (tty_init_dev) from [<c032d464>] (tty_open+0x2a8/0x5bc)
[ 31.029034] [<c032d464>] (tty_open) from [<c00f5f88>] (chrdev_open+0xcc/0x1c4)
[ 31.036228] [<c00f5f88>] (chrdev_open) from [<c00f04e0>] (do_dentry_open+0x1c4/0x284)
[ 31.044019] [<c00f04e0>] (do_dentry_open) from [<c00f08f8>] (finish_open+0x34/0x4c)
[ 31.051642] [<c00f08f8>] (finish_open) from [<c00feb04>] (do_last.isra.39+0x3f0/0xca8)
[ 31.059529] [<c00feb04>] (do_last.isra.39) from [<c00ff46c>] (path_openat+0xb0/0x638)
[ 31.067322] [<c00ff46c>] (path_openat) from [<c0100a34>] (do_filp_open+0x2c/0x80)
[ 31.074770] [<c0100a34>] (do_filp_open) from [<c00f1284>] (do_sys_open+0x104/0x1cc)
[ 31.082392] [<c00f1284>] (do_sys_open) from [<c0009460>] (ret_fast_syscall+0x0/0x2c)
[ 31.090105] Code: e2850018 e5933000 e59f10e0 e59f20e0 (e5843034)
[ 31.096243] ---[ end trace 880740b8f9c01e10 ]---
[ 31.101472] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 31.101472]
[ 31.110564] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 31.110564]
[ 123.892224] random: nonblocking pool is initialized
(END)

[moonman]

There is l2 cache on theses cpus, it just needs to be disabled to decompress kernel, and then it gets reenabled. It seems this is not the problem here. Try a newer uboot, and as a last resort append a dtb to the kernel and try that. Boardfiles support os going away so i really couldn't tell you what's wrong. Device tree support for this device has been mainlined, but afaik there still is no uboot that can easily handle it

[ReedWood]

There is a half finished uboot update from the manufacturer which I use via kwboot. With this uboot version, the system boots, but has its own set of issues (mostly the network interfaces do not come up).

Given the mainline uboot post on the kirkwood forum, does arch linux arm expect in the future this uboot version on supported devices?

[moonman]

I remember that. You can bypass this problem by installing "macspoof" package.
Install the package
Edit /etc/conf.d/macspoof and put the interface macs one on each line
Then systemctl enable macspoof@eth0 and systemctl enable macspoof@eth1 (IIRC there are 2 interfaces on this device). Do that on a working system, test with a different from default mac. Then fire up kwboot and try to boot and see if the tenwork comes up automatically.

None of the developers have this device so it is not going to be fully supported in the way other devices are. Mainline u-boot does not support it. I added support into the kernel I because I could and out of curiosity i suppose, and blindly (you tested for me). I merely ported an existing patch to a newer kernel.

[ReedWood]

Thank you for your answer.

To your first point: I will try.

To your second point:
I see that it is a problem when none of the developers have the device I am seeking support for. Let me thank you again for adding support for this device to the kernel. While you fixing and me testing is a nice thing from a social point of view, it does not scale. Can you point me to documentation on u-boot and the ALARM Mainline u-boot such that I can try to improve device support on my own?

If there is a developer located in Germany who wants to hack on the device, I would also offer to borrow mine.

Thank you in advance.

[moonman]

Afaik there is noone in Germany. The biggest thing is to get UBoot ported to your device. All other pieces are there. You could ask the uboot guys @ denx.de.

[ReedWood]

I have the feeling that porting u-boot is a little above my head, but I will try my best. Thanks for the info.

[moonman]

Any luck with macspoof?

[ReedWood]

Not yet tried sorry, however, I guess it should work.

Right now I am working on porting the mainline u-boot to the bubba3. Since the mainline u-boot seems to be the vanilla u-boot from denk plus the configs for the devices, I see a real change in porting, since the config for the b3 are given in the last two commits of
https://github.com/Excito/uboot-updated

If there is anything to report, I will open a new thread.

[ReedWood]

I tried macspoof, and while the service did not come up on boot, a manual call to systemctrl successfully set the MAC of the network device. So, macspoof works. I did not investigate why it did not start on boot since the newest kernel again works with the stock uboot.

Thanks for the help.