f2fs kernel bug (using 3.9.4-1 latest) [SOLVED]

[crashmaster]

Yesterday f2fs throws a kernel bug, but system continues running and until now i didn't find any insufficiencies or errors but who knows ...

Is there a tool to check the fs, f.e. something like fsck.f2fs ?

$this->bbcode_second_pass_code('', '
[ 157.925117] ------------[ cut here ]------------
[ 157.927112] kernel BUG at fs/f2fs/node.c:900!
[ 157.928922] Internal error: Oops - BUG: 0 [#1] PREEMPT ARM
[ 157.931146] Modules linked in: leds_gpio led_class spi_bcm2708 bcm2708_wdog w1_therm w1_gpio wire cn snd_bcm2835 snd_pcm snd_page_alloc snd_timer snd i2c_bcm2708 i2c_dev 8192cu
[ 157.939754] CPU: 0 Not tainted (3.9.4-1-ARCH+ #1)
[ 157.941850] PC is at get_node_page+0xa8/0xb4
[ 157.971578] LR is at read_node_page+0x4c/0x5c
[ 158.001357] pc : [<c02f0c30>] lr : [<c02f0ae0>] psr: 20000013
sp : caba3e68 ip : cab00600 fp : 00000562
[ 158.091374] r10: 00006c03 r9 : cab00400 r8 : 00000000
[ 158.121234] r7 : 00000001 r6 : 00000000 r5 : 00006c03 r4 : c09fbc40
[ 158.177876] r3 : 00000000 r2 : 000ce320 r1 : 00000041 r0 : 00000000
[ 158.236101] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
[ 158.295819] Control: 00c5387d Table: 09688008 DAC: 00000017
[ 158.356263] Process f2fs_gc-179:2 (pid: 52, stack limit = 0xcaba21b0)
[ 158.417316] Stack: (0xcaba3e68 to 0xcaba4000)
[ 158.448061] 3e60: c681a007 00000001 00000000 c02ed040 00000000 c05b594c
[ 158.508194] 3e80: 00000001 cab04720 000001b1 00000001 caba3ec8 00000001 000003f2 c09fb340
[ 158.569531] 3ea0: caba2000 c681a000 00000000 00000000 000ae900 00000000 cab00600 00000000
[ 158.632348] 3ec0: 00000000 00000562 caba3ec8 caba3ec8 000104cd 000104cd 000d26d3 00000000
[ 158.696092] 3ee0: 00000000 00000000 00000000 00000000 00000000 caab3314 cab00578 cab04204
[ 158.762540] 3f00: c00452d4 caba2000 00007530 c087938c cab00400 c02edb90 00000000 0006d200
[ 158.829628] 3f20: 000d3c00 000573d2 000d3c00 00000000 ffbf77ff 00000000 caba0f60 c00452d4
[ 158.897212] 3f40: caba3f40 caba3f40 c02ed9b8 ca831ce4 00000000 cab00400 c02ed9b8 00000000
[ 158.965216] 3f60: 00000000 00000000 00000000 c00447bc fffbfffd 00000000 fffefeff cab00400
[ 159.032965] 3f80: 00000000 caba3f84 caba3f84 00000000 caba3f90 caba3f90 caba3fac ca831ce4
[ 159.101585] 3fa0: c0044718 00000000 00000000 c000dc38 00000000 00000000 00000000 00000000
[ 159.170954] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 159.239670] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 fbffffff fffeebff
[ 159.308130] [<c02f0c30>] (get_node_page+0xa8/0xb4) from [<c02ed040>] (f2fs_gc+0x21c/0xb94)
[ 159.377317] [<c02ed040>] (f2fs_gc+0x21c/0xb94) from [<c02edb90>] (gc_thread_func+0x1d8/0x37c)
[ 159.447006] [<c02edb90>] (gc_thread_func+0x1d8/0x37c) from [<c00447bc>] (kthread+0xa4/0xb0)
[ 159.516843] [<c00447bc>] (kthread+0xa4/0xb0) from [<c000dc38>] (ret_from_fork+0x14/0x3c)
[ 159.586546] Code: e8bd8070 e3e0400b e1a00004 e8bd8070 (e7f001f2)
[ 159.655672] ---[ end trace b25ffacb1bf5afd2 ]---
')

After a reboot it happened again (after 196 secs), maybe a bad sector on the sdcard but shouldn't this be managed by f2fs?

[moonman]

As far as I know there isn't fsck for f2fs but they are working on it. Never' had an oops like that, but I would say it is possible. Actually lack of fsck almost made me fo back to ext4 when I was having problems, turned out it wasn't f2fs though.

The only (crappy) way that may resolve your problem if it is due to corruption is to cp -a all your data to other storage, format, and cp -a back. If you have time to try it, would be nice to find out if the oops is due to corruption.

Function that's responsible for oops:
$this->bbcode_second_pass_code('', 'struct page *get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid)
{
int err;
struct page *page;
struct address_space *mapping = sbi->node_inode->i_mapping;

page = grab_cache_page(mapping, nid);
if (!page)
return ERR_PTR(-ENOMEM);

err = read_node_page(page, READ_SYNC);
if (err) {
f2fs_put_page(page, 1);
return ERR_PTR(err);
}

BUG_ON(nid != nid_of_node(page));
mark_page_accessed(page);
return page;
}
')

It is triggered by

BUG_ON(nid != nid_of_node(page));

I don't really understand what it is though.
nid - I suppose this is node id, nid_of_node probably gives node id of a page.

[crashmaster]

Moonman, thanks for the detailed info!

In the meantime i booted several times and it happens again and again but only once during a session, always after ca. 150 secs. But everything works reliable, can't see any data loss or damage. I shouldn't look too much into the logs.
If i'm fancy i also will try it with a refreshed partition but at the moment there seems no reason to do so. It's not a real problem until now.

[crashmaster]

New result:
I just did an update and it installed a newer version of "netctl". After a reboot wlan didn't work. I mounted the sdcard on pc and it shows some netctl systemd files under "/". I simply moved them to the right place in "/usr/lib/systemd/system", it boots fine and the best: no kernel bug, system runs nearly 30 mins now.
Seems the reason was a "twisted" node 'only'.
Let's wait until it happens (hopefully not) again.

EDIT:
Oh no, updating netctl on another rpi also installed the files under "/". Probably an insufficient pkg file. But this (hopefully) repaired my f2fs. That's why i choosed the nick "crashmaster"

EDIT2:
...and yes, it throws the error again after 2600 secs. Thanks, this keeps all understandable.

[crashmaster]

$this->bbcode_second_pass_quote('moonman', '
')The only (crappy) way that may resolve your problem if it is due to corruption is to cp -a all your data to other storage, format, and cp -a back. If you have time to try it, would be nice to find out if the oops is due to corruption.

[crashmaster]

Finally i can say that all works fine now.

After it seems to work reliable for some weeks, I believed to have a save working overclocking configuration. In addition i believed to have suitable sdcards. My fault.

Switching overclocking off, the exception has gone but stressing the filesystem (ext4 or f2fs) i've got something like "mmcblk0 timeout: waiting for int ...". After testing several kinds of sdcards (stressing the filesystem) i can say:
If you see this warning in the logs (dmesg) although you are using default values in your config file: forget the sdcard!
There's a long discussion about this in other forums without a generally working solution, i think this is the reason.

I bought some recommended class 10 sdcards now and switched cpu frequency up to 850 mhz but no overclocking. All error and warning messages have gone (did this on 4 RPi's)

In config.txt i added "sdhci-bcm2708.enable_llm=0", didn't test it but maybe it's also significant.

Regards, mmi