trying to understand fsck / e2fsck on Odroid U3

[kultex]

we are running a web server with nginx and postgreSQL on Odroid U3. /root and /home are running from SD - data is stored in EMMC. But the EMMC is not the original Odroid EMMC, the EMMC is from my Rock64, because the original EMMC from hardkernel does not boot from SD, when the original Ordoid EMMc is attached - https://forum.odroid.com/viewtopic.php? ... 36#p292736

The project was developed with Banana-Pi M1 and Armbian. The project got bigger, so we had to find a SBC with more power and ram - the U3 won against Rock64 and the brand new Pi4. But we have the problem, that the the EMMC module crashes - you cannot predict, sometimes 2 month, sometimes 2 weeks. The SSD on the Banana had no problems for 3 years.

Here is the part in journalctl - I did not find any errors regaring the emmc before

$this->bbcode_second_pass_code('', 'Mar 18 16:56:53 alarm systemd[23350]: Started Curl Script for Update spDYN.
Mar 18 16:56:53 alarm systemd[23350]: dyn-update.service: Main process exited, code=exited, status=127/n/a
Mar 18 16:56:53 alarm systemd[23350]: dyn-update.service: Failed with result 'exit-code'.
Mar 18 16:56:53 alarm bash[14880]: /usr/bin/bash: /home/alarm/update.sh: No such file or directory
Mar 18 16:57:55 alarm kernel: EXT4-fs warning (device mmcblk1p1): ext4_dirblock_csum_verify:370: inode #261638: comm postgres: No space for directory leaf checksum. Please run e2fsck -D.
Mar 18 16:57:55 alarm kernel: EXT4-fs error (device mmcblk1p1): __ext4_find_entry:1540: inode #261638: comm postgres: checksumming directory block 0
Mar 18 16:57:55 alarm kernel: Aborting journal on device mmcblk1p1-8.
Mar 18 16:57:55 alarm kernel: EXT4-fs (mmcblk1p1): Remounting filesystem read-only
Mar 18 16:58:04 alarm audit[386]: ANOM_ABEND auid=4294967295 uid=977 gid=977 ses=4294967295 subj=_ pid=386 comm="postgres" exe="/usr/bin/postgres" sig=6 res=1
Mar 18 16:58:04 alarm kernel: audit: type=1701 audit(1584547084.761:2068): auid=4294967295 uid=977 gid=977 ses=4294967295 subj=_ pid=386 comm="postgres" exe="/usr/bin/postgres" sig=6 res=1
Mar 18 16:58:04 alarm postgres[386]: 2020-03-18 16:58:04.761 CET [386] PANIC: could not write to log file 000000010000000000000005 at offset 172032, length 8192: Read-only file system ')

the emmc is quite empty:

$this->bbcode_second_pass_code('', '[alarm@alarm ~]$ df -m /dev/mmcblk1p1
Filesystem 1M-blocks Used Available Use% Mounted on
/dev/mmcblk1p1 14503 2918 10830 22% /media/daten')

the first time, this error occurred, I rebooted, because I thought, fsck will repair it automatic on boot, but it did not. I had to go 200 km to repair the server.

I am not familiar with arch and systemd, because I normally use debian without systemd. I thought when it is defined in fstab, fsck will check and repair it on every reboot - here my fstab:

$this->bbcode_second_pass_code('', 'UUID=xxx / ext4 errors=remount-ro,noatime 0 1
UUID=xxx /media/daten ext4 errors=remount-ro,noatime,defaults 0 2')

so my 1 question: I want that all partitions are checked on every boot and where I see the logs?, because I dont find them in journalctl - what do I have to do?

2 question, does systemd have any routine to check the EMMC or do I have to make 2 scripts, like every sunday, stop the server, run e2fsck and start it again?

[kultex]

I hope I found the error: Maximum mount count was set to -1 - means, it is never checked and I think this overrules the fstab

$this->bbcode_second_pass_code('', '[root@alarm alarm]# tune2fs -l /dev/mmcblk1p1 | grep Maximum
Maximum mount count: -1')

then I set it to 1 with:

$this->bbcode_second_pass_code('', '[root@alarm alarm]# tune2fs -c 1 /dev/mmcblk1p1
tune2fs 1.45.6 (20-Mar-2020)
Setting maximal mount count to 1')

and I hope, it will work now - I cannot reboot now

[themoseon]

$this->bbcode_second_pass_quote('', 'w')e are running a web server with nginx and postgreSQL on Odroid U3. /root and /home are running from SD - data is stored in EMMC. But the EMMC is not the original Odroid EMMC, the EMMC is from my Rock64, because the original EMMC from hardkernel does not boot from SD, when the original Ordoid EMMc is attached - https://forum.odroid.com/viewtopic.php? ... 36#p292736

The project was developed with Banana-Pi M1 and Armbian. The project got bigger, so we had to find a SBC with more power and ram - the U3 won against Rock64 and the brand new Pi4. But we have the problem, that the the EMMC module crashes - you cannot predict, sometimes 2 month, sometimes 2 weeks. The SSD on the Banana had no problems for 3 years.

Here is the part in journalctl - I did not find any errors regaring the emmc before

CODE: SELECT ALL
$this->bbcode_second_pass_code('', 'Mar 18 16:56:53 alarm systemd[23350]: Started Curl Script for Update spDYN.
Mar 18 16:56:53 alarm systemd[23350]: dyn-update.service: Main process exited, code=exited, status=127/n/a
Mar 18 16:56:53 alarm systemd[23350]: dyn-update.service: Failed with result 'exit-code'.
Mar 18 16:56:53 alarm bash[14880]: /usr/bin/bash: /home/alarm/update.sh: No such file or directory
Mar 18 16:57:55 alarm kernel: EXT4-fs warning (device mmcblk1p1): ext4_dirblock_csum_verify:370: inode #261638: comm postgres: No space for directory leaf checksum. Please run e2fsck -D.
Mar 18 16:57:55 alarm kernel: EXT4-fs error (device mmcblk1p1): __ext4_find_entry:1540: inode #261638: comm postgres: checksumming directory block 0
Mar 18 16:57:55 alarm kernel: Aborting journal on device mmcblk1p1-8.
Mar 18 16:57:55 alarm kernel: EXT4-fs (mmcblk1p1): Remounting filesystem read-only
Mar 18 16:58:04 alarm audit[386]: ANOM_ABEND auid=4294967295 uid=977 gid=977 ses=4294967295 subj=_ pid=386 comm="postgres" exe="/usr/bin/postgres" sig=6 res=1
Mar 18 16:58:04 alarm kernel: audit: type=1701 audit(1584547084.761:2068): auid=4294967295 uid=977 gid=977 ses=4294967295 subj=_ pid=386 comm="postgres" exe="/usr/bin/postgres" sig=6 res=1
Mar 18 16:58:04 alarm postgres[386]: 2020-03-18 16:58:04.761 CET [386] PANIC: could not write to log file 000000010000000000000005 at offset 172032, length 8192: Read-only file system')


the emmc is quite empty:

CODE: SELECT ALL
$this->bbcode_second_pass_code('', '[alarm@alarm ~]$ df -m /dev/mmcblk1p1
Filesystem 1M-blocks Used Available Use% Mounted on
/dev/mmcblk1p1 14503 2918 10830 22% /media/daten')


the first time, this error occurred my coles roster, I rebooted, because I thought, fsck will repair it automatic on boot, but it did not. I had to go 200 km to repair the server.

I am not familiar with arch and systemd, because I normally use debian without systemd. I thought when it is defined in fstab, fsck will check and repair it on every reboot - here my fstab:

CODE: SELECT ALL
$this->bbcode_second_pass_code('', 'UUID=xxx / ext4 errors=remount-ro,noatime 0 1
UUID=xxx /media/daten ext4 errors=remount-ro,noatime,defaults 0 2
')

so my 1 question: I want that all partitions are checked on every boot and where I see the logs?, because I dont find them in journalctl - what do I have to do?

2 question, does systemd have any routine to check the EMMC or do I have to make 2 scripts, like every sunday, stop the server, run e2fsck and start it again?

[ak404]

one of my laptops emmc goes awry every so often it forces me to keep a bootable usb drive handy. Never figure out to do it on boot.