[wandboard quad]reboot command does not boot [nearly solved]

[hansan]

Dear all,

I wrote already in a different thread http://archlinuxarm.org/forum/viewtopic.php?f=45&t=7042&start=60#p41233 that I could not reboot my wandboard quad (Rev B1) by the reboot or shutdown -r now command.

I have checked a few older images:

The following (kernel) versions properly boot after a reboot command:

Linux alarm 3.0.35-5-ARCH #1 SMP PREEMPT Thu Jan 23 14:43:31 MST 2014 armv7l GNU/Linux
Linux alarm 3.0.35-6-ARCH #1 SMP PREEMPT Wed Mar 19 16:46:49 MDT 2014 armv7l GNU/Linux

after pacman -Suy is this upgraded to version:

Linux alarm 3.10.17-3-ARCH #1 SMP Tue Jul 15 20:38:48 MDT 2014 armv7l GNU/Linux

With this kernel version is a boot after the reboot command not working anymore.
The boot process hangs after "Starting Kernel...."
During the shutdown I do see a few errors pointing on the watchdog and uboot detects that the reboot has to do with the watchdog "Reset cause: WDOG" can it be that something is not correctly initialized?

Shutdown and unsuccessful boot log:
$this->bbcode_second_pass_code('', '
uname -a
Linux wandboard 3.10.17-3-ARCH #1 SMP Tue Jul 15 20:38:48 MDT 2014 armv7l GNU/Linux
[root@wandboard ~]# reboot
[ OK ] Stopped target Sound Card.
Stopping User Manager for UID 0...
[ OK ] Stopped target Graphical Interface.
[ OK ] Stopped target Multi-User System.
Stopping Network Time Service...
Stopping OpenSSH Daemon...
Stopping Entropy Harvesting Daemon...
Stopping Automatic wired network connection using netctl profiles...
Stopping Login Service...
[ OK ] Stopped target Login Prompts.
Stopping Getty on tty1...
Stopping Serial Getty on ttymxc0...
Stopping D-Bus System Message Bus...
Starting Generate shutdown-ramfs...
[ OK ] Stopped OpenSSH Daemon.
[ OK ] Stopped Login Service.
[ OK ] Stopped D-Bus System Message Bus.
[ OK ] Stopped Entropy Harvesting Daemon.
[ OK ] Stopped Network Time Service.
[ OK ] Stopped Automatic wired network connection using netctl profiles.
[ OK ] Stopped Getty on tty1.
[ OK ] Stopped Serial Getty on ttymxc0.
[ OK ] Stopped User Manager for UID 0.
[ OK ] Removed slice user-0.slice.
[ OK ] Removed slice system-serial\x2dgetty.slice.
[ OK ] Removed slice system-getty.slice.
Stopping Permit User Sessions...
[ OK ] Removed slice system-netctl\x2difplugd.slice.
[ OK ] Started Generate shutdown-ramfs.
[ OK ] Stopped Permit User Sessions.
[ OK ] Stopped target Basic System.
[ OK ] Stopped target Slices.
[ OK ] Removed slice User and Session Slice.
[ OK ] Stopped target Paths.
[ OK ] Stopped target Timers.
[ OK ] Stopped target Sockets.
[ OK ] Closed PC/SC Smart Card Daemon Activation Socket.
[ OK ] Closed D-Bus System Message Bus Socket.
[ OK ] Stopped target System Initialization.
Stopping Load/Save RF Kill Switch Status of rfkill2...
Stopping Load/Save RF Kill Switch Status of rfkill1...
Stopping Load/Save RF Kill Switch Status of rfkill0...
Stopping Load/Save Screen Backlight Brightness of ba...ight_lvds.29...
Stopping Load/Save Screen Backlight Brightness of ba...light_lcd.28...
[ OK ] Stopped target Encrypted Volumes.
Stopping Load/Save Random Seed...
Stopping Apply Kernel Variables...
[ OK ] Stopped Apply Kernel Variables.
Stopping Setup Virtual Console...
[ OK ] Stopped Setup Virtual Console.
Stopping Update UTMP about System Boot/Shutdown...
[ OK ] Stopped target Swap.
[ OK ] Stopped target Remote File Systems.
[ OK ] Stopped Load/Save RF Kill Switch Status of rfkill2.
[ OK ] Stopped Load/Save RF Kill Switch Status of rfkill1.
[ OK ] Stopped Load/Save RF Kill Switch Status of rfkill0.
[ OK ] Stopped Load/Save Screen Backlight Brightness of bac...klight_lvds.29.
[ OK ] Stopped Load/Save Screen Backlight Brightness of bac...cklight_lcd.28.
[ OK ] Stopped Load/Save Random Seed.
[ OK ] Stopped Update UTMP about System Boot/Shutdown.
Stopping Create Volatile Files and Directories...
[ OK ] Stopped Create Volatile Files and Directories.
[ OK ] Stopped target Local File Systems.
Unmounting /run/user/0...
Unmounting Temporary Directory...
[ OK ] Removed slice system-systemd\x2dbacklight.slice.
[ OK ] Removed slice system-systemd\x2drfkill.slice.
[ OK ] Unmounted /run/user/0.
[ OK ] Unmounted Temporary Directory.
[ OK ] Reached target Unmount All Filesystems.
[ OK ] Stopped target Local File Systems (Pre).
Stopping Create Static Device Nodes in /dev...
[ OK ] Stopped Create Static Device Nodes in /dev.
Stopping Remount Root and Kernel File Systems...
[ OK ] Stopped Remount Root and Kernel File Systems.
[ OK ] Reached target Shutdown.
imx2-wdt 20bc000.wdog: Unexpected close: Expect reboot!
systemd-shutdown[1]: Sending SIGTERM to remaining processes...
systemd-journald[168]: Received SIGTERM from PID 1 (systemd-shutdow).
systemd-shutdown[1]: Sending SIGKILL to remaining processes...
systemd-shutdown[1]: Hardware watchdog 'imx2+ watchdog', version 0
systemd-shutdown[1]: Unmounting file systems.
systemd-shutdown[1]: Unmounting /sys/kernel/debug.
systemd-shutdown[1]: Unmounting /sys/fs/fuse/connections.
systemd-shutdown[1]: Unmounting /dev/mqueue.
EXT4-fs (mmcblk0p1): re-mounted. Opts: (null)
EXT4-fs (mmcblk0p1): re-mounted. Opts: (null)
EXT4-fs (mmcblk0p1): re-mounted. Opts: (null)
systemd-shutdown[1]: All filesystems unmounted.
systemd-shutdown[1]: Deactivating swaps.
systemd-shutdown[1]: All swaps deactivated.
systemd-shutdown[1]: Detaching loop devices.
systemd-shutdown[1]: All loop devices detached.
systemd-shutdown[1]: Detaching DM devices.
systemd-shutdown[1]: All DM devices detached.
systemd-shutdown[1]: Successfully changed into root pivot.
systemd-shutdown[1]: Returning to initrd...
imx2-wdt 20bc000.wdog: Unexpected close: Expect reboot!
Sending SIGTERM to remaining processes...
Sending SIGKILL to remaining processes...
Hardware watchdog 'imx2+ watchdogcgroup: option changes via remount are deprecated (pid=1 comm=shutdown)
', version 0
Unmounting file syscgroup: option changes via remount are deprecated (pid=1 comm=shutdown)
tems.
Unmounting /oldroot/sys/fscgroup: option changes via remount are deprecated (pid=1 comm=shutdown)
/cgroup/perf_event.
Unmounting /cgroup: option changes via remount are deprecated (pid=1 comm=shutdown)
oldroot/sys/fs/cgroup/blkio.
Unmcgroup: option changes via remount are deprecated (pid=1 comm=shutdown)
ounting /oldroot/sys/fs/cgroup/frcgroup: option changes via remount are deprecated (pid=1 comm=shutdown)
eezer.
Unmounting /oldroot/sys/fcgroup: option changes via remount are deprecated (pid=1 comm=shutdown)
s/cgroup/devices.
Unmounting /olcgroup: option changes via remount are deprecated (pid=1 comm=shutdown)
droot/sys/fs/cgroup/memory.
Unmounting /oldroot/sys/fs/cgroup/cpu,cpuacct.
Unmounting /oldroot/sys/fs/cgroup/cpuset.
Unmounting /oldroot/sys/fs/cgroup/systemd.
Unmounting /oldroot/sys/fs/cgroup.
Unmounting /oldroot/run.
Unmounting /oldroot/dev/pts.
UEXT4-fs (mmcblk0p1): re-mounted. Opts: (null)
nmounting /oldroot/dev/shm.
Unmounting /oldroot/sys/kernel/security.
Unmounting /oldroot/proc.
Unmounting /oldroot/sys.
Unmounting /oldroot/dev.
Unmounting /oldroot.
All filesystems unmounted.
Deactivating swaps.
All swaps deactivated.
Detaching loop devices.
All loop devices detached.
Detaching DM devices.
All DM devices detached.
Rebooting.
sd 0:0:0:0: [sda] Synchronizing SCSI cache
imx2-wdt 20bc000.wdog: Device shutdown: Expect reboot!
Restarting system.


U-Boot 2014.04 (May 17 2014 - 13:26:22) Arch Linux ARM
CPU: Freescale i.MX6Q rev1.2 at 792 MHz
Reset cause: WDOG
Board: Wandboard
DRAM: 2 GiB
MMC: FSL_SDHC: 0, FSL_SDHC: 1
*** Warning - bad CRC, using default environment
In: serial
Out: serial
Err: serial
Net: FEC [PRIME]
Hit any key to stop autoboot: 1  0
mmc0 is current device
SD/MMC found on device 0
102 bytes read in 70 ms (1000 Bytes/s)
Importing environment from mmc (uEnv.txt)...
Checking if uenvcmd is set ...
Running default loadzimage ...
3600688 bytes read in 253 ms (13.6 MiB/s)
43443 bytes read in 78 ms (543 KiB/s)
Booting from mmc ...
Kernel image @ 0x12000000 [ 0x000000 - 0x36f130 ]
## Flattened Device Tree blob at 18000000
Booting using the fdt blob at 0x18000000
Using Device Tree in place at 18000000, end 1800d9b2

Starting kernel ...

')
Needless to say that the system does boot when I power cycle or push on the reset button.

Using the mainline kernel does not have this problem. Does this kernel support the vpuenc / vpudec options in gstreamer? If so, then using this kernel would be an option for me.

Greetings,

Han

[andrenam]

Hello Han,

No news from my side, I'm afraid. I don't know how to debug or fix this problem.
There were two other users (in the kernel thread) who were able reproduce this.
I'm just hoping someone will someday figure it out.

[hansan]

Hello Andrenam,

I did not find a solution yet.
The latest linux-wandboard kernel has still the same problem.
I did try a different 3.10.17 kernel from https://github.com/wandboard-org/linux/tree/wandboard_imx_3.10.17_1.0.1_ga_test which boots correctly (I have to admit that I did not test all features) and does also not reboot.

I find it strange that others do not have this problem. I operate the system headless (no hdmi monitor connected), maybe that is the difference?

I have no real plan yet; or I have to diff this wandboard 3.10.17 kernel with the 3.14+ mainline armv7 kernel. Or I have to try to check over the jtag interface where the kernel hangs.

Probably I will first check if the VPU hardware encoding / decoding is supported in the armv7 kernel; when I looked at it before I didn't see a mentioning of the VPU in the bootlog, from which I assumed that it is/was not supported. I should also check with gstreamer.

Greetings

[Alarmed]

can you try to use one of the release candidate versions? linux-armv7-rc I think.

[hansan]

Sorry for the late reply.
I have tried the linux-armv7-rc and linux-armv7 kernels and they reboot normally. Only the wandboard kernel doesn't reboot.

I don't think I can use the linux-armv7(-rc) kernel because the IMX6 vpu support does not seems to be included. That is; the /dev/mxc_vpu is missing. However I have to admit that I did not get gstreamer with the VPU encoder/decoder working on the wandboard kernel either. But that is a different subject than my reboot problem.

[hansan]

I think I have solved the problem; or at least I made some progress.

My theory is that the reboot is a problem between the state that the kernel is expecting and the real state of the soc after a soft reset. Somehow are not al devices(?) of the soc reseted during a soft reset and U-boot is also not properly configuring them.

With this in mind I started to try a different u-boot versions. The mainline u-boot of denx (u-boot-2014.10-rc2) is not solving the problem either; it gives the same behavior as with the current u-boot (u-boot-2014.07).

However with the Freescale supplied u-boot (http://git.freescale.com/git/cgit.cgi/i ... t-imx.git/ branch imx_v2014.04_3.10.31_1.1.0_beta) is it possible to reboot the wandboard. (There is some work needed to get this working on a wandboard.)

I will try to find in the coming days some time to diff both trees to see what is actually the difference to come up with a patch.
I will also check if the new kernel linux-armv7 (3.16.x / 3.17.x) is also working with this modified u-boot.

[jurafxp]

hello Hans,

I have also the same problem with kernel 3.10.17, could you please describe the modifications you did to u-boot?
Were you able to find the root cause?

Also, when I tried kernel 3.14 (linux-linaro-stable-mx6), it was possible to reboot without problem.

[hansan]

I unfortunately did not find time to investigate the problem / differences in the U-boot versions further. The Freescale system has quite some changes, but my feeling is that most of them are for other devices and or for support of Android. But again I did not investigate this further.

What i did was that I downloaded the 2014.04 source from the freescale git repository. And then I have patched that with the alarm.patch used in the archlinux build system. That patch failed on the bootcmd parameters. Basically this patch only changes the includes/config/wandboard.h file. In the end I have copied the relevant part back from the archlinux version of this file. So this is a bit of a messy procedure.
Then I followed the normal build procedures for u-boot to get my version and that I have copied (with dd) over the original installed version. This made it possible to reboot with the 3.10.17 kernel.

While my problem is "solved", it is not satisfying at all, because I still do not know what is wrong. I did notice as well that other kernel versions don't have this problem. But they do not support the VPU subsystem, which I need to encode video. That is why I think that I am stuck with the 3.10.17 kernel.

Does the 3.14 kernel of linaro support the VPU subsystem?

Greetings

[pepedog]

This does for cubox-i https://github.com/linux4kix/linux-linaro-stable-mx6

[hansan]

$this->bbcode_second_pass_quote('pepedog', 'T')his does for cubox-i https://github.com/linux4kix/linux-linaro-stable-mx6