linux-raspberrypi4 broken?

This forum is for topics dealing with problems with software specifically in the AArch64 repo.

Re: linux-raspberrypi4 broken?

Postby Amphitryon » Mon Jun 21, 2021 10:51 am

The forum is being awkward and won't let me quote previous posts! Anyway, doing it manually:
Is your EEPROM up-to-date (`rpi-eeprom-update`)?

Code: Select all
*** UPDATE AVAILABLE ***
BOOTLOADER: update available
   CURRENT: Thu  3 Sep 12:11:43 UTC 2020 (1599135103)
    LATEST: Thu 29 Apr 16:11:25 UTC 2021 (1619712685)
   RELEASE: critical (/lib/firmware/raspberrypi/bootloader/critical)

  VL805_FW: Using bootloader EEPROM
     VL805: up to date
   CURRENT: 000138a1
    LATEST: 000138a1

If booting without a display manager / X works, it might be a temporary issue due to the graphics stack packages being updated while you ran `pacman -Syu`. Does running `pacman -Syu` now resolve your issue?

I couldn't log in via ssh either. Looking at the journal on the root partition I coped off it, it did not write anything at all to the journal during either of the failed boots so the problem seems much more fundamental than graphics.
Have you installed xf86-video-fbdev (not sure if strictly necessary, but I never had any issues with it)?

No, this is not installed.
Have you any special settings in /boot/config.txt? Mine reads

I did have some settings for sound as that wasn't previously working but I can't find a copy of the config.txt as it was. It now has:
Code: Select all
dtoverlay=vc4-kms-v3d
initramfs initramfs-linux.img followkernel

which seems completely standard.
Amphitryon
 
Posts: 19
Joined: Fri Apr 30, 2021 7:17 pm

Re: linux-raspberrypi4 broken?

Postby Amphitryon » Mon Jun 21, 2021 11:02 am

If you have indeed narrowed the issue down to the raspberrypi4 kernel...

It is back working now. As far as I know, the only difference are the kernel - I have put it in the ignore list for pacman to stop it upgrading from 5.10.43-2 => 5.10.44, and the config.txt file.

Unfortunately I can't get anything out of dmesg/journalctl as I can't get a command prompt, either locally or via ssh, and grabbing the journal files from the card afterwards shows nothing has been written for that boot. If I could work out how to backup up the configuration so I can restore it in the event that it doesn't boot again I could test whether a simpler config.txt and/or updated EEPROM would solve the problem.
Amphitryon
 
Posts: 19
Joined: Fri Apr 30, 2021 7:17 pm

Re: linux-raspberrypi4 broken?

Postby graysky » Mon Jun 21, 2021 11:08 am

graysky
 
Posts: 1021
Joined: Sun Jun 26, 2011 6:56 am
Location: /run/user/1000

Re: linux-raspberrypi4 broken?

Postby Amphitryon » Mon Jun 21, 2021 11:24 am

@graysky,

Yes, that does now seem like the same issue, or at least part of it. I wasn't so sure at first as in my case I wasn't getting an error message and the problem seemed to be a much more serious lock-up rather than just a graphics failure. With the EEPROM up-to-date, the simpler config.txt file and having run 'systemctl disable gdm' to give me a console login before trying to do anything graphical I have been able to boot it, verify that it works up until trying to start gdm, and also check the journal which this time does seem to be being written to and the error messages do indeed look very similar:
Jun 21 12:17:56 alarm gnome-shell[497]: Registering session with GDM
Jun 21 12:17:57 alarm kernel: ------------[ cut here ]------------
Jun 21 12:17:57 alarm kernel: WARNING: CPU: 2 PID: 205 at drivers/gpu/drm/drm_modeset_lock.c:246 drm_modeset_lock+0xdc/0x104 [drm]
Jun 21 12:17:57 alarm kernel: Modules linked in: snd_soc_hdmi_codec hid_elecom btsdio bluetooth joydev ecdh_generic vc4 ecc bcm2835_codec(C) bcm2835_v4l2(C) rpivid_m>
Jun 21 12:17:57 alarm kernel: CPU: 2 PID: 205 Comm: kworker/u8:4 Tainted: G C 5.10.44-1-ARCH #1
Jun 21 12:17:57 alarm kernel: Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT)
Jun 21 12:17:57 alarm kernel: Workqueue: events_unbound commit_work [vc4]
Jun 21 12:17:57 alarm kernel: pstate: 80000005 (Nzcv daif -PAN -UAO -TCO BTYPE=--)
Jun 21 12:17:57 alarm kernel: pc : drm_modeset_lock+0xdc/0x104 [drm]
Jun 21 12:17:57 alarm kernel: lr : drm_atomic_get_new_or_current_crtc_state+0x6c/0x114 [drm]
Jun 21 12:17:57 alarm kernel: sp : ffffffc011fcbcb0
Jun 21 12:17:57 alarm kernel: x29: ffffffc011fcbcb0 x28: ffffff810bdd9000
Jun 21 12:17:57 alarm kernel: x27: ffffff812ec02180 x26: ffffffc008d7c000
Jun 21 12:17:57 alarm kernel: x25: 0000000000000000 x24: ffffff8100880480
Jun 21 12:17:57 alarm kernel: x23: ffffff8101ec1080 x22: 0000000000000038
...
Unable to handle kernel paging request at virtual address fffffffffffffff9
Jun 21 12:17:57 alarm kernel: Mem abort info:
Jun 21 12:17:57 alarm kernel: ESR = 0x96000005
Jun 21 12:17:57 alarm kernel: EC = 0x25: DABT (current EL), IL = 32 bits
Jun 21 12:17:58 alarm kernel: SET = 0, FnV = 0
Jun 21 12:17:58 alarm kernel: EA = 0, S1PTW = 0
Jun 21 12:17:58 alarm kernel: Data abort info:
Jun 21 12:17:58 alarm kernel: ISV = 0, ISS = 0x00000005
Jun 21 12:17:58 alarm kernel: CM = 0, WnR = 0
Jun 21 12:17:58 alarm kernel: swapper pgtable: 4k pages, 39-bit VAs, pgdp=0000000001157000
Jun 21 12:17:58 alarm kernel: [fffffffffffffff9] pgd=0000000000000000, p4d=0000000000000000, pud=0000000000000000


etc. Has this alreay been reported upstream?
Last edited by Amphitryon on Mon Jun 21, 2021 11:27 am, edited 1 time in total.
Amphitryon
 
Posts: 19
Joined: Fri Apr 30, 2021 7:17 pm

Re: linux-raspberrypi4 broken?

Postby graysky » Mon Jun 21, 2021 11:26 am

graysky wrote:Results of just stepping through kernels is below. It seems 5.10.43-1 was the last-known-good. Someone should reproduce my findings here and open a ticket upstream: https://github.com/raspberrypi/linux/issues

Mention that the last-known-good commit is https://github.com/raspberrypi/linux/co ... 06a9016cef (this is what our 5.10.43-1 sourced).


I did not see anything relating to it when I quickly looked through the open issues.
graysky
 
Posts: 1021
Joined: Sun Jun 26, 2011 6:56 am
Location: /run/user/1000

Re: linux-raspberrypi4 broken?

Postby robg » Tue Jun 22, 2021 4:09 pm

This is enough evidence to confirm that the issue is indeed related to the latest upstream kernel update.
For that it's worth, to quote 6by9 at https://github.com/raspberrypi/linux/issues/4401:
This looks like the same error that Laurent has just seen in trying to bring up a DSI display - https://lists.freedesktop.org/archives/ ... 11819.html

Somethings clipped the text for the last known good commit - 06a9016cef is not a valid hash in the rpi-5.10.y branch.
Copy/paste from the archlinux forum thread, 86b1bed. That's the merge commit from 5.10.42 to 5.10.43.

I'd suspect it's part of #4365 that has triggered this.

Note that #4365 is a fix for a previous regression in the graphics stack related to a clock frequency error occuring during HDMI mode set. It was possible to overcome this previous regression by disabling frequency scaling with `force_turbo=1` in /boot/config.txt. To confim 6by9's suspicion (and that the current issue is still related to frequency scaling), can you try launching X (LXQt, Gnome, ...) with `force_turbo=1`?
robg
 
Posts: 61
Joined: Tue Jan 05, 2021 8:22 am

Re: linux-raspberrypi4 broken?

Postby k0os » Thu Jun 24, 2021 5:50 am

I had the same problem. Your rootfs is fine. You need to fix your boot files.

Steps to resolve boot. All as root on the workstation you are using to fix your sd:

1. Back up your sd card

2. Download and extract these files into a working directory ​
https://archlinuxarm.org/packages/aarch ... spberrypi4

3. Copy the files to your sd cards boot partition
​and overwrite/merge all files and folders.

4. Check the rootfs for the additional directories downloaded above to see if the files exist (usr/ & /etc). We mostly want to be sure /etc/mkinitcpio.d/ contains only one config file (unless your fancy with custom kernals and image outputs). It should contain either aarch64 or raspi preset file. Updates will create the arch version if it doesn't exist so it's optimal to use the arch preset file... they do the same thing because they have the same script.
Archive which ever one you don't want by changing the extension to old. We will need this later for mkiniocio -P (More than one preset file will generate images for each confif file and just creates unnecessary work for mkinit).

5. make sure /etc/fstab contains the correct mount points for /proc / and /boot accordingly.

6. edit boot/cmdline.text to the following text, changing the mount to the rootfs of your rootfs device. You can read more about it on the raspi-config / eeprom config wiki on the rpi site. I use serial console for setup but the following is the default.

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

7. Reseat your SD card and boot. If the pi boots into kernel panic [rootfs] use commands:
mount /dev/<your rfs device> /new_mount press return
exit press return.

mount /dev/mmcblk0p2 /new_root
exit

Your rootfs should boot.

8. Once logged in open terminal. As root run command mkinitpcio -P. Your boot init should generate at this point (only once per the changes we made in #3 above). At this point you'll want to back up your boot partition. I created an update.sh that runs updates as sudo, waits for the process to be done then copies my boot files back to the boot partition as sudo and replaces them.

let me know how your fare.

K
k0os
 
Posts: 1
Joined: Thu Jun 24, 2021 4:55 am

Re: linux-raspberrypi4 broken?

Postby acorbi » Thu Jun 24, 2021 9:54 am

Hi all,

Using sway (wayland), on a rpi4 8Gb model, as my main desktop I was hit today by this bug.

The latest working kernel for me is : linux-raspberrypi4-5.10.44-2

Neither linux-raspberrypi4-5.10.44-3 nor linux-raspberrypi4-5.10.44-4 work. Both of them leave me with a black screen, not even I get a text based console to login.

I mean, I can see the boot process text messages but when I should get my login tty text ( I don't use graphical login managers) the screen is completely black.

Reverting to linux-raspberrypi4-5.10.44-2 makes the pi4 work again.
acorbi
 
Posts: 19
Joined: Wed Aug 05, 2020 10:31 am

Re: linux-raspberrypi4 broken?

Postby graysky » Thu Jun 24, 2021 11:23 am

Does sway have any logs you can post? Anything in dmesg? Post to the github issue if you have evidence that your wayland observation is related: https://github.com/raspberrypi/linux/issues/4401 .. if unrelated, create a new issue.

Note that for us, 5.10.44-2 used upstream 0088fda but reverting PR#4365. 5.10.44-4 used upstream 4915134.

You can share that in your upstream bug report.
graysky
 
Posts: 1021
Joined: Sun Jun 26, 2011 6:56 am
Location: /run/user/1000

Re: linux-raspberrypi4 broken?

Postby acorbi » Thu Jun 24, 2021 11:44 am

Hi graysky,

I think I've not explained correctly.
I can't even start sway because I can't login into te pi4.

I don't use graphical login managers (i.e. gdm or alike).
My (normal) boot process ends up (should) in a text terminal where I've configured auto-login and from there, when I want, I type 'sway' and that's all.

Well, what I wanted to say is that after the kernel/systemd boot messages dissappear from the console and I should land into good old text based vt1 all I get is a blackscreen, so I can't even start sway or any X11 based desktop.

With kernel 5.10.44-2 all seems to work ok.
acorbi
 
Posts: 19
Joined: Wed Aug 05, 2020 10:31 am

PreviousNext

Return to ARMv8

Who is online

Users browsing this forum: No registered users and 4 guests