linux-rpi-16k-6.18-1-aarch64.pkg not booting on rpi5

[bloke]

Hey Folks,
Just letting you know the latest upgrade made my rpi5 unbootable, its got a nvme, pineberry bottom hatdrive. I recovered the boot with a kernel downgrade to 6.12.55-1-rpi-16k from emergency usb drive.

Just letting you know in the 1st instance to maybe help others from boot issues.

With the system booting linux-rpi-16k-6.18-1-aarch64
At first the system hangs just after [ OK ] Activated swaps

after removing swaps from /etc/fstab it gets too

[ OK ] Finished Virtual Console Setup.

In fact it seems to stop at different random places around there... not sure the system is actually hung at that point as there seems to be drive activity ...

Any suggestions / ideas about this issue?

I thought maybe I updated during releases of packages, or at a bad time.

I have a working system with
# uname -r
6.12.55-1-rpi-16k

and following packages outstanding to be upgraded

# pacman -Syu
:: Synchronizing package databases...
core is up to date
extra is up to date
alarm is up to date
aur is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (10) libmakepkg-dropins-18-1 libplacebo-7.351.0-4
linux-rpi-16k-6.18-1 openresolv-3.17.3-1
pacman-7.1.0.r7.gb9f7d4a-1 shaderc-2025.5-1
spirv-tools-1:1.4.335.0-1 sysstat-12.7.9-1
vulkan-headers-1:1.4.335.0-1 vulkan-icd-loader-1.4.335.0-1

Total Download Size: 5.42 MiB
Total Installed Size: 101.65 MiB
Net Upgrade Size: 2.21 MiB

:: Proceed with installation? [Y/n] n

I have repeated this twice - ie upgrading to linux-rpi-16k-6.18-1 with a not so much desired outcome.

my config.txt

[code]# See /boot/overlays/README for all available options

initramfs initramfs-linux.img followkernel

[display]
dtoverlay=vc4-kms-v3d
max_framebuffers=2
disable_overscan=1
#disable_fw_kms_setup=1
arm_64bit=1
hdmi_force_hotplug:1=1
hdmi_force_hotplug:0=1
hdmi_enable_4kp60=0

[usb]
otg_mode=1

[pi5]
dtparam=nvme
dtparam=pciex1_gen=3
[/code]

[graysky]

My reference RPi5 booted fine (uSD). Are you booting off the NVMe or uSD?

[bloke]

Booting off the NVMe

been booting it like that since

# ls -ltr linux-rpi-16k-*
-rw-r--r-- 1 root root 25564416 Jan 5 2024 linux-rpi-16k-6.6.9-2-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jan 5 2024 linux-rpi-16k-6.6.9-2-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25551696 Jan 9 2024 linux-rpi-16k-6.6.10-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jan 9 2024 linux-rpi-16k-6.6.10-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25551180 Jan 14 2024 linux-rpi-16k-6.6.11-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jan 14 2024 linux-rpi-16k-6.6.11-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25539536 Jan 16 2024 linux-rpi-16k-6.6.11-2-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jan 16 2024 linux-rpi-16k-6.6.11-2-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25544028 Jan 20 2024 linux-rpi-16k-6.6.12-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jan 20 2024 linux-rpi-16k-6.6.12-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25540732 Jan 24 2024 linux-rpi-16k-6.6.13-2-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jan 24 2024 linux-rpi-16k-6.6.13-2-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25707156 Jan 31 2024 linux-rpi-16k-6.6.14-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jan 31 2024 linux-rpi-16k-6.6.14-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25723648 Feb 7 2024 linux-rpi-16k-6.6.16-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Feb 7 2024 linux-rpi-16k-6.6.16-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25723112 Feb 14 2024 linux-rpi-16k-6.6.16-2-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Feb 14 2024 linux-rpi-16k-6.6.16-2-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25723796 Feb 17 2024 linux-rpi-16k-6.6.16-3-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Feb 17 2024 linux-rpi-16k-6.6.16-3-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25727016 Feb 24 2024 linux-rpi-16k-6.6.18-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Feb 24 2024 linux-rpi-16k-6.6.18-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25727032 Mar 20 2024 linux-rpi-16k-6.6.22-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Mar 20 2024 linux-rpi-16k-6.6.22-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25729564 Mar 29 2024 linux-rpi-16k-6.6.23-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Mar 29 2024 linux-rpi-16k-6.6.23-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 25731520 Apr 6 2024 linux-rpi-16k-6.6.25-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Apr 6 2024 linux-rpi-16k-6.6.25-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26096144 Apr 20 2024 linux-rpi-16k-6.6.28-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Apr 20 2024 linux-rpi-16k-6.6.28-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26322848 May 14 2024 linux-rpi-16k-6.6.30-3-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 May 14 2024 linux-rpi-16k-6.6.30-3-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26325012 May 16 2024 linux-rpi-16k-6.6.30-4-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 May 16 2024 linux-rpi-16k-6.6.30-4-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26359432 May 25 2024 linux-rpi-16k-6.6.31-2-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 May 25 2024 linux-rpi-16k-6.6.31-2-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26373988 Jun 7 2024 linux-rpi-16k-6.6.32-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jun 7 2024 linux-rpi-16k-6.6.32-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26281376 Jun 13 2024 linux-rpi-16k-6.6.33-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jun 13 2024 linux-rpi-16k-6.6.33-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26283368 Jun 25 2024 linux-rpi-16k-6.6.35-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jun 25 2024 linux-rpi-16k-6.6.35-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26296040 Jul 2 2024 linux-rpi-16k-6.6.36-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jul 2 2024 linux-rpi-16k-6.6.36-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26288716 Jul 17 2024 linux-rpi-16k-6.6.40-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jul 17 2024 linux-rpi-16k-6.6.40-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26300248 Jul 20 2024 linux-rpi-16k-6.6.41-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jul 20 2024 linux-rpi-16k-6.6.41-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26299188 Jul 27 2024 linux-rpi-16k-6.6.42-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jul 27 2024 linux-rpi-16k-6.6.42-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26403204 Aug 7 2024 linux-rpi-16k-6.6.44-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Aug 7 2024 linux-rpi-16k-6.6.44-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26404208 Aug 14 2024 linux-rpi-16k-6.6.45-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Aug 14 2024 linux-rpi-16k-6.6.45-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26402812 Aug 16 2024 linux-rpi-16k-6.6.45-2-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Aug 16 2024 linux-rpi-16k-6.6.45-2-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26446884 Aug 31 2024 linux-rpi-16k-6.6.47-3-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Aug 31 2024 linux-rpi-16k-6.6.47-3-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26464876 Sep 11 2024 linux-rpi-16k-6.6.50-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Sep 11 2024 linux-rpi-16k-6.6.50-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26470172 Sep 15 2024 linux-rpi-16k-6.6.51-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Sep 15 2024 linux-rpi-16k-6.6.51-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26492476 Oct 11 2024 linux-rpi-16k-6.6.56-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Oct 11 2024 linux-rpi-16k-6.6.56-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26501600 Oct 18 2024 linux-rpi-16k-6.6.57-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Oct 18 2024 linux-rpi-16k-6.6.57-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26509484 Oct 23 2024 linux-rpi-16k-6.6.58-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Oct 23 2024 linux-rpi-16k-6.6.58-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26548148 Nov 2 2024 linux-rpi-16k-6.6.59-2-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Nov 2 2024 linux-rpi-16k-6.6.59-2-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 26584160 Nov 19 2024 linux-rpi-16k-6.6.62-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Nov 19 2024 linux-rpi-16k-6.6.62-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27633596 Nov 26 2024 linux-rpi-16k-6.12.1-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Nov 26 2024 linux-rpi-16k-6.12.1-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27658840 Dec 21 2024 linux-rpi-16k-6.12.6-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Dec 21 2024 linux-rpi-16k-6.12.6-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27698984 Jan 28 2025 linux-rpi-16k-6.12.11-2-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jan 28 2025 linux-rpi-16k-6.12.11-2-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27719960 Feb 1 2025 linux-rpi-16k-6.12.11-3-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Feb 1 2025 linux-rpi-16k-6.12.11-3-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27948864 Mar 1 2025 linux-rpi-16k-6.12.17-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Mar 1 2025 linux-rpi-16k-6.12.17-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27957688 Mar 11 2025 linux-rpi-16k-6.12.18-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Mar 11 2025 linux-rpi-16k-6.12.18-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27952908 Mar 19 2025 linux-rpi-16k-6.12.19-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Mar 19 2025 linux-rpi-16k-6.12.19-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27987616 Mar 26 2025 linux-rpi-16k-6.12.20-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Mar 26 2025 linux-rpi-16k-6.12.20-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27992560 Apr 1 2025 linux-rpi-16k-6.12.21-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Apr 1 2025 linux-rpi-16k-6.12.21-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 27994128 Apr 11 2025 linux-rpi-16k-6.12.22-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Apr 11 2025 linux-rpi-16k-6.12.22-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28138720 May 8 2025 linux-rpi-16k-6.12.27-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 May 8 2025 linux-rpi-16k-6.12.27-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28142940 May 15 2025 linux-rpi-16k-6.12.28-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 May 15 2025 linux-rpi-16k-6.12.28-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28197200 May 24 2025 linux-rpi-16k-6.12.30-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 May 24 2025 linux-rpi-16k-6.12.30-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28760556 Jul 1 02:56 linux-rpi-16k-6.12.35-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jul 1 02:56 linux-rpi-16k-6.12.35-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28791096 Jul 17 05:14 linux-rpi-16k-6.12.38-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Jul 17 05:15 linux-rpi-16k-6.12.38-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28787492 Aug 1 21:53 linux-rpi-16k-6.12.41-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Aug 1 21:54 linux-rpi-16k-6.12.41-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28793132 Aug 20 05:45 linux-rpi-16k-6.12.42-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Aug 20 05:45 linux-rpi-16k-6.12.42-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28814312 Aug 27 05:50 linux-rpi-16k-6.12.43-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Aug 27 05:51 linux-rpi-16k-6.12.43-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28809992 Sep 7 00:39 linux-rpi-16k-6.12.45-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Sep 7 00:40 linux-rpi-16k-6.12.45-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28828792 Sep 16 01:07 linux-rpi-16k-6.12.46-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Sep 16 01:08 linux-rpi-16k-6.12.46-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28828760 Oct 25 06:07 linux-rpi-16k-6.12.55-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Oct 25 06:07 linux-rpi-16k-6.12.55-1-aarch64.pkg.tar.xz.sig
-rw-r--r-- 1 root root 28680676 Nov 25 04:34 linux-rpi-16k-6.12.59-1-aarch64.pkg.tar.xz
-rw-r--r-- 1 root root 566 Nov 25 04:34 linux-rpi-16k-6.12.59-1-aarch64.pkg.tar.xz.sig

as mentioned iam using a bottom hat pineberry with a crucial p3 500Gb

Also, I thought i'd try the upgrade on a usb boot RPi5 and it halts at activating swap

[graysky]

I experimented with a Geekworm X1001 a while ago but never tried booted from it. Since our config for linux-rpi-16k is upstream's bcm2712_defconfig plus some additional options, might be a good idea to report the boot failure:
make_clickable_callback(MAGIC_URL_FULL, '
', 'https://github.com/raspberrypi/linux/issues', '', ' class="postlink"')

What do you see on the screen when the boot hangs? Do you see the rainbow screen at least? What are the last few lines of dmesg output? Does it get far enough that something is logged to journalctl?

[bloke]

See this on boot of a usb boot - its the 4 raspberrys on screen up top and prior is the rainbow
[attachment=0]pi5boot.jpg[/attachment]

yes, i have journalctl files i'll get the one from the usb boot which matches this image and send in next post

[bloke]

please see attached.
getting late here in Oz

There must be other users out there booting off usb / nvme's ? This cant just be me. Has anyone else hit this issue?

[graysky]

You probably have enough to open an issue upstream.