NanoPi-T4 Can't update kernel

[nottledim]

I have 5.17.1-1-aarch64-ARCH which has to be pinned so it doesn't update. When I update to any of 5.18 up to current 6.2 it replaces Image and initramfs as it should but it won't boot. uboot just says starting kernel and sits and looks at me (i.e. no messages on system console). If I downgrade kernel to 5.17 again it all works again.

What could be changing between these versions which I'm not accounting for.

Kernel cmdline is
$this->bbcode_second_pass_code('', 'root=UUID=37d8b35a-6f23-489a-8867-d24aa2e78afd rw rootwait loglevel=1')

I'm booting off the emmc disk and wondering if there's a uboot (dts) config which needs to be changed.

[nottledim]

More info: I've increased log level. I'm not sure it helps but if anybody can suggest something.
$this->bbcode_second_pass_code('', '
Starting kernel ...

[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[ 0.000000] Linux version 6.2.1-1-aarch64-ARCH (builduser@leming) (aarch64-unknown-linux-gnu-gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT_DYNAMIC Sat Feb 25 15:18:35 MST 2023
[ 0.000000] Machine model: FriendlyElec NanoPC-T4
[ 0.000000] efi: UEFI not found.
[ 0.000000] earlycon: uart0 at MMIO32 0x00000000ff1a0000 (options '1500000n8')
[ 0.000000] printk: bootconsole [uart0] enabled
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000200000-0x00000000f7ffffff]
[ 0.000000] DMA32 empty
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000200000-0x00000000f7ffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000200000-0x00000000f7ffffff]
[ 0.000000] On node 0, zone DMA: 512 pages in unavailable ranges
[ 0.000000] cma: Reserved 64 MiB at 0x00000000ed400000
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv1.0 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
[ 0.000000] psci: SMC Calling Convention v1.0
[ 0.000000] percpu: Embedded 21 pages/cpu s46632 r8192 d31192 u86016
[ 0.000000] Detected VIPT I-cache on CPU0
[ 0.000000] CPU features: detected: GIC system register CPU interface
[ 0.000000] CPU features: detected: ARM erratum 845719
[ 0.000000] alternatives: applying boot alternatives
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 999432
[ 0.000000] Kernel command line: console= root=UUID=37d8b35a-6f23-489a-8867-d24aa2e78afd rw rootwait loglevel=7
[ 0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.000000] Memory: 3863444K/4061184K available (19328K kernel code, 4544K rwdata, 10224K rodata, 7552K init, 978K bss, 132204K reserved, 65536K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
[ 0.000000] Dynamic Preempt: full
[ 0.000000] rcu: Preemptible hierarchical RCU implementation.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=6.
[ 0.000000] Trampoline variant of Tasks RCU enabled.
[ 0.000000] Tracing variant of Tasks RCU enabled.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=6
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[ 0.000000] GICv3: 256 SPIs implemented
[ 0.000000] GICv3: 0 Extended SPIs implemented
[ 0.000000] Root IRQ handler: gic_handle_irq
[ 0.000000] GICv3: GICv3 features: 16 PPIs
[ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000fef00000
[ 0.000000] ITS [mem 0xfee20000-0xfee3ffff]
[ 0.000000] ITS@0x00000000fee20000: allocated 65536 Devices @480000 (flat, esz 8, psz 64K, shr 0)
[ 0.000000] ITS: using cache flushing for cmd queue
[ 0.000000] GICv3: using LPI property table @0x0000000000440000
[ 0.000000] GIC: using cache flushing for LPI property table
[ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000000450000
[ 0.000000] GICv3: GIC: PPI partition interrupt-partition-0[0] { /cpus/cpu@0[0] /cpus/cpu@1[1] /cpus/cpu@2[2] /cpus/cpu@3[3] }
[ 0.000000] GICv3: GIC: PPI partition interrupt-partition-1[1] { /cpus/cpu@100[4] /cpus/cpu@101[5] }
[ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.000000] arch_timer: cp15 timer(s) running at 24.00MHz (phys).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns
[ 0.000001] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns
[ 0.004179] Console: colour dummy device 80x25
[ 0.004667] Calibrating delay loop (skipped), value calculated using timer frequency.. 48.00 BogoMIPS (lpj=24000)
[ 0.005675] pid_max: default: 32768 minimum: 301
[ 0.006409] LSM: initializing lsm=capability,yama
[ 0.006907] Yama: becoming mindful.
[ 0.007541] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.008299] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.011741] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 0.013916] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.014636] cblist_init_generic: Setting shift to 3 and lim to 1.
[ 0.015416] cblist_init_generic: Setting shift to 3 and lim to 1.
[ 0.016447] rcu: Hierarchical SRCU implementation.
[ 0.016908] rcu: Max phase no-delay instances is 400.
[ 0.020337] Platform MSI: interrupt-controller@fee20000 domain created
[ 0.021528] PCI/MSI: /interrupt-controller@fee00000/interrupt-controller@fee20000 domain created
[ 0.022709] fsl-mc MSI: interrupt-controller@fee20000 domain created
[ 0.028804] EFI services will not be available.
[ 0.030470] smp: Bringing up secondary CPUs ...
[ 0.032084] Detected VIPT I-cache on CPU1
[ 0.032232] cacheinfo: Unable to detect cache hierarchy for CPU 1
[ 0.032251] GICv3: CPU1: found redistributor 1 region 0:0x00000000fef20000
[ 0.032273] GICv3: CPU1: using allocated LPI pending table @0x0000000000460000
[ 0.032336] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[ 0.033714] Detected VIPT I-cache on CPU2
[ 0.033846] cacheinfo: Unable to detect cache hierarchy for CPU 2
[ 0.033862] GICv3: CPU2: found redistributor 2 region 0:0x00000000fef40000
[ 0.033883] GICv3: CPU2: using allocated LPI pending table @0x0000000000470000
[ 0.033927] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[ 0.035260] Detected VIPT I-cache on CPU3
[ 0.035396] cacheinfo: Unable to detect cache hierarchy for CPU 3
[ 0.035413] GICv3: CPU3: found redistributor 3 region 0:0x00000000fef60000
[ 0.035433] GICv3: CPU3: using allocated LPI pending table @0x0000000000500000
[ 0.035474] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[ 0.036736] CPU features: detected: Spectre-v2
[ 0.036750] CPU features: detected: Spectre-v4
[ 0.036756] CPU features: detected: Spectre-BHB
[ 0.036764] CPU features: detected: ARM erratum 1742098
[ 0.036772] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[ 0.036778] Detected PIPT I-cache on CPU4
[ 0.036908] cacheinfo: Unable to detect cache hierarchy for CPU 4
[ 0.036924] GICv3: CPU4: found redistributor 100 region 0:0x00000000fef80000
[ 0.036942] GICv3: CPU4: using allocated LPI pending table @0x0000000000510000
[ 0.036988] CPU4: Booted secondary processor 0x0000000100 [0x410fd082]
[ 0.038309] Detected PIPT I-cache on CPU5
[ 0.038421] cacheinfo: Unable to detect cache hierarchy for CPU 5
[ 0.038436] GICv3: CPU5: found redistributor 101 region 0:0x00000000fefa0000
[ 0.038454] GICv3: CPU5: using allocated LPI pending table @0x0000000000520000
[ 0.038489] CPU5: Booted secondary processor 0x0000000101 [0x410fd082]
[ 0.038661] smp: Brought up 1 node, 6 CPUs
[ 0.056246] SMP: Total of 6 processors activated.
[ 0.056704] CPU features: detected: 32-bit EL0 Support
[ 0.057215] CPU features: detected: 32-bit EL1 Support
[ 0.057713] CPU features: detected: CRC32 instructions
[ 0.058356] CPU: All CPU(s) started at EL2
[ 0.058766] alternatives: applying system-wide alternatives
[ 0.063547] devtmpfs: initialized
[ 0.082453] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 0.083425] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[ 0.085735] pinctrl core: initialized pinctrl subsystem
[ 0.089586] DMI not present or invalid.
[ 0.091024] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.093835] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[ 0.095081] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[ 0.096288] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[ 0.097158] audit: initializing netlink subsys (disabled)
[ 0.097985] audit: type=2000 audit(0.082:1): state=initialized audit_enabled=0 res=1
[ 0.099741] thermal_sys: Registered thermal governor 'fair_share'
[ 0.099752] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.100356] thermal_sys: Registered thermal governor 'step_wise'
[ 0.100934] thermal_sys: Registered thermal governor 'user_space'
[ 0.101520] thermal_sys: Registered thermal governor 'power_allocator'
[ 0.102200] cpuidle: using governor ladder
[ 0.103265] cpuidle: using governor menu
[ 0.103940] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 0.104873] ASID allocator initialised with 65536 entries
[ 0.108291] Serial: AMBA PL011 UART driver
[ 0.152357] platform ff770000.syscon:phy@f780: Fixing up cyclic dependency with fe330000.mmc
[ 0.162428] platform ff940000.hdmi: Fixing up cyclic dependency with ff8f0000.vop
[ 0.163233] platform ff940000.hdmi: Fixing up cyclic dependency with ff900000.vop
[ 0.175354] gpio gpiochip0: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.176806] rockchip-gpio ff720000.gpio: probed /pinctrl/gpio@ff720000
[ 0.178131] gpio gpiochip1: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.179337] rockchip-gpio ff730000.gpio: probed /pinctrl/gpio@ff730000
[ 0.180631] gpio gpiochip2: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.181843] rockchip-gpio ff780000.gpio: probed /pinctrl/gpio@ff780000
[ 0.183050] gpio gpiochip3: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.184240] rockchip-gpio ff788000.gpio: probed /pinctrl/gpio@ff788000
[ 0.185363] gpio gpiochip4: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.186539] rockchip-gpio ff790000.gpio: probed /pinctrl/gpio@ff790000
[ 0.200995] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.201672] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 0.202312] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages
[ 0.202973] HugeTLB: 508 KiB vmemmap can be freed for a 32.0 MiB page
[ 0.203595] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.204253] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.204866] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages
[ 0.205525] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page
[ 0.206888] cryptd: max_cpu_qlen set to 1000
[ 0.225001] raid6: neonx8 gen() 1784 MB/s
[ 0.242174] raid6: neonx4 gen() 1741 MB/s
[ 0.259340] raid6: neonx2 gen() 1448 MB/s
[ 0.276513] raid6: neonx1 gen() 1050 MB/s
[ 0.293680] raid6: int64x8 gen() 1013 MB/s
[ 0.310840] raid6: int64x4 gen() 984 MB/s
[ 0.328027] raid6: int64x2 gen() 963 MB/s
[ 0.345192] raid6: int64x1 gen() 733 MB/s
[ 0.345612] raid6: using algorithm neonx8 gen() 1784 MB/s
[ 0.362346] raid6: .... xor() 1237 MB/s, rmw enabled
[ 0.362835] raid6: using neon recovery algorithm
[ 0.364592] ACPI: Interpreter disabled.
[ 0.373377] iommu: Default domain type: Translated
[ 0.373854] iommu: DMA domain TLB invalidation policy: strict mode
[ 0.375554] SCSI subsystem initialized
[ 0.376454] usbcore: registered new interface driver usbfs
[ 0.377034] usbcore: registered new interface driver hub
[ 0.377586] usbcore: registered new device driver usb
[ 0.379654] pps_core: LinuxPPS API ver. 1 registered
[ 0.380147] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.381055] PTP clock support registered
[ 0.381718] EDAC MC: Ver: 3.0.0
[ 0.384820] FPGA manager framework
[ 0.385260] Advanced Linux Sound Architecture Driver Initialized.
[ 0.386781] NetLabel: Initializing
[ 0.387129] NetLabel: domain hash size = 128
[ 0.387550] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO
[ 0.388176] NetLabel: unlabeled traffic allowed by default
[ 0.389175] vgaarb: loaded
[ 0.389864] clocksource: Switched to clocksource arch_sys_counter
[ 0.390911] VFS: Disk quotas dquot_6.6.0
[ 0.391339] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.392373] pnp: PnP ACPI: disabled
[ 0.406842] NET: Registered PF_INET protocol family
[ 0.407666] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.413253] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.414236] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.415015] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.416136] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 0.418143] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.419150] MPTCP token hash table entries: 4096 (order: 4, 98304 bytes, linear)
[ 0.420034] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.420814] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.421880] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.423162] RPC: Registered named UNIX socket transport module.
[ 0.423738] RPC: Registered udp transport module.
[ 0.424221] RPC: Registered tcp transport module.
[ 0.424676] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.425318] PCI: CLS 0 bytes, default 64
[ 0.426350] Trying to unpack rootfs image as initramfs...
[ 0.431112] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available
[ 0.432558] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[ 0.434232] kvm [1]: IPA Size Limit: 40 bits
[ 0.437326] kvm [1]: vgic-v2@fff20000
[ 0.437713] kvm [1]: GIC system register CPU interface enabled
[ 0.438627] kvm [1]: vgic interrupt IRQ18
[ 0.439396] kvm [1]: Hyp mode initialized successfully
[ 0.442897] Initialise system trusted keyrings
[ 0.443692] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[ 0.444459] zbud: loaded
[ 0.446358] NFS: Registering the id_resolver key type
[ 0.446916] Key type id_resolver registered
[ 0.447324] Key type id_legacy registered
[ 0.447755] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[ 0.448422] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[ 0.449174] ntfs3: Max link count 4000
[ 0.449540] ntfs3: Read-only LZX/Xpress compression included
[ 0.450379] SGI XFS with ACLs, security attributes, quota, no debug enabled
[ 0.517038] NET: Registered PF_ALG protocol family
[ 0.517523] xor: measuring software checksum speed
[ 0.521396] 8regs : 2899 MB/sec
[ 0.524990] 32regs : 3113 MB/sec
[ 0.529749] arm64_neon : 2273 MB/sec
[ 0.530181] xor: using function: 32regs (3113 MB/sec)
[ 0.530675] Key type asymmetric registered
[ 0.531083] Asymmetric key parser 'x509' registered
[ 0.531671] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)
[ 0.532737] io scheduler mq-deadline registered
[ 0.533193] io scheduler kyber registered
[ 0.533655] io scheduler bfq registered
[ 0.583796] IPMI message handler: version 39.2
[ 0.618463] dma-pl330 ff6d0000.dma-controller: Loaded driver for PL330 DMAC-241330
[ 0.619217] dma-pl330 ff6d0000.dma-controller: DBUFF-32x8bytes Num_Chans-6 Num_Peri-12 Num_Events-12
[ 0.622087] dma-pl330 ff6e0000.dma-controller: Loaded driver for PL330 DMAC-241330
[ 0.622825] dma-pl330 ff6e0000.dma-controller: DBUFF-128x8bytes Num_Chans-8 Num_Peri-20 Num_Events-16
[ 0.646253] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.650787] ff180000.serial: ttyS0 at MMIO 0xff180000 (irq = 37, base_baud = 1500000) is a 16550A
[ 0.651892] serial serial0: tty port ttyS0 registered
[ 0.653686] ff1a0000.serial: ttyS2 at MMIO 0xff1a0000 (irq = 38, base_baud = 1500000) is a 16550A

')
On the 5.17 system it prints to the ttys and and then loads USB stuff
On 6.2 just stops, no errors.

[nottledim]

Resolved: Some idiot (i.e. me) didn't have console specified in boot args. It seems to matter.

$this->bbcode_second_pass_code('', '
console=ttyS2,1500000 root=UUID=37d8b35a-6f23-489a-8867-d24aa2e78afd rw rootwait loglevel=7
')