No USB detected on Raspberry Pi 4 with aarch64

[pierrec]

Hi guys, great finding.
Which revision do you use for bcm2711-rpi-4-b.dtb? I haven't found one that works for me yet (RPi 4B 8GB too).
The link https://github.com/raspberrypi/firmware ... aster/boot points to master which is regularly updated.

I tried the current master (and a few older revisions) but didn't work for me
I used
$this->bbcode_second_pass_code('', '
# curl -L https://github.com/raspberrypi/firmware/raw/master/boot/bcm2711-rpi-4-b.dtb -o /boot/bcm2711-rpi-4-b.dtb
# md5sum /boot/bcm2711-rpi-4-b.dtb
915076abebc8a1c1d6b3c97a230b599f /boot/bcm2711-rpi-4-b.dtb
# uname -a
Linux alarm 5.8.9-2-ARCH #1 SMP Tue Sep 15 00:48:59 UTC 2020 aarch64 GNU/Linux
')

Still getting:
$this->bbcode_second_pass_code('', '
[ 22.136091] xhci_hcd 0000:01:00.0: can't setup: -110
[ 22.144343] xhci_hcd 0000:01:00.0: USB bus 1 deregistered
[ 22.153413] xhci_hcd 0000:01:00.0: init 0000:01:00.0 fail, -110
')

I see there are some dtb's installed by the kernel package as well:
$this->bbcode_second_pass_code('', '
# pacman -Qo /boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb
/boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb is owned by linux-aarch64 5.8.9-2
')
I'm not sure which is used

[pierrec]

USB doesn't seem to be work at uboot stage either, "Hit any key to stop autoboot" at the very start.
So I added an echo to print fdtfile in my boot.txt in order to check which dtb is loaded
https://archlinuxarm.org/packages/aarch ... oot.txt.v3
$this->bbcode_second_pass_code('', '
...
setenv bootargs console=ttyS1,115200 console=tty0 root=PARTUUID=${uuid} rw rootwait smsc95xx.macaddr="${usbethaddr}"
echo "fdtfile ${fdtfile}"

if load ${devtype} ${devnum}:${bootpart} ${kernel_addr_r} /Image; then
if load ${devtype} ${devnum}:${bootpart} ${fdt_addr_r} /dtbs/${fdtfile}; then
...
')
And it prints broadcom/bcm2711-rpi-4-b.dtb so this means the dtb is loaded from /boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb .
This one is provided by the kernel package
$this->bbcode_second_pass_code('', '
# pacman -Qo /boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb
/boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb is owned by linux-aarch64 5.8.9-2
')
I tried to update this one manually from github.com/raspberrypi/firmware but didn't make usb working for me

[pierrec]

hmmmm finally got it working , I took ubuntu-20.04.1-preinstalled-server-arm64+raspi.img.xz, extracted its /boot and copied several things like *.dat *.elf *.dtb, I'll have to figure out which file fixed it

Edit: I'm not familiar with the rpi boot process but even though uboot loads the linux kernel with /boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb an earlier boot stage requires /boot/bcm2711-rpi-4-b.dtb to exist. The rpi doesn't start if it doesn't exist.

So far I've only been able to make usb working with files taken from other distributions (with ubuntu-20.04.1-preinstalled-server-arm64+raspi.img.xz there). Both the .dtb file and {.dat,.elf} need to be changed in my case.
$this->bbcode_second_pass_code('', '
$ md5sum /boot/*.{dtb,dat,elf}
1a057460a7dde1c64554deaa48282015 /boot/bcm2711-rpi-4-b.dtb
55ce4d97cccb5bb2812a5e761ccde0c7 /boot/fixup.dat
adf4bea7f82b3885a28ad2d2eee7aa47 /boot/fixup4.dat
e8791f73b80dca05a94114e77977bd4a /boot/fixup4cd.dat
de906b2ab58f5618fcfc59485973c1be /boot/fixup4db.dat
1e92094b1b83755e78d60fd06f33cf42 /boot/fixup4x.dat
e8791f73b80dca05a94114e77977bd4a /boot/fixup_cd.dat
b84e89eb3e8bbbe6fc61cc4e7ec9a634 /boot/fixup_db.dat
b1762abe51ea08086d3a253dfadfeb01 /boot/fixup_x.dat
6d0c89fa489c569ef24b7be5b911ec43 /boot/start.elf
fc7e4f36e378ab6ac18f5e09b9746b95 /boot/start4.elf
28f6b0c343045182251e473538cbb3c1 /boot/start4cd.elf
3a36db27502ad78b44713889dae83439 /boot/start4db.elf
8323ce5c94818adf12dc53da6427da4a /boot/start4x.elf
2ea65c4440be09b98d967eff0444b774 /boot/start_cd.elf
6338c0f233c43bc5c09c7c36d2168a7b /boot/start_db.elf
13d2f173a4056c468682b6bb41addfe8 /boot/start_x.elf
')
I'll leave it like that for now... I've spent enough time on it so far

[Fincer]

Thanks pierrec!

Your solution solves the issue for me, too. Now I have Raspberry Pi 4B 8GB using aarch64, USB 2 & 3 ports functioning and whole 8GB RAM usable. I have not tested USB devices in a long run but it looks good so far, as seen:

$this->bbcode_second_pass_code('', '
[alarm@alarm ~]$ uname -a
Linux alarm 5.10.0-rc5-1-v7l #1 SMP Fri Dec 18 13:44:01 UTC 2020 aarch64 GNU/Linux
[alarm@alarm ~]$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 152d:0578 JMicron Technology Corp. / JMicron USA Technology Corp. JMS578 SATA 6Gb/s
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
[alarm@alarm ~]$ sudo dmesg | grep xhci_hcd
[ 1.376051] xhci_hcd 0000:01:00.0: xHCI Host Controller
[ 1.376101] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1
[ 1.376716] xhci_hcd 0000:01:00.0: hcc params 0x002841eb hci version 0x100 quirks 0x0000018000000890
[ 1.379074] xhci_hcd 0000:01:00.0: xHCI Host Controller
[ 1.379106] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2
[ 1.379137] xhci_hcd 0000:01:00.0: Host supports USB 3.0 SuperSpeed
[ 1.718512] usb 1-1: new high-speed USB device number 2 using xhci_hcd
[ 2.166506] usb 1-1.3: new high-speed USB device number 3 using xhci_hcd
[alarm@alarm ~]$ free -hm
total used free shared buff/cache available
Mem: 7.6Gi 60Mi 7.4Gi 0.0Ki 126Mi 7.5Gi
Swap: 0B 0B 0B
[alarm@alarm ~]$ cat /proc/cpuinfo | tail -4

Hardware : BCM2835
Serial : 1000000084abd797
Model : Raspberry Pi 4 Model B
[alarm@alarm ~]$
')

More precisely, replacing files bcm2711-rpi-4-b.dtb, start4.elf and fixup4.dat was enough for my headless RPi 4B, at least for USB 3.

This made me wonder what is the Ubuntu specific code which make USB work? Therefore, I compared two kernel device tree files:

- github.com/raspberrypi/linux - bcm2711-rpi-4-b.dts
- arch/arm/boot/dts/bcm2711-rpi-4-b.dts, extracted from linux-raspi-5.4_5.4.0.orig.tar.gz

The only noticiable difference is the following missing line which is present in Ubuntu kernel archive (kernel file: arch/arm/boot/dts/bcm2711-rpi-4-b.dts):
$this->bbcode_second_pass_code('', '
#include "bcm283x-rpi-usb-peripheral.dtsi"
')

The referred dtsi file is not used (or present) in every Raspberry Pi kernel archives, at least, although it exists in the Raspberry Pi kernel Github repository file tree itself.

Instead of Arch Linux ARM kernel linux-aarch64, I use Raspberry Pi kernel 5.10.0-rc5-1-v7l (aarch64/ARM64, link), compiled with a custom PKGBUILD file, and with the default bcm2711_defconfig kernel configuration provided by Raspberry Pi kernel Github repository.

I decided to re-compile the kernel once again, applying the following patch:

$this->bbcode_second_pass_code('', '
--- a/arch/arm/boot/dts/bcm2711-rpi-4-b.dts 2020-12-18 13:20:23.973176269 +0000
+++ b/arch/arm/boot/dts/bcm2711-rpi-4-b.dts 2020-12-18 13:20:51.072542212 +0000
@@ -2,6 +2,7 @@
/dts-v1/;
#include "bcm2711.dtsi"
#include "bcm2835-rpi.dtsi"
+#include "bcm283x-rpi-usb-peripheral.dtsi"

#include <dt-bindings/reset/raspberrypi,firmware-reset.h>

--- /dev/null 2020-12-15 00:20:37.047999998 +0000
+++ a/arch/arm/boot/dts/bcm283x-rpi-usb-peripheral.dtsi 2020-12-18 13:25:45.090219029 +0000
@@ -0,0 +1,7 @@
+// SPDX-License-Identifier: GPL-2.0
+&usb {
+ dr_mode = "peripheral";
+ g-rx-fifo-size = <256>;
+ g-np-tx-fifo-size = <32>;
+ g-tx-fifo-size = <256 256 512 512 512 768 768>;
+};
')

The latter patch section is needed only if the file bcm283x-rpi-usb-peripheral.dtsi is missing in downloaded Raspberry Pi kernel archive, as it was in my case.

Basically, the patch covers missing code portions of device tree file bcm2711-rpi-4-b.dtb. It does not cover elf/dat files, which must be taken care separately as explained below.

Needed elf/dat replacement files can be found, for instance, in Ubuntu package linux-firmware-raspi2 (more precicely: archive linux-firmware-raspi2_2.orig-boot.tar.gz), link: packages.ubuntu.com - linux-firmware-raspi2.

On Arch Linux ARM, package raspberrypi-bootloader provides official raspberrypi/firmware elf/dat files which are not suitable so we need ones provided by Ubuntu. I simply customized raspberrypi-bootloader PKGBUILD to use Ubuntu source archive instead.

--------------------

For those who wish to extract required files directly from Ubuntu image file: ubuntu-20.04.1-preinstalled-server-arm64+raspi.img.xz.

[pierrec]

Thank you for the follow up @Fincer

So what are you using for:
* /boot/bcm2711-rpi-4-b.dtb: which is used by the first stages of boot (as far as I know)
normally provided by uboot-raspberrypi
* /boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb: which is used by the kernel
provided by linux-aarch64
?

In my case /boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb seems fine:
$this->bbcode_second_pass_code('', '
$ fdtdump /boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb | grep -w 'usb@.*{' -A 20
usb@7e980000 {
compatible = "brcm,bcm2835-usb";
reg = <0x7e980000 0x00010000>;
interrupts = <0x00000000 0x00000049 0x00000004>;
#address-cells = <0x00000001>;
#size-cells = <0x00000000>;
clocks = <0x00000011>;
clock-names = "otg";
phys = <0x00000012>;
phy-names = "usb2-phy";
power-domains = <0x0000000b 0x00000006>;
dr_mode = "peripheral";
g-rx-fifo-size = <0x00000100>;
g-np-tx-fifo-size = <0x00000020>;
g-tx-fifo-size = <0x00000100 0x00000100 0x00000200 0x00000200 0x00000200 0x00000300 0x00000300>;
phandle = <0x0000007e>;
};
')

But /boot/bcm2711-rpi-4-b.dtb seems really outdated.
https://archlinuxarm.org/packages/aarch ... s/PKGBUILD
It uses https://github.com/raspberrypi/firmware ... 758b3/boot which is 11 month old
$this->bbcode_second_pass_code('', 'fdtdump /boot/bcm2711-rpi-4-b.dtb | grep -w 'usb@.*{' -A 20
usb@7e980000 {
compatible = "brcm,bcm2708-usb";
reg = <0x7e980000 0x00010000 0x7e00b200 0x00000200>;
interrupts = <0x00000000 0x00000049 0x00000004 0x00000000 0x00000028 0x00000004>;
#address-cells = <0x00000001>;
#size-cells = <0x00000000>;
clocks = <0x00000017>;
clock-names = "otg";
phys = <0x00000018>;
phy-names = "usb2-phy";
power-domains = <0x00000012 0x00000006>;
interrupt-names = "usb", "soft";
status = "disabled";
phandle = <0x000000ad>;
};
')

----

Actually now that I think of it, I tried to copy /boot/dtbs/broadcom/bcm2711-rpi-4-b.dtb to /boot/bcm2711-rpi-4-b.dtb and it runs fine. I had tried that originally but I also had the wrong *.{elf,dat} so it didn't work.
I think we can fix all our issues by just updating raspberrypi-bootloader in the end
Edit: raspberrypi-bootloader also needs to be updated for the *.{elf,dat}

[pierrec]

Hmmm actually there's already a pull request to update uboot-raspberrypi https://github.com/archlinuxarm/PKGBUILDs/pull/1831 of October
I'll try this one, and maybe even update it more
Also raspberrypi-bootloader has been updated yesterday https://archlinuxarm.org/packages/any/r ... loader/log so this one is ok

[pierrec]

I put my changes in https://github.com/piec/PKGBUILDs/commits/master
I do 2 things:
1. update uboot-raspberrypi: https://github.com/piec/PKGBUILDs/commi ... 91e7262dd1
to use the latest .dtb

2. revert raspberrypi-bootloader: https://github.com/piec/PKGBUILDs/commi ... 4c182cd021
the latest version packaged in archlinuxarm is too recent, I found that the most recent .{elf,dat} are broken
I tried the following revisions of https://github.com/raspberrypi/firmware/commits/master:
* e15ef4e4fe8be99cd816cec901d977224a1eb07e ok "firmware: xhci: Don't reset BCM2711 XHCI from filesys in start.elf "
* 08ed7a0c9ad4d9db559aaec462520ab435c7ce1c ok "firmware: Switch DA9121 PMIC to PWM mode when ARM > 600 MHz"
* 919aee0ed75f7db48a38b8b96c13228a7584cfd7 bad "kernel: Bump to 5.4.81"
* cfb5b76811f853612bc89c76da77d3d74d5f22ca bad "kernel: gzip the 64-bit kernel"

[graysky]

You guys are using the linux-aarch64 package which is built off mainline kernel. If you want to try to use a kernel package that's based off rpi-5.10.y (official raspberrypi foundation fork), I have been building one for a while now. You can either build your own or hit my repo directly.

Instructions for accessing the repo: http://repo-ck.com/aarch64-test/
My PKGBUILD repo: https://github.com/graysky2/PKGBUILDs/commits/5.10.y

The advantage (or disadvantage depending on your perspective) of using this kernel is you get all of the rpi foundation driver/libs. I have not seen any issues. USB works for me.

[pierrec]

Hi graysky, ok thanks for your answer I'll try that as well.
By the way which model of RPI 4 do you use?

[graysky]

I believe there is only one model, the 4B. Tested on the 8 and 4 GB versions.