All I did was add aarch64 to arch line:
$this->bbcode_second_pass_code('', 'arch=('x86_64' 'aarch64')')
Took ~2hrs to build on RockPro64 SATA SSD. Works fine.
To run it under Wayland (Sway in my case) you need sdl2-git until 2.0.17 comes out because of this bug (not related to CPU architecture).