Hi.
I managed to merge the 64bit RPI kernel and 64bit userland based on the Alarm-RPI3-64 package for the PI3B+.
$this->bbcode_second_pass_code('', '
Linux alarm64 4.19.76-rt25-sc1 #1 SMP PREEMPT RT Sun Oct 6 14:11:00 CEST 2019 aarch64 GNU/Linux
')
$this->bbcode_second_pass_code('', '
#file /usr/bin/ls
/usr/bin/ls: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=d3d0049a9af71d4be7a4dafff7d668b248b75319, for GNU/Linux 3.7.0, stripped
')
* I took the PI3 64bit image.
* removed /boot/Image* files, otherwise the mmc1 (boot) partition would have been too small
* /boot/overlays had to be created
* copied my crosscompiled 64bit kernel and associated files modules, .dtb(s) etc.
* copied cmdline and config.txt from my 32bit userland installation
Note: config.txt requires below lines to boot the RPI64bit kernel
$this->bbcode_second_pass_code('', '
kernel=kernel8.img
arm_64bit=1
')
* and finally rebooted
And that was it.
As you can see above I'm running my own 64-bit rt-kernel, which comes with a pretty much stripped down .config based on bcmrpi3_defconfig.
I'm running the PI headless, no fancy graphics or alike. I can't tell if all that "fancy" stuff works flawless.
My audio HAT and Wifi works - that'll do for me.

Great stuff.