Looking in my system in /boot there is:
# ls /boot/
boot.scr boot.txt.orig Image initramfs-linux-fallback.img mkscr u-boot.itb
boot.txt dtbs Image.gz initramfs-linux.img rksd_loader.img
and the u-boot seems to use boot.scr that is generated from the text file boot.txt - which has
# cat /boot/boot.txt
# After modifying, run ./mkscr
# MAC address (use spaces instead of colons)
setenv macaddr da 19 c8 7a 6d f4
part uuid ${devtype} ${devnum}:${bootpart} uuid
setenv bootargs console=ttyS2,1500000 root=PARTUUID=${uuid} rw rootwait earlycon=uart8250,mmio32,0xff130000
setenv fdtfile rockchip/rk3328-rock64.dtb
if load ${devtype} ${devnum}:${bootpart} ${kernel_addr_r} /boot/Image; then
if load ${devtype} ${devnum}:${bootpart} ${fdt_addr_r} /boot/dtbs/${fdtfile}; then
fdt addr ${fdt_addr_r}
fdt resize
fdt set /ethernet@ff540000 local-mac-address "[${macaddr}]"
if load ${devtype} ${devnum}:${bootpart} ${ramdisk_addr_r} /boot/initramfs-linux.img; then
booti ${kernel_addr_r} ${ramdisk_addr_r}:${filesize} ${fdt_addr_r};
else
booti ${kernel_addr_r} - ${fdt_addr_r};
fi;
fi;
fi
which references the initramfs file - so simply removing the latter would likely lead to the system failing to boot - and given this is a pure headless system accessed by ssh only on a LAN, loss of boot means a complete re-install. So unless the boot process can be changed in a sure fire way that would not lose the ability to boot the kernel, then it wouldn't seem sensible to make critical changes without understanding the boot process in details.