Yea I figured that you suggested for me to remove that "gateway=" and I did, but it didn't work. Shortly after that I found that rogue config file and removed it.
I see what you're saying but it seems that the way it works is the gateway= thing is purely for the rpi's physical connection and that is information that is only used for networkd to set the wlan0 adapter IP. Devices connecting to the ap receive their gateway information from the "option routers" line in dhcpd.conf. In the documentation, it says that this let's devices know which IP address is the gateway to the internet. Given there is no internet connection being bridged, I assume this is just sent to the clients and sits in the dhcp connection config on the client. Either way, the only complaint from the devices I get is that there is no internet connection. It works for the purposes I want now though so thanks again.