I found this thread that seems pretty recent with the same issues I was having.
https://forums.raspberrypi.com/viewtopic.php?p=1933863
I tried pulseaudio with KMS and I had the same issue someone in that thread was having, with loud static and noise. I'll just stick to FKMS, since it at least works for what I need.