What became clear to me is that these issues occurs when ran on PHPโs built-in server as I have setup. I followed the steps on Kirbyโs โGet up and runningโ page to have it working.
โphp -S localhost:8080 kirby/router.phpโ
When I upload it to my server or run it via MAMP, I have no issues (and thus no 404).
Im not technically equipped (yet) to understand what is causes these issues. But maybe this helps to find the solution as in 2024 this issue still pertains.