What became clear to me is that these issues occurs when ran on PHP’s built-in server as I have setup. I followed the steps on Kirby’s “Get up and running” page to have it working.
“php -S localhost:8080 kirby/router.php”
When I upload it to my server or run it via MAMP, I have no issues (and thus no 404).
Im not technically equipped (yet) to understand what is causes these issues. But maybe this helps to find the solution as in 2024 this issue still pertains.