PHP 8 support was introduced with Kirby 4.6.0.
Also, I’d try another PHP version which might run another ImageMagick version.
It would also make sense to run the convert command directly on the command line to check if there are any issues with webp or avif conversion. While your version is diffferent from the one here, you never know: https://forum.getkirby.com/t/imagemagik-not-converting-processing-images/