The primary key thing can indeed become a problem sometimes. You could of course work around that but not too easily. Your friend in general is the new router. Redirections have become so easy with that. Especially for the conference site I’d really add redirections for speaker pages - maybe even dedicated urls for each speaker:
http://yourconference.com/speaker-name
and redirect that to wherever it makes sense. You could even write some small code which automatically looks for speakers by template for example and creates the redirect no matter where the speaker page is at the moment. That way you wouldn’t need to update it at all.
Basically what you get with Kirby is a hybrid between something entirely static and something database-driven. With an entirely static site you would have exactly the same issues and there’s hardly anything I can do about it. I guess there are always some pros and cons for those kinds of aspects and in the end it’s all about what is more important for your project or how easy is it to work around the issues.