I solved it. It was not that hard. Just needed to write down the question here to activate my brain cells.
if ( $page->hasPrev() ) {
$prev = $page->prev()->url();
} else {
$prev = $page->parents()->children()->last()->url();
}
if ( $page->hasNext() ) {
$next = $page->next()->url();
} else {
$next = $page->parents()->children()->first()->url();
}