How to exclude all unlisted Pages in sitemap.xml with the SiteMeta-Plugin

Kirby 3 Solved ✅
1

Hi all,

I use SiteMeta to get a sitemap.xml and have excluded some pages by template. that works fine.

i also would like to exclude all unlisted pages.
i did several attemps to config the plugin. none is working. i get some type errors.
does anyone know, what is the right way to add all unlisted pages to the exclude.pages-array?
is it even possible to access site() in config?

'meta' => [
  'exclude' => [
    //'pages' => site()->index()->unlisted(),
    'pages' => site()->index()->unlisted()->toArray(),
    //'pages' => 'site()->index()->unlisted()->toArray,
    'templates' => [
      'default',
      'error',
      'xyz'
    ]
  ]
],

as a workaround i just add listed to the foreach inside the plugin. that works,
but i don’t like to override the plugin itself, to still be able to require that by composer.

  //foreach (site()->index() as $item) {
  foreach (site()->index()->listed() as $item) {
      if (in_array($item->intendedTemplate()->name(), $excludeTemplates)) {
          continue;
      }

Thx!

2 
$exclude = site()->index()->unlisted()->pluck('id', ',');

Will give you an array with all ids of all unlisted pages

3

i added that line directly to the plugin:

$pages = site()->index()->unlisted()->pluck('id', ',');

...

foreach (site()->index() as $item) {
  if (in_array($item->intendedTemplate()->name(), $excludeTemplates)) {
    continue;
  }
  if (preg_match('/^(?:' . implode('|', $pages) . ')$/i', $item->id()))
    continue;
  }
 
  ...
}

unfortenetly that results in in the following error :man_shrugging:

grafik

grafik
grafik811×165 22.1 KB

if i var_dump the array, it takes > 3.5sec - for 266 mostly virtual pages

var_dump( site()->index()->unlisted()->pluck('id', ','))
4

I don’t understand why you need a costly preg_match here instead of testing for in_array(). but more importantly, why don’t you use that array in the config instead of adding the code in the plugin?

5

I don’t understand why you need a costly preg_match here instead of testing for in_array()

i did not add that preg_match-function. that’s already in the plugins code.

but more importantly, why don’t you use that array in the config instead of adding the code in the plugin?

i added that line in the config.php first, but there was no difference in pages in the sitemap.xml
still listed and unlisted pages were shown.
i dumped the options. dont know why, but pages was an empty array

config.php

'exclude' => [
  'pages' => site()->index()->unlisted()->pluck('id', ','),
  'pages_test' => ['test1', 'test2', 'test3'],
  'templates' => [
      'default',
      'error',

SiteMeta.php

$pages         = option('meta.exclude.pages', []);
$pagesTest     = option('meta.exclude.pages_test', []);
echo '<pre>' , var_dump($pages) , '</pre>';
echo '<pre>' , var_dump($pagesTest) , '</pre>';

Result

array(0) {
}
array(3) {
  [0]=>
  string(5) "test1"
  [1]=>
  string(5) "test2"
  [2]=>
  string(5) "test3"
}

is site()->index()->unlisted()->pluck('id', ',') already available in the config?
why is it an empty array?
it is not the options default-array. i can rule that out…
for further tests, i added the array directly into the plugin, but then i get the error from above.

independently of that issues:
on the subject of costs, i am totaly unsure, if it’s a good idea, to use a big array with atm around 250-350 unlisted pages to work as some kind of filter by preg_match, when i can easily get only the listed pages in the foreach by adding listed()

// foreach (site()->index() as $item) {
foreach (site()->index()->listed() as $item) {

Unfortenetly the SiteMeta-plugin does not provide an option for that, but others do… so i think about switching to another one is maybe a better choice with that amount of pages.

6

No, you would have to use that inside the ready config option because you don’t have access to the kirby or site object otherwise.

7

No, you would have to use that inside the ready config option because you don’t have access to the kirby or site object otherwise.

thx, i thought so already.
now i added that to the ready function, but that’s still weird:

the following is working.
the ‘page-xy’ is excluded in the sitemap.xml

// config.php
'ready' => function () {
    return [
        'meta' => [
            'exclude' => [
                'pages' => ['page-xy'],
                'templates' => [
                    'default',
                    'error',

if i add some subpage, i get the error i already get, when i use site()->index()… directly in the plugin:

// config.php
'ready' => function () {
    return [
        'meta' => [
            'exclude' => [
                'pages' => ['page-xy', 'page-xy/subpage'],
                'templates' => [
                    'default',
                    'error',

grafik
grafik1178×439 81.9 KB

if i use site()->index()… i get a database error.
lots of pages are virtual pages
if i comment out the exclude.pages config, the site works as expected.

// config.php

...

// Database
'db' => [
    'host'     => env('MYSQL_HOST', false),
    'database' => env('MYSQL_DATABASE', false),
    'user'     => env('MYSQL_ROOT_USER', false),
    'password' => env('MYSQL_ROOT_PASSWORD', false),
],

'ready' => function () {
    return [
        'meta' => [
            'exclude' => [
                'pages' => site()->index()->unlisted()->pluck('id', ','),
                'templates' => [
                    'default',
                    'error',

grafik
grafik1192×545 108 KB

8

Hm, I added this in my config now:

return [
    'debug' => true,
    'ready' => function () {
        return [
            'meta' => [
                'exclude' => [
                    'pages' => ['home', 'error', 'home/new-page-1'],
                    'templates' => [
                        'default',
                        'error',
                    ]
                ]
            ]
        ];
    }
];

And then in my home.php template, added a some simple test code:

<?php


$exclude =  option('meta.exclude.pages');
foreach (site()->index() as $item) {

  if (preg_match('!^(?:' . implode('|', $exclude) . ')$!i', $item->id())) {
      echo $item->title() . ': matches<br>';
  }
}

This works without any issues.

What does your $exclude array give you? Wonder if there are any fancy characters that should be escaped.

9

Thank you very much for your example. That solves another thing.
Unfortenetly still not the main-problem :frowning:

For the sake of simplicity, i told i use the SiteMeta-Plugin from Bastian.
But actually i use the Kirby-Extended-Plugin from Johann, that forks the Site-Meta from Bastian into a Group of Plugins. Besides some propertyNames, it really looks the same.
But there is unfortenetly one little diffenerence, i was not able to see!
It’s very little :innocent:

// Orininal from Bastian:
if (preg_match('!^(?:' . implode('|', $excludePages) . ')$!i', $item->id())) {

// Fork of Johann - dont know why he changed that, or maybe ist an older version??
if (preg_match('/^(?:' . implode('|', $excludePages) . ')$/i', $item->id())) {

In the fork, there is a slash, not an exclamation mark.
So, when i replace the line with your example, it also works with subpages now!

But i still have the database error,

when i add site()->index()... to the config!

i tried to simplify the config-ready-function.
i just add some simple options:

<?php

$base = dirname(__DIR__, 2);
\KirbyExtended\Env::load($base);

return [
    // Debug
    'debug' => env('KIRBY_DEBUG', false),

    // Database
    'db' => [
        'host'         => env('MYSQL_HOST', false),
        'database' => env('MYSQL_DATABASE', false),
        'user'         => env('MYSQL_USER', false),
        'password' => env('MYSQL_PASSWORD', false),
    ],
    
    ...

    // Ready
    'ready' => function () {
        return [
            'test' => 'test',
            'test2' => site()->title(),
            //'test3' => site()->index(),

that works as expected.
i can echo the site-title by option(‘test2’)

but if i comment in 'test3' => site()->index() i get the database error!

grafik
grafik1436×656 131 KB

WTF has this configuration to do, with my database-connection?
Is the ready function still to early to get virtual pages from DB?
If i echo site()->index() later, i.e. in the Sitemap-Plugin, it works like a charm…
By the way: for some reasons, this project is still running K3.5.8

10

Could you show me the complete stack trace for the DB error, please? No idea why this is happening, have you tried to debug this with xDebug? If not, could you provide a minimal example for testing?

11

PDOException thrown with message “SQLSTATE[HY000] [2002] No such file or directory”

Stacktrace:
#17 PDOException in /var/www/web/dev/vendor/getkirby/cms/src/Database/Database.php:211
#16 PDO:__construct in /var/www/web/dev/vendor/getkirby/cms/src/Database/Database.php:211
#15 Kirby\Database\Database:connect in /var/www/web/dev/vendor/getkirby/cms/src/Database/Database.php:153
#14 Kirby\Database\Database:__construct in /var/www/web/dev/vendor/getkirby/cms/src/Database/Db.php:59
#13 Kirby\Database\Db:connect in /var/www/web/dev/vendor/getkirby/cms/src/Database/Db.php:82
#12 Kirby\Database\Db:table in /var/www/web/dev/vendor/getkirby/cms/src/Database/Db.php:150
#11 Kirby\Database{closure} in /var/www/web/dev/vendor/getkirby/cms/src/Database/Db.php:126
#10 Kirby\Database\Db:__callStatic in /var/www/web/dev/site/models/voices.php:21
#9 VoicesPage:getDataFromDataBase in /var/www/web/dev/site/models/voices.php:16
#8 VoicesPage:__construct in /var/www/web/dev/vendor/getkirby/cms/src/Cms/Page.php:894
#7 Kirby\Cms\Page:model in /var/www/web/dev/vendor/getkirby/cms/src/Cms/Page.php:475
#6 Kirby\Cms\Page:factory in /var/www/web/dev/vendor/getkirby/cms/src/Cms/Pages.php:171
#5 Kirby\Cms\Pages:factory in /var/www/web/dev/vendor/getkirby/cms/src/Cms/HasChildren.php:43
#4 Kirby\Cms\Site:children in /var/www/web/dev/vendor/getkirby/cms/src/Cms/HasChildren.php:208
#3 Kirby\Cms\Site:index in /var/www/web/dev/site/config/config.php:117
#2 Kirby\Toolkit\F:{closure} in /var/www/web/dev/vendor/getkirby/cms/src/Cms/App.php:948
#1 Kirby\Cms\App:optionsFromReadyCallback in /var/www/web/dev/vendor/getkirby/cms/src/Cms/App.php:133
#0 Kirby\Cms\App:__construct in /var/www/web/dev/public/index.php:15

12

i am going to enable xDebug tmrw.

13

I has a closer look at the source code and noticed that the exclude.pages options cannot only take an array but a callable as argument. So instead of putting the call to site()->index() into the ready option, try and do it inside the callable

Here is the example code from the getkirby.com website:

<?php

return [
    'exclude' => [
        'pages' => function () {
            $pages = [];

            foreach (page('docs/reference/objects')->grandChildren() as $page) {
                if (ReferenceClassesPage::isFeatured($page->id()) === false) {
                    $pages[] = $page->id() . '.*';
                }
            }

            return $pages;
        },
        'templates' => [
            'error',
            'link',
            'reference-classes',
            'reference-shortlink',
            'separator'
        ]
    ],
];
14

Oh yeah, callable works!
No need for the config ready function.

As you said: That line returns all unlisted pages as an array

site()->index()->unlisted()->pluck('id', ',');

So that’s the final solution to exclude all unlisted pages in the sitemap.xml for SiteMeta-Plugin

return [
    // Debug
    'debug' => env('KIRBY_DEBUG', false),

    ...

    // Sitemap
    'meta' => [
      'exclude' => [
        'pages' => function () {
           // exclude all unlisted pages
           return site()->index()->unlisted()->pluck('id', ',');
         },
        'templates' => [
          'default',
          'error',
          'xyz'
        ]
    ]
],

In the kirby-extended-plugin i have to change the pregmatch-function. i will add an issue in their repo.

//if (preg_match('/^(?:' . implode('|', $excludePages) . ')$/i', $item->id())) {
if (preg_match('!^(?:' . implode('|', $excludePages) . ')$!i', $item->id())) {

Thank you very much again!!
Have a good day :sunny: