# How to exclude all unlisted Pages in sitemap.xml with the SiteMeta-Plugin

**URL:** <https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397>\
**Category:** Questions\
**Tags:** v3\
**Created:** [May 3, 2022, 5:59pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397 "2022-05-03T17:59:12Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![dennisb](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/dennisb/32/11676_2.png) [@dennisb](https://forum.getkirby.com/u/dennisb)\
**Post date:** [May 3, 2022, 5:59pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/1 "2022-05-03T17:59:12Z")

</div>

Hi all,

I use [SiteMeta](https://github.com/getkirby/getkirby.com/blob/main/site/plugins/meta/src/SiteMeta.php) to get a sitemap.xml and have excluded some pages by template. that works fine.

i also would like to `exclude all unlisted pages`.  
i did several attemps to config the plugin. none is working. i get some type errors.  
does anyone know, what is the right way to add `all unlisted pages` to the exclude.pages-array?  
is it even possible to access site() in config?

```auto
'meta' => [
  'exclude' => [
    //'pages' => site()->index()->unlisted(),
    'pages' => site()->index()->unlisted()->toArray(),
    //'pages' => 'site()->index()->unlisted()->toArray,
    'templates' => [
      'default',
      'error',
      'xyz'
    ]
  ]
],

```

as a workaround i just add listed to the foreach inside the plugin. that works,  
but i don’t like to override the plugin itself, to still be able to require that by composer.

```auto
  //foreach (site()->index() as $item) {
  foreach (site()->index()->listed() as $item) {
      if (in_array($item->intendedTemplate()->name(), $excludeTemplates)) {
          continue;
      }

```

Thx!

---

<div class="post-metadata">

**Author:** ![texnixe](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/texnixe/32/5754_2.png) [@texnixe](https://forum.getkirby.com/u/texnixe)\
**Post date:** [May 3, 2022, 6:12pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/2 "2022-05-03T18:12:24Z")

</div>

```auto
$exclude = site()->index()->unlisted()->pluck('id', ',');

```

Will give you an array with all ids of all unlisted pages

---

<div class="post-metadata">

**Author:** ![dennisb](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/dennisb/32/11676_2.png) [@dennisb](https://forum.getkirby.com/u/dennisb)\
**Post date:** [May 3, 2022, 7:58pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/3 "2022-05-03T19:58:59Z")

</div>

i added that line directly to the plugin:

```auto
$pages = site()->index()->unlisted()->pluck('id', ',');

...

foreach (site()->index() as $item) {
  if (in_array($item->intendedTemplate()->name(), $excludeTemplates)) {
    continue;
  }
  if (preg_match('/^(?:' . implode('|', $pages) . ')$/i', $item->id()))
    continue;
  }
 
  ...
}

```

unfortenetly that results in in the following error 🤷‍♂️

![grafik](https://europe1.discourse-cdn.com/flex017/uploads/getkirby/original/2X/8/827eca3194b0e238bffccb6166bebfece618618f.png)

 ![grafik](https://europe1.discourse-cdn.com/flex017/uploads/getkirby/original/2X/c/c122b332194398104c44fa1cd7bde52e3dce3ca2.png)

if i var\_dump the array, it takes \> 3.5sec - for 266 mostly virtual pages

```auto
var_dump( site()->index()->unlisted()->pluck('id', ','))

```

---

<div class="post-metadata">

**Author:** ![texnixe](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/texnixe/32/5754_2.png) [@texnixe](https://forum.getkirby.com/u/texnixe)\
**Post date:** [May 3, 2022, 8:14pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/4 "2022-05-03T20:14:13Z")

</div>

I don’t understand why you need a costly preg\_match here instead of testing for `in_array()`. but more importantly, why don’t you use that array in the config instead of adding the code in the plugin?

---

<div class="post-metadata">

**Author:** ![dennisb](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/dennisb/32/11676_2.png) [@dennisb](https://forum.getkirby.com/u/dennisb)\
**Post date:** [May 3, 2022, 9:02pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/5 "2022-05-03T21:02:16Z")

</div>

> I don’t understand why you need a costly preg\_match here instead of testing for `in_array()`

i did not add that preg\_match-function. that’s already in the plugins code.

> but more importantly, why don’t you use that array in the config instead of adding the code in the plugin?

i added that line in the config.php first, but there was no difference in pages in the sitemap.xml  
still listed and unlisted pages were shown.  
i dumped the options. dont know why, but pages was an empty array

config.php

```auto
'exclude' => [
  'pages' => site()->index()->unlisted()->pluck('id', ','),
  'pages_test' => ['test1', 'test2', 'test3'],
  'templates' => [
      'default',
      'error',

```

SiteMeta.php

```auto
$pages = option('meta.exclude.pages', []);
$pagesTest = option('meta.exclude.pages_test', []);
echo '<pre>' , var_dump($pages) , '</pre>';
echo '<pre>' , var_dump($pagesTest) , '</pre>';

```

Result

```auto
array(0) {
}
array(3) {
  [0]=>
  string(5) "test1"
  [1]=>
  string(5) "test2"
  [2]=>
  string(5) "test3"
}

```

is `site()->index()->unlisted()->pluck('id', ',')` already available in the config?  
why is it an empty array?  
it is not the options default-array. i can rule that out…  
for further tests, i added the array directly into the plugin, but then i get the error from above.

**independently of that issues:**  
on the subject of costs, i am totaly unsure, if it’s a good idea, to use a big array with atm around 250-350 unlisted pages to work as some kind of filter by preg\_match, when i can easily get only the listed pages in the foreach by adding listed()

```auto
// foreach (site()->index() as $item) {
foreach (site()->index()->listed() as $item) {

```

Unfortenetly the SiteMeta-plugin does not provide an option for that, but others do… so i think about switching to another one is maybe a better choice with that amount of pages.

---

<div class="post-metadata">

**Author:** ![texnixe](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/texnixe/32/5754_2.png) [@texnixe](https://forum.getkirby.com/u/texnixe)\
**Post date:** [May 3, 2022, 9:52pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/6 "2022-05-03T21:52:02Z")

</div>

> [@dennisb](#):
>
> is `site()->index()->unlisted()->pluck('id', ',')` already available in the config?  
> why is it an empty array?

No, you would have to use that inside the `ready` config option because you don’t have access to the kirby or site object otherwise.

---

<div class="post-metadata">

**Author:** ![dennisb](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/dennisb/32/11676_2.png) [@dennisb](https://forum.getkirby.com/u/dennisb)\
**Post date:** [May 4, 2022, 9:43am UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/7 "2022-05-04T09:43:06Z")

</div>

> No, you would have to use that inside the `ready` config option because you don’t have access to the kirby or site object otherwise.

thx, i thought so already.  
now i added that to the ready function, but that’s still weird:

the following is working.  
the ‘page-xy’ is excluded in the sitemap.xml

```auto
// config.php
'ready' => function () {
    return [
        'meta' => [
            'exclude' => [
                'pages' => ['page-xy'],
                'templates' => [
                    'default',
                    'error',

```

if i add some subpage, i get the error i already get, when i use site()-\>index()… directly in the plugin:

```auto
// config.php
'ready' => function () {
    return [
        'meta' => [
            'exclude' => [
                'pages' => ['page-xy', 'page-xy/subpage'],
                'templates' => [
                    'default',
                    'error',

```

 ![grafik](https://europe1.discourse-cdn.com/flex017/uploads/getkirby/original/2X/e/e05b5ce5bc474eb5e45730300340aaf4e085f332.png)

if i use site()-\>index()… i get a database error.  
lots of pages are virtual pages  
if i comment out the exclude.pages config, the site works as expected.

```auto
// config.php

...

// Database
'db' => [
    'host' => env('MYSQL_HOST', false),
    'database' => env('MYSQL_DATABASE', false),
    'user' => env('MYSQL_ROOT_USER', false),
    'password' => env('MYSQL_ROOT_PASSWORD', false),
],

'ready' => function () {
    return [
        'meta' => [
            'exclude' => [
                'pages' => site()->index()->unlisted()->pluck('id', ','),
                'templates' => [
                    'default',
                    'error',

```

 ![grafik](https://europe1.discourse-cdn.com/flex017/uploads/getkirby/original/2X/c/c9051f020f2f449f62108b034941afe861f998f8.png)

---

<div class="post-metadata">

**Author:** ![texnixe](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/texnixe/32/5754_2.png) [@texnixe](https://forum.getkirby.com/u/texnixe)\
**Post date:** [May 4, 2022, 11:44am UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/8 "2022-05-04T11:44:55Z")

</div>

Hm, I added this in my config now:

```auto
return [
    'debug' => true,
    'ready' => function () {
        return [
            'meta' => [
                'exclude' => [
                    'pages' => ['home', 'error', 'home/new-page-1'],
                    'templates' => [
                        'default',
                        'error',
                    ]
                ]
            ]
        ];
    }
];

```

And then in my home.php template, added a some simple test code:

```auto
<?php

$exclude = option('meta.exclude.pages');
foreach (site()->index() as $item) {

  if (preg_match('!^(?:' . implode('|', $exclude) . ')$!i', $item->id())) {
      echo $item->title() . ': matches<br>';
  }
}

```

This works without any issues.

What does your $exclude array give you? Wonder if there are any fancy characters that should be escaped.

---

<div class="post-metadata">

**Author:** ![dennisb](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/dennisb/32/11676_2.png) [@dennisb](https://forum.getkirby.com/u/dennisb)\
**Post date:** [May 4, 2022, 8:42pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/9 "2022-05-04T20:42:19Z")

</div>

Thank you very much for your example. That solves another thing.  
Unfortenetly still not the main-problem ☹

For the sake of simplicity, i told i use the `SiteMeta-Plugin` from Bastian.  
But actually i use the `Kirby-Extended-Plugin` from Johann, that forks the Site-Meta from Bastian into a Group of Plugins. Besides some propertyNames, it really looks the same.  
But there is unfortenetly one little diffenerence, i was not able to see!  
It’s very little 😇

```auto
// Orininal from Bastian:
if (preg_match('!^(?:' . implode('|', $excludePages) . ')$!i', $item->id())) {

// Fork of Johann - dont know why he changed that, or maybe ist an older version??
if (preg_match('/^(?:' . implode('|', $excludePages) . ')$/i', $item->id())) {

```

In the fork, there is a `slash`, not an `exclamation mark`.  
So, when i replace the line with your example, it also works with subpages now!

### But i still have the database error,

when i add `site()->index()...` to the config!

i tried to simplify the config-ready-function.  
i just add some simple options:

```auto
<?php

$base = dirname( __DIR__ , 2);
\KirbyExtended\Env::load($base);

return [
    // Debug
    'debug' => env('KIRBY_DEBUG', false),

    // Database
    'db' => [
        'host' => env('MYSQL_HOST', false),
        'database' => env('MYSQL_DATABASE', false),
        'user' => env('MYSQL_USER', false),
        'password' => env('MYSQL_PASSWORD', false),
    ],
    
    ...

    // Ready
    'ready' => function () {
        return [
            'test' => 'test',
            'test2' => site()->title(),
            //'test3' => site()->index(),

```

that works as expected.  
i can echo the site-title by option(‘test2’)

**but** if i `comment in` `'test3' => site()->index()` i get the database error!

 ![grafik](https://europe1.discourse-cdn.com/flex017/uploads/getkirby/original/2X/1/15bd66bfe45a3a6fc31432a3b10ea422ac7e7f35.png)

WTF has this configuration to do, with my database-connection?  
Is the `ready function` still to early to get virtual pages from DB?  
If i echo `site()->index()` later, i.e. in the Sitemap-Plugin, it works like a charm..  
By the way: for some reasons, this project is still running K3.5.8

---

<div class="post-metadata">

**Author:** ![texnixe](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/texnixe/32/5754_2.png) [@texnixe](https://forum.getkirby.com/u/texnixe)\
**Post date:** [May 4, 2022, 9:01pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/10 "2022-05-04T21:01:14Z")

</div>

Could you show me the complete stack trace for the DB error, please? No idea why this is happening, have you tried to debug this with xDebug? If not, could you provide a minimal example for testing?

---

<div class="post-metadata">

**Author:** ![dennisb](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/dennisb/32/11676_2.png) [@dennisb](https://forum.getkirby.com/u/dennisb)\
**Post date:** [May 4, 2022, 9:08pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/11 "2022-05-04T21:08:48Z")

</div>

PDOException thrown with message “SQLSTATE[HY000] [2002] No such file or directory”

Stacktrace:  
#17 PDOException in /var/www/web/dev/vendor/getkirby/cms/src/Database/Database.php:211  
#16 PDO:\_\_construct in /var/www/web/dev/vendor/getkirby/cms/src/Database/Database.php:211  
#15 Kirby\Database\Database:connect in /var/www/web/dev/vendor/getkirby/cms/src/Database/Database.php:153  
#14 Kirby\Database\Database:\_\_construct in /var/www/web/dev/vendor/getkirby/cms/src/Database/Db.php:59  
#13 Kirby\Database\Db:connect in /var/www/web/dev/vendor/getkirby/cms/src/Database/Db.php:82  
#12 Kirby\Database\Db:table in /var/www/web/dev/vendor/getkirby/cms/src/Database/Db.php:150  
#11 Kirby\Database{closure} in /var/www/web/dev/vendor/getkirby/cms/src/Database/Db.php:126  
#10 Kirby\Database\Db:\_\_callStatic in /var/www/web/dev/site/models/voices.php:21  
#9 VoicesPage:getDataFromDataBase in /var/www/web/dev/site/models/voices.php:16  
#8 VoicesPage:\_\_construct in /var/www/web/dev/vendor/getkirby/cms/src/Cms/Page.php:894  
#7 Kirby\Cms\Page:model in /var/www/web/dev/vendor/getkirby/cms/src/Cms/Page.php:475  
#6 Kirby\Cms\Page:factory in /var/www/web/dev/vendor/getkirby/cms/src/Cms/Pages.php:171  
#5 Kirby\Cms\Pages:factory in /var/www/web/dev/vendor/getkirby/cms/src/Cms/HasChildren.php:43  
#4 Kirby\Cms\Site:children in /var/www/web/dev/vendor/getkirby/cms/src/Cms/HasChildren.php:208  
#3 Kirby\Cms\Site:index in /var/www/web/dev/site/config/config.php:117  
#2 Kirby\Toolkit\F:{closure} in /var/www/web/dev/vendor/getkirby/cms/src/Cms/App.php:948  
#1 Kirby\Cms\App:optionsFromReadyCallback in /var/www/web/dev/vendor/getkirby/cms/src/Cms/App.php:133  
#0 Kirby\Cms\App:\_\_construct in /var/www/web/dev/public/index.php:15

---

<div class="post-metadata">

**Author:** ![dennisb](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/dennisb/32/11676_2.png) [@dennisb](https://forum.getkirby.com/u/dennisb)\
**Post date:** [May 4, 2022, 9:13pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/12 "2022-05-04T21:13:38Z")

</div>

i am going to enable xDebug tmrw.

---

<div class="post-metadata">

**Author:** ![texnixe](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/texnixe/32/5754_2.png) [@texnixe](https://forum.getkirby.com/u/texnixe)\
**Post date:** [May 4, 2022, 9:43pm UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/13 "2022-05-04T21:43:23Z")

</div>

I has a closer look at the source code and noticed that the `exclude.pages` options cannot only take an array but a callable as argument. So instead of putting the call to `site()->index()` into the `ready` option, try and do it inside the callable

Here is the example code from the [getkirby.com](http://getkirby.com) website:

```auto
<?php

return [
    'exclude' => [
        'pages' => function () {
            $pages = [];

            foreach (page('docs/reference/objects')->grandChildren() as $page) {
                if (ReferenceClassesPage::isFeatured($page->id()) === false) {
                    $pages[] = $page->id() . '.*';
                }
            }

            return $pages;
        },
        'templates' => [
            'error',
            'link',
            'reference-classes',
            'reference-shortlink',
            'separator'
        ]
    ],
];

```

---

<div class="post-metadata">

**Author:** ![dennisb](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.getkirby.com/dennisb/32/11676_2.png) [@dennisb](https://forum.getkirby.com/u/dennisb)\
**Post date:** [May 5, 2022, 10:14am UTC](https://forum.getkirby.com/t/how-to-exclude-all-unlisted-pages-in-sitemap-xml-with-the-sitemeta-plugin/25397/14 "2022-05-05T10:14:57Z")

</div>

Oh yeah, `callable` works!  
No need for the `config ready function`.

As you said: That line returns all unlisted pages as an array

```auto
site()->index()->unlisted()->pluck('id', ',');

```

So that’s the final solution to exclude all unlisted pages in the sitemap.xml for [SiteMeta-Plugin](https://github.com/getkirby/getkirby.com/tree/main/site/plugins/meta)

```auto
return [
    // Debug
    'debug' => env('KIRBY_DEBUG', false),

    ...

    // Sitemap
    'meta' => [
      'exclude' => [
        'pages' => function () {
           // exclude all unlisted pages
           return site()->index()->unlisted()->pluck('id', ',');
         },
        'templates' => [
          'default',
          'error',
          'xyz'
        ]
    ]
],

```

In the [kirby-extended-plugin](https://github.com/johannschopplich/kirby-extended) i have to change the pregmatch-function. i will add an issue in their repo.

```php
//if (preg_match('/^(?:' . implode('|', $excludePages) . ')$/i', $item->id())) {
if (preg_match('!^(?:' . implode('|', $excludePages) . ')$!i', $item->id())) {

```

Thank you very much again!!  
Have a good day ☀