Kirby3 Boost - makes loading a lot of pages in one request fast + unique ID

I am creating posts for my plugins to make it easier to find them using the forum search and not just the docs search.

Boost the speed of Kirby by having content files of pages cached, with automatic unique ID, fast lookup and Tiny-URL.

  • up to 3x faster content loading
  • fastest page lookup and resolution of relations

Usecase

If you have to process within a single request a lot of page objects (1000+) or if you have a lot of relations between page objects to resolve then consider using this plugin. With less page objects you will propably not gain enough to justify the overhead.

How does this plugin work?

  • It caches all content files and keeps the cache up to date when you add or modify content. This cache will be used when constructing page objects making everything that involves page objects faster (even the Panel).
  • It provides a benchmark to help you decide which cachedriver to use.
  • It can add an unique ID for page objects that can be used create relations that do not break even if the slug or directory of a page object changes.
  • It provides a very fast lookup for page objects via id, diruri or the unique id.
  • It provides you with a tiny-url for page objects that have an unique id.

site/blueprints/pages/default.yml

preset: page

fields:
  # visible field
  boostid:
    type: boostid
  
  # hidden field
  #boostid:
  #  extends: fields/boostid

  one_relation:
    extends: fields/boostidpage

  many_related:
    extends: fields/boostidpages

site/models/default.php

class DefaultPage extends \Kirby\Cms\Page
{
    use \Bnomei\PageHasBoost;
}

// or

class DefaultPage extends \Bnomei\BoostPage
{
    
}

PHP

// page lookup
$page = page($somePageId); // slower
$page = boost($somePageId); // faster
$page = boost($somePageDirUri); // fastest
$page = boost($boostId); // will use fastest internally

// resolving one relation
$pageOrNull = $page->one_relation()->fromBoostID();

// resolving many relations
$pagesCollectionOrNull = $page->many_related()->fromBoostIDs();

Cache Drivers

A cache driver is a piece of code that defines where get/set commands for the key/value store of the cache are directed to. Kirby has built in support for File, Apcu, Memcached and Memory. I have created additional cache drivers for MySQL , Redis and SQLite.

site/config/config.php

<?php

return [
    // other options

    // example apcu
    'bnomei.boost.cache' => [
        'type'     => 'apcu',
    ],
];

Interactive Demo

I created an interactive demo to compare various cache drivers and prove how much your website can be boosted. It kind of ended up as a love-letter to the KQL Plugin as well. You can find the benchmark and interactive demos running on server sponsored by Kirbyzone here:

3 Likes

Hi @bnomei, thanks for another great plugin. I wanted to use the included Benchmark on a production server to decide which driver to use, but was unable to perform the test. Maybe I forgot some configs, but I couldn’t figure it out.

My php file looks like this:

<?php

$caches = [
    \Bnomei\BoostCache::sqlite(),
    \Bnomei\BoostCache::redis(),
    \Bnomei\BoostCache::memcached()
];

$results = \Bnomei\CacheBenchmark::run($caches, 1, site()->index()->count());
print_r($results['results']);

But it returns an empty array.

My models/default.php like this:

<?php
class DefaultPage extends \Kirby\Cms\Page
{
    use \Bnomei\PageHasBoost;
}

Do i need pages with the boostid-field for the benchmark test?

you do not need real pages to run the CacheBenchmark. but that is only a quick way to compare some caches. like the readme says you should verify real performance lateron using the boostmark() pages method as well. for example on my demo server the redis driver ranks 4th place in the generic benchmark but 2nd place when using the boostmark with real page objects.

the only situation for the array to come up emtpy would be that the drivers are not available as far as i can tell right now. so maybe var_dump the $caches array and see if there are objects in there or just null values.

ah yeah, i hadn’t configured the caches correctly. thanks for the hint.

unfortunately, i am unable to activate APCu and Memcached on the production server due to limitations (it’s a managed server). will there be support for the OPcache with the boost plugin? that would be great.

afaik opache does not have a user cache. it just caches the compiled php files.
but my sqlite driver is a pretty fast solution if you have sqlite3 support.

2 Likes

yes, sqlite does work. thanks for your help!

Hi again :slight_smile:

A few days later, i wanted to setup everything using the SQL-Lite server, but i get this Error:

Fatal error: Uncaught Error: The SQLite3 object has not been correctly initialised or is already
 closed in .../site/plugins/kirby3-sqlite-
cachedriver/classes/SQLiteCache.php:175 Stack trace: #0
.../site/plugins/kirby3-sqlite-cachedriver/classes/SQLiteCache.php(175): SQLite3Stmt-
>bindValue(':id', '...',
 3) #1 ...//site/plugins/kirby3-sqlite-cachedriver/classes/SQLiteCache.php(107): 
Bnomei\SQLiteCache->remove(...')
 #2 .../site/plugins/kirby3-sqlite-cachedriver/classes/SQLiteCache.php(102): Bnomei\SQLiteCache->removeAndSet('index', Array, 0) 
#3 /.../site/plugins/kirby3-boost/classes/BoostIndex.php(54): Bnomei\SQLiteCache->set('index', Array, 0) 
#4 /.../site/plugins/kirby3-boost/classes/BoostIndex.php(37): Bnomei\BoostIndex->write() #5 [internal function]: 
Bnomei\BoostIndex->__destruct() #6 {main} thrown in 
/.../site/plugins/kirby3-sqlite-cachedriver/classes/SQLiteCache.php on line 175

When running

site()->boost();

or

var_dump(site()->boostmark());

Do i need to initialize the SQLite Cachedriver plugin somehow?

Thank you!

i created an issue SQLite3 object has not been correctly initialised or is already closed · Issue #1 · bnomei/kirby3-sqlite-cachedriver · GitHub
lets continue discussing the sqlite driver issue there.

1 Like

Thanks for the fix!

One more question: does the Boost-plugin also speed up the Kirby query-language inside a blueprint? Like when used for my many-to-many plugin:

foreignkey:
        label: Project
        type: multiselect
        min: 1
        max: 1
        options: query
        query:
          fetch: site.find('projects').childrenAndDrafts
          text: "{{ page.title }}" #<-- will this be quicker with boost?
          value: "{{ page.boostid }}"

Will this speed up the fetching of the titles?

Thanks :slight_smile:

yes. once you have a page boosted accessing any content field will be faster since the initial load was made from cache.
in your example the title will be loaded the same time as the boostid. but that load will be from cache and thus faster than from disk.

it does not however reduce computational load like from using site->find or site->index. you would need to create a statically cached collection/sitemethod for these. static because it might be called by kirby more than once in a single request. i will publish a short example later.

1 Like

with boost its quiet easy to measure/see the performance gain. just toggle the debug mode. boost will always write to the cache but only load in non-debug mode.

1 Like

Thanks, thats very helpful. I will make a new release of my plugin using boost then soon.

i explained a bit how to create static cached collections here:

New version 1.8.0 adds a site index crawler with lower memory footprint than core kirby. This makes it possible to run a callback on a huge amount of pages.

Using site()->index() in Kirby will load all Pages into memory at the same time. This plugin provides a way to iterate over the index with having only one page loaded at a time.

$boostedCount = 0;
$indexCount = \Bnomei\Bolt::index(function ($page) use (&$boostedCount) {
    // do something with that $page like...
    $boostedCount += $page->boost() ? 1 : 0;
});
// or just
$boostedCount = site()->boost();

Within some limitations you could use \Bnomei\Bolt::index($callback) for your own code as well like when counting or updating pages.

The new version 1.9.0 add support for content caching of files and users. So if you have to read the content files of many of them then do consider using the Boost plugin to speed up the load time up to 3-4x times.