Can the cache be used for better search performance?

Kirby 3 Questions 🤔
1

We build a library catalog, currently of about 6300 records, and would like to use Kirby’s internal search at frontend. Each record is a page with 25 fields. The search results are good, but unfortunately the performance is not. Each request already takes about three seconds, and the number of pages will increase significantly in the future.

We’ve tried different approaches:

  1. Algolia beats everything in terms of performance, but monthly costs are a real drawback for our project (backend searches and page updates via panel count as units, too).

  2. Then there’s the search plugin by distantnative. It offers SQLite for the search index? Fantastic, this is exactly what we need! Unfortunately it turned out that for our usecase, some important functions are not supported. For example to process a single search query in two different fields. We don’t get results for “chasper rimas”, where “chasper” is in the field “author” and “rimas” is in the field “title”. Also it’s not possible to limit search to certain fields. And finally, there’s a bug with “Umlaute” at the moment, and most of the books are in german language.

  3. Frontend modules like fuse.js or lunar.js require the search index to be available on client side. But in our case, a test index already weights about 3 MB, which is obviously too heavy.

  4. There’s a PHP port of fuse.js, but fuse still builds its search index at runtime, which again means parsing through the whole file system. So we thought about generating the index for fuse beforehand and store it as a json file. The drawback would be that this index needs to be rebuilt every time a page was updated in the panel.

Hence the question if one of Kirby’s cache options could be used as a static index for the default Kirby search method? And if so, how can the cached data be accessed? First attempts with the plugins “page-sqlite” and “lapse” by @bnomei have not been very promising. My understanding of Kirby’s caching routines and methods is not very profound, though.

1 Like
2

If I look at the readme, you can very well do that.

That probably depends how you send the query to the search method? So whether or not you do and and or or search of the string?

Have you been in contact with @distantnative about these features?

3

Yes. With the plugin and SQLite, limitation to certain fields is only possible beforehand, when generating the index, not at search time.

Different string operators such as AND or OR are currently not documented and, considering the source code of the plugin, seem to be not supported. I’m not a PHP expert, though. After a quick chat with the plugin author via Discord I opened an issue on Github.

4

Did you manage to make it work the the operators AND or OR? There seems to be not working out of the box, I wonder if it requires additional settings?

This didn’t work for me in the search controller:

return function ($site) {
    $query = trim(get('q'));
    if (empty($query) === false) {
        $results = $site->search($query, '['operator' => 'AND', 'limit' => 50]');
    …
5

The second parameter must be an array, not an array not a string, so you have to remove the quotes around the brackets:

$results = $site->search($query, ['operator' => 'AND', 'limit' => 50]);
6

Thanks for the quick reply. Actually I mistyped the example above, even removing the quote the operator didn’t work for me. It does find entry when one word is used.

Excerpt of the controller full code:

return function ($site) {

    $query = trim(get('q'));
    if (empty($query) === false) {
        //$results = $site->search($query, 'title|text|description|blocks');
        $results = $site->search($query, ['operator' => 'AND', 'limit' => 50]);
    }
    // apply pagination
    $results = $results->filterBy('status', 'listed');
    $total = $results->count();
    $results   = $results->paginate(12);
    $pagination = $results->pagination();
    return [
        'query' =>  html(strip_tags($query), false),
        'total' =>  $total,
        'results' => $results ?? [],
        'pagination' => $pagination
    ];
};
7

Did you install the plugin?

8

Yes I did. The build index also works (14.5MB). On the plugin page there seems to be other persons struggling with that but there is not reply from the author.