Indexing long documents with Algolia search plugin

I’m loving the Algolia plugin but am running into length issues when I try to index the entire page content. Anything less than an enterprise level subscription has a fixed quota amount, which seems small to me. Anyway, Algolia support recommends breaking the page into smaller sections, e.g. paragraphs, per this technique.

I’m looking to modify the plugin to allow this dividing long content by paragraph, but don’t know where to start. Or is this already possible? Thanks for any help :slight_smile:

PHP Simple Dom might be useful here, since allows you to do jQuery like manipulations on the dom structure, only with php. In other words, you can get all the the P tags and loop through them to break them up as described in the article you linked too. See here.

I used simple dom a little while working on a plugin. Shouldn’t be too hard to wrap it up into a plugin so you can use it for Algolia.

It actually doesn’t care if you dont feed it a full and complete HTML page. It will be quite happy if you just fed it the result of kirbytext()… it doesn’t need to see html, head, and body tags etc to work. A fragment of HTML will work.

This is a crazy idea perhaps, but you might be able to programatically create a virtual page (Assuming this is a kirby 3 project) that sets each P tag into a field, then feed those fields through to the Algolia plugin. That way you would not have to modify the Algolia plugin.

Very cool PHP library! This is a Kirby 2 project and I may ask the client to settle for meta field searching instead of true content searching. The Simple Dom seems straightforward but then sending those as separate sections to Algolia seems daunting. Thanks so much for the lead!

No worries :slight_smile: Well, is it imperative that you use Algolia? I’ve been meaning for some time to make a kirby plugin for Lunr which will add powerful search, but keep the data on the server without using a third party service.

Maybe i should just put my head down and do it… :slight_smile: It’s popular with Jekyll users, and there are plenty of guides out there explaining how to use it. Shouldn’t be too hard to convert one of those guides to be Kirby flavoured.

Lunr looks like a neat Javascript-based search. I don’t need to use Algolia, I just need to be able to do boolean searches and phrase queries with quotation marks. I’d be willing to help build a plugin with Lunr if it’s capable of that, and if I can help!

OK lets do it :slight_smile: The way it searches is described here. If thats a fit then let’s go ahead. DM and we can continue on skype or slack or something.

1 Like