Memory limit issue with displaying subpages, way to work around it?

The CMS Questions
1

https://getkirby.com/docs/panel/blueprints/subpages-settings

For most all of the content from the website I’m converting over from a custom MySQL database approach over to Kirby things have worked fine, but I’m running into an issue with this one particular section now.

This site has different headline sections, i.e. Features, Previews, and lastly News. The last one is the issue at hand, as it has currently a little over 11,000 entries just in that section.

On my local environment, I noticed somewhere after the 6,000 subpage entry (been importing them in batches of 1,000) it exhausted my memory limit when it tries displaying the subpages. I confirmed this by using the setting in the docs page linked above to hide it, but the side effect of that is that I can’t add any new headlines since the Add button goes away.

I tried to see if there was a way to just show the last X amount of headlines, but the filters I thought would do it all triggered the same memory error.

I’m guessing, based on what I’ve seen browsing other topics in the forum, that I’m going to have to split them up somehow so they’re all not under one section? (Probably would have to do it by year in that case.)

(Somewhat related, but it looks like this amount of entries also causes memory limit errors in the console when using Kirby’s search functionality when logged into the panel, also.)

2

HI @ShawnCollier, yes, that’s correct, with a couple of thousand pages as subpages of a single folder you will run into performance issues not only in the panel but also on the frontend.

You might get around the memory issue by adjusting your PHP settings, but as you have already seen in other performance related topics, a tree-like structure is better suited once you have more than maybe several hundred pages in one subfolder.

3

Hi texnixe,

Sorry for the delay in a reply, I’ve been swamped with work-related stuff (this Kirby project is a side project for me, so my day-time work projects take priority).

I was able to switch things over to a year -> month folder system, which greatly helped with optimizing the memory usage (now there’s no more than ~250 entries per each “month” folder, with a maximum of 12 “month” folders inside each “year” folder).

If you don’t mind, I had one minor other question which slightly relates to a filtering question I think you helped me out with a while back.

For these headline pages I’m needing to have “tag” filters, but based on what I recalled seeing earlier in my testing I’m pretty sure doing a filtering using something like ->index() to grab all of the great-grandChildren (in my case) would cause a out-of-memory error since it’s having to sort through all of the records at once.

If I went through each year and filtered that way (or by year+month if that’s necessary due to memory issues with even the year), is there any way to merge/concatenate filtered $page->…->paginate() calls, so they’re all in one variable?

If I couldn’t do that, I’d have to do something where I print out the year/months where the tag is found in and they have to click through to get to it, instead of seeing the headlines directly.

EDIT: I think this might be the right one I need to use after deeply searching around the docs in Google, it seem to work based on some quick testing?

https://getkirby.com/docs/cheatsheet/helpers/pages

4

Yes, the pages() helper is a good option.

Just for completeness’ sake, here are two alternatives:

//Merge two collections
<?php 
  $collections1 = $pages->find('blog')->children()->visible()->filterBy('field', 'some_value');
  $collection2  = $pages->find('news')->children()->visible();
  $collection3 = new Pages(array($collection1, $collection2));
?>

//Appending stuff to a collection:
<?php 
  $collection1 = new Pages();
  $collection1->add($collection2);
  $collection1->add( $collection3);
// etc...
?>
5

Ah, that second one could come in handy for cases when I need to dynamically loop through a large amount of folders (since the first one is meant more when you know exactly what you’re putting in there).

I think I’m good with this in here now, so I’ll mark this topic as completed then. Thank you so much for your help! :slight_smile:

1 Like