Performance for 10000 pages by tags with pagination?

Has anyone built a big site with a few thousand pages? If my testing with Kirby 2.1 goes well I will use it on big projects.

Let’s say I have this:

  • 10 000 item pages
  • Tags or categories to every page
  • List pages by a tag for example /books
  • Pagination with 20 pages per page
  • 20 fields per item page
  • The item pages will remove and add 20 item pages each day

Questions

  • Kirby needs to go through all 10 000 item pages to find out which pages uses what tags, right? How fast / slow is that? Any tests? I know it’s impossible to say because of the server, the file sizes and so on but… is it a good idea to solve it that way? Would you?
  • Would you recommend using an additional database for it? I know Kirby have database support.
  • How good is caching if the pages changes every day?

Sidenote

Is it “safe” to let a script add and delete new files and folders everyday, in compare to a database? I know it can be a readwrite error on files and folders. Probability?

2 Likes

No responses. Has no one built a big site with Kirby?

Would you be interested in the result of such a test with the requirements in my post? Maybe compared to the same setup with Wordpress? No cache enabled.

If there is not enough interest I will probably not spend time on it. Like this comment or reply to show interest. If enough I might make the speed tests.

6 Likes

You already tried this? I think to testing with a simple php script for create 10k pages and see what happen :smiley:

1 Like

I have no answer to performance questions, but I took a look at Bastian’s Library project and it uses an SQLite database as an index. That could be a way to avoid parsing zillions of files and folders frequently.

Nice project i can use this for the small api site, but for use kirbycms with panel we need to testing with huge pages.

I think we can found more solutions, but the most important is the panel, i don’t know how to see over 10k pages in the same page parent…

I prepare my mac for the testing, tomorrow or next day I think have the response :smile:

1 Like

Oh, that reminds me of this discussion. It was about viewing files, but the same is true for subpages.

10 thousand subpages may be more than Kirby can take. I remember Bastian writing that it should work well up to a few hundred subpages, but it has not been designed for huge “vertical” page trees. With that many contents a CMS with a database is probably better suited.

I think the only limit here is this… btw i wanna try before discard for my new project.

I just found Bastian’s post about the number of files.

1 Like

Nice, i have tested with 1 page and 10.000 subpage.
The pagination working fine and the cache in this case is useless because i not see any difference without testing the relative ms. ofc the speed we see isn’t the same with few pages, but if i compare with the wordpress the speed is more of wp.
Now i need only to testing with some fields and search by field, but i think is really nice results.

God to hear! Did you test locally on your computer or online? The kind of drive or filer used can have a dramatic effect on performance. What works on a good SSD hosting can be very slow on a cheap one.

I tried on my old macbook 4gb coreduo 2.0ghz 500gb nossd, i think with ssd and xeon server it’s much better :smiley:

Oh sure. That said, inexpensive hosting is usually memory constrained, and I wonder how much a huge index would require. I can provide a cheap hosting plan if you ever want to try your test site on it :wink:

I have one i can try after the local testing because i have one client with a 9k posts, so before use wordpress i want to try this, i think the best option for my client, but the index is more important too… i update this post.