General question: where are the bottlenecks in Kirby right now? How can we as a community improve the performance and reduce the response time? Ideas?
After lots of use cases with Kirby I can say that bottlenecks happen when you put a lot (like a couple hundred or even more than thousand) subpages in a single folder. That is when scanning the list of folders gets messy. Other than that Kirby can handle A LOT of content if it’s structured more tree-ish. My biggest client website has a couple thousand subpages in a rather wide-spread tree structure and it runs extremely smooth. No page has more than 200-300 subpages and so I couldn’t find a single bottleneck even with this massive structure and more than 4 GB or files. You can check it out here: http://nimbus-lighting.com
On the other hand I had some performance issues that clearly came from servers. Some hosting providers have terribly slow disks or use some self-made “cloud” storage, which slows down Kirby more than necessary. I’m looking at Media Temples Grid system here as an example. A client went from the Grid system to Webfaction and the response times went down from 400-500ms to less than 100ms avg.
With more and more hosting providers - even the cheap ones - switching to SSD I’m quite optimistic for even better environments for file-based systems like Kirby.
Do you have any negative experiences performance-wise?
3 Likes
Literally just pulled the site off the MT Grid last week and got the boost you describe. Really good to know!
I’m really glad to hear that! May I ask where you moved your site?
I put the Kirby sites onto a shared plan on Dreamhost, and I’m going to look into Webfaction at some point. (But that’s just two sites or so, and not, like, any kind of diss on MT or endorsement of DH or request for a bunch of hosting people to try to sell me on anything. 
Just curious: do you run http://nimbus-lighting.com with or without caching?
Edit: As I got a reply from Bastian over email I might as well post his answer below:
The nimbus site is running on an SSD and we combine it with Memcache for caching.
1 Like
Maybe a thread with SSD hoster recommendations is a good idea?
Haven’t found any (trustworthy) managed/webspace with SSD yet. Only pricey servers and ‘MySQL on SSD only’.
if you need trustworthy shared SSD hosting, I’d go for webfaction (not sponsored :))
1 Like
Right now I’m trying to migrate www.taeglich.me to Kirby – the latest version runs on Wordpress and well…we have so much problems with it, at least there is one issue with the database per month. It really pisses me off and since I’m running other projects with kirby very well and without any problems I think it’s time maybe to migrate it to Kirby.
This will be an interesting task, since I don’t even know how kirby will behave with our structure. It is a digital newspaper website, and our journalists are developing news about several cities. The biggest city has nearly 6000 articles. Do you have any advice to prevent the site of being slow regarding the logic of a kirby-nested structure? Any advice will be appreciated and would help me alot!
What are the more content-intensive tasks, where you need to query a lot of content to make indexes, search, etc.?
Maybe you should have a separate database or cache or search index where a copy of your content lives (updated periodically), and query that? For search, you could perhaps use Solr (or something similar) and write your own script/plugin that feeds it content. For indexes… well they could be search queries too, under the hood.
Would it make sense to organise the articles under subfolders organized by date?
2015/
01/
.../
2016/
01/
Thx to @fvsch and @samnabi for your suggestions.
Indeed, it would make sense to organise all the articles chronologically into years and months and even days?!
We have even different types of news: Main Articles, Short Articles, Sports Articles, Special Articles…
So it’s better to have a deep tree rather than just one folder with all the articles in it?
For example:
2015/
01/
01/
main/
.../
shorties/
.../
sports/
.../
02/
main/
.../
shorties/
.../
sports/
.../
and so forth.
Sure, that would work. You could also do it like this:
main/
2015/
01/
...
...
shorties/
2015/
01/
...
...
...
Or you could leave out the days and only use years and months. That all depends on the number of articles per category. Too much nesting is going to be more work for the editors.
1 Like
Well, I think this can be affordable, as long as the page runs smooth because of it.
The primary thing is: can Kirby handle so many subpages?
A few numbers: 3.300 short-news, 2.000 main-news, 700 sports-news, average articles per month: 220 (all numbers per city!)
So I think if you break it down to the folder structure:
city1/
main/
2015/
01/ -> 120 Subpages
shorties/
2015/
01/ -> 80 Subpages
sports/
2015/
01/ -> 40 Subpages
city2/
main/
2015/
01/ -> 120 Subpages
shorties/
2015/
01/ -> 80 Subpages
sports/
2015/
01/ -> 40 Subpages
Well it would be fun trying to achive this, maybe with some help from the advices of @fvsch
Those numbers sound doable with the month structure. 
What are your most complex data fetching, sorting etc. use cases?
Do you have full-site search? This one can be deported to a specific service (e.g. Algolia), so it’s not a deal-breaker in itself, but you’re probably not going to use Kirby’s built-in search method for thousands of articles.
Do you have content lists that mix content from all cities? Something which basically answers “I want to see the latest 20 stories from any city that are Featured and from the Sports category”. If you have one content repository (one folder) per city, and no sub-folders, that means:
- For each city:
- Get all stories
- Filter on Category=Sports
- Filter on Featured=true
- Then merge all those collections
- Sort by date (descending)
- Keep the first 20
That could be an intensive operation, without a really fast database or search index that keeps a flat list of content and lets your retrieve that.
Of course you can make subfolders to limit the number of pages per folder. But for any query that must list content across folders, you’re going to lose whatever speed gain that offers for other scenarios.
I don’t want to be alarmist. Maybe Kirby can manage the content listing and cross-referencing you need just fine. One way to figure out would be to export your current text content to a flat structure, maybe with a cityname/[thousands of folders here] structure only. Then write PHP code using Kirby’s API that makes the most important and more challenging content lists you need (which listing would be the less performant one? which one would be most common?). Then measure how it performs. It doesn’t have to look like a website, just basic links, maybe with a text extract for each article, no CSS.
A few attempts to make a WordPress-to-Kirby exporter:
If exporting the current text content is too hard, you could make a script that generates thousands of pages full of Lorem Ipsum. If you do that, generate twice as many articles as you have now.
With that kind of test, you should be able to tell if things go smoothly or if some parts are way too slow and you need a perf layer on top (Solr or something else) and for which parts.