Projects might contain 1000 or more blog posts, which can be tough for Kirby to handle when they are in text files.
I wish that Kirby also would be able to load the data from a database, into the panel.
How to
I use MySql. Other people might prefer another database. Then it might seem hard to support them all, especially all the different database table setups.
I have an idea for a solution to that problem. Don’t let Kirby handle those things at all. Let the developer be responsible for making the input data in a format that Kirby can read. When it’s in a Kirby format the panel load the data and add the content into the field form.
I don’t know if it would be possible with panel hooks, but it would still require the content files for the panel pages to show up.
I think you need to approach the site structure a little bit differently, because that many subpages (articles) per page (blog) is just very much for a file-based system in general (not just for the Panel).
The rule of thumb for Kirby subpage performance is: If you know that a page will have very many subpages (how many depends on the site, server and type of page) and you can restructure the content so that the many pages are split up into multiple containers, do that.
Example for your use-case: Create a subpage per year so that “old” blog posts are archived away to the year’s subpage. This should make your site perform much better as Kirby doesn’t have a problem with many subpages, only with many subpages per page.
I don’t mean to use the same method to show the content on the frontend as the panel. For the panel it just need to read the data for the page that is currently being edited.
When grouping files as for example year it’s faster to load the pages. But if we for example want to view all the pages that have the tag “Web design” in the whole blog, then it still needs to go though all the years. Tag clouds and category listings are not that uncommon.
The old Kirby forum (which was based on Kirby) used an SQLite database for caching.
I think what Kirby should do is to improve on caching of collections, raw data and custom queries. That way the file system would still be the primary data store but the performance could be improved even more by using tools like Memcached.
Supporting other types of primary data store would make the whole API more fragile in my opinion.
As long as that number is the UID and not added before a dash, this works just fine. I use this for a site I created not long ago, but I think this has always worked.
If using caching it probably means that it caches the data when I visit for example a tag. With 30 tags it need to cache 30 times. If I change one post that has 5 tags, it need to recache all the 5 tags again.
My worries are:
How slow will it be for the first user of a specific query? No one can know that in before hand, maybe just guess.
More complex sites have more queries. A search function with filters and so on almost have unlimited query possibilities. When changing one thing, how many places needs to be recached?
When do the recache accur? On visit or on panel save? The load has to be somewhere.
In a perfect world the caching should not be noticed by the user on the frontend or panel.
The biggest worry is probably how long it will take the first time before the content is cached and even how many of those waits it will be on a more query-heavy site with more pages and how often it needs to recache (wait more).
This is not so much of a question, just some thoughts. I hope it could work, I really really do!
I don’t think this is really an issue. Of course a site should be as fast as possible, but given a complex site with many users, most users will benefit from this while one single user needs to wait a bit.
Users are accustomed to wait for some pages to take a long time to load, especially since more and more websites become bloated with ads on the frontend. I think it is OK to wait for a custom query to be parsed and built as long as the load time is increased by less than a second by that (which will most definitely not happen).
But caching does not only mean to cache specific queries. It also means that Kirby could create indices of queried fields so that it is easy to dynamically fetch any specific query from that index.
Yes, maybe the issues I’m afraid of is not such a big problem after all then.
Still, we have sites that have 100 000 pages, in a blog (or a web shop). At some point it might be too much? At that point I still belive in my first suggestion in this thread.
I think that 100000 pages are a lot for any CMS. At that point you definitely need to think a lot about content structure, caching and other custom performance improvements. I don’t think any CMS on earth could be absolutely perfect regarding performance for such sites out of the box.
I will actually build that kind of project very soon. I don’t know how many pages but it’s an affilate site with products that I will fetch from some APIs.
The products change every day so I need to constantly update the products, add some, remove others.
In that project I will not have the guts to rely on the file system. I will use MySql with Kirby. I’m probably going to have custom content on the categories and generated content on the products. This is because the categories will not change, the products will.
In that case it would have been nice to see the products in the panel but I will probably need to build some own kind of admin for it.
I can add the result to “Made with Kirby and <3” (looks like an icecream ;))
Yes excatly. Maybe the future feature “panel admin pages” will make that possible. It seems to be a feature close by, but @bastianallgeier never make any promises, so you never really know.
To make that kind of API work like the MySQL case it would require two things.
List of links to the “fake” pages (probably rows from a database table that leads to “fake” panel pages)
The “fake” panel page with the data.
An ugly workaround could be to have a page with a single field that contain all the functions for it but having a field to create a whole admin part feels wrong.
I think @jenstornell concerns are valid. If Kirby becomes popular for lots of use cases, at some point the community will need to address the performance issues that will naturally arise with trying to handle thousands of files.
There are different approaches to how one might solve this. One possibility is to have pluggable persistence, one user might opt to store everything in disk, another might wish to use a database.
Another possibility, which I find much more appealing, is to use databases for indexing content, so the real content would still remain on disk, but dynamic querying would be efficiently handled by database backed up rest services.
In the meantime there’s nothing preventing us from experimenting. I mean, if Kirby provides the hooks, people can implement the indexing/querying stuff around Kirby without needing to change it.
That could be used for search inside contents too.
Another way to approach to performance problems is static pages. It won’t help for every site, but lots of pages on the interwebs are basically static.
Absolutely. One could even build a plugin for the panel which exports/statifies the content, so clients could still edit their content via the panel and just hit “statify” or “publish” once they are finished. Many things to be aware of (URL handling, one tiny edit shouldn’t lead to a complete re-statify-process, …), but in the end it could be a very nice feature.
