I’m curious to hear when you think its time to upgrade your CMS based on your growing needs (e.g having hundreds of blog posts, different authors, etc.) or if you would need to upgrade CMSs at all.
There might be minor breaking changes with each version but I think the biggest difference is between Kirby 1 and Kirby 2. After version 2 I think it’s not that much difference, except for new features.
But always test in localhost or development environment first.
1 Like
If you grow with pages in a flat hierarchy you can add a database for speed but then you will not be able to use the panel for them.
In my latest project I use a database for storing external data in combination with text files.
1 Like
Sure, it might be a project need to switch the underlying CMS or even programming language.
Let’s say the project started with something small or medium-sized and Kirby + one or two custom plugins were a good fit. You were comfortable with Kirby and plain PHP so you were more productive with that CMS than with having to configure and restrict a more heavyweight CMS (Drupal, WordPress even…). So that’s how the project got done.
Then the client comes up with needs for evolving or redesigning the website, big new features, lots more content or user-provided data, etc. Maybe there’s a need to integrate data from a webservice that serves hundreds of megabytes of JSON content that needs to be cached locally in a database and updated every night? That’s a new project in and of itself. So you evaluate your choices:
- What would it cost to do it with Kirby + PHP + maybe a custom MySQL database?
- What would it cost with this or that heavyweight CMS and some plugins/modules that seem to meet the client’s needs?
- Or maybe a hybrid approach where you have Kirby for the information parts and maybe a different application for serving other pages or fragments of pages, managing a database and CRUD operations, etc. Perhaps a Kirby+Laravel hybrid?
And for each solution, asking not only how much it would cost to do it but: would the result be reliable and easy to expand upon in the future?
Also, consider that a project’s “growing needs” doesn’t just imply tech choices. Most importantly, it implies available skills and time. If you’re a web designer with HTML+CSS skills, and just a bit of PHP (or other programming language) skills, which is perfectly alright of course, you can set up a simple project with Kirby but when the project needs become more complex they are not growing out of Kirby, they are growing out of you. As in, a single-person team with mostly front-end skills and no database / data / storage / devops / deployment skills. You then need more people (and time and money) to complete the team’s skill-set, and maybe the tech choices will go towards a different tech stack but maybe it will still build on Kirby.
That’s a bit like when Twitter started with a Ruby On Rails website but it couldn’t keep up with the load at all, and they rewrote most of their website. They must have seen many more rewrites of different services since then. In my opinion they didn’t grow out of Ruby On Rails per se, they grew out of having few server/ops/data people in their team to begin with and being late to address those needs. Maybe ditching Ruby On Rails was a given, but let’s not look at the wrong things here: the important part is the skills and people and resources, and planning/investing ahead. The tech choices (and changes and partial rewrites) must come, but they’re a secondary concern: if you don’t have anyone on staff to see (and foresee) the needs and do the evolutions (on top of your existing stack) or partial rewrites (swapping limited bits of your existing stack for more capable implementations), it won’t help one bit that you know that Kirby+PHP can do a lot or that a switch to a big Drupal or Laravel-based CMS is an option. 
7 Likes
@fvsch that’s a very nice write-up how to approach a growing project.
Right now we struggle with the “small website CMS” label. Kirby really isn’t. But it totally depends on the requirements. I often combine Kirby with other tools to achieve better results. For example many clients ask for shop systems. It’s possible to build a small shop with Kirby, but it’s not perfect for a full-blown shop solution. On the other hand most shop systems are horrible content management systems. So it often makes sense to combine tools to get the best out of them. Discourse as a forum, maybe Shopify or something like Magento for a shop, Mailchimp or Campaignmonitor for Newsletter/Subscriber management, Aloglia or a custom elastic search server for a full-blown search etc. It’s rather “easy” to grow a site in such a modular way.
As @fvsch already wrote, it’s of course also possible to extend Kirby even more. Add a database for big datasets, use Kirby’s memcache driver to get faster performance, use multiple servers with load balancers to react to more traffic, etc.
Finally, if you really grow out of Kirby, I think that we offer one of the most convincing formats to do exports. You can use Kirby’s Page and File APIs to get access to anything and just write it to a database or create a CVS, JSON, XML, Excel or whatever export format you need to port your content to another system.
I totally believe in reducing vendor lock-in. Of course we want to keep you using Kirby, but we also want you to feel in control of your data.
7 Likes
i created a page with about 100 editors and more than 10k content pages and versioning on server with git.
if you do proper optimization of your template code (not calling $site0->index()) and keep folder hirachy flat (so panel stays fast) content wise kirby is a beast.
having tools available for export like @bastianallgeier said makes it easy to change the structure if needed.
4 Likes
@bastianallgeier you mentioned using Kirby alongside other systems - i’d be very interested to hear if anyone has any more to say on that.
For instance, I can imagine using Magento for the product pages and checkout on a site, but Kirby for all the other content sections. Would they have to run on separate subdomains though, with perhaps a shared CDN for all assets, or could they actually run parallel with some clever routing setup?
You could put Magento in a subdirectory of your Kirby site. The standard Kirby .htaccess will allow requests to existing directories go through.
Actually, I’m working on a website that has “grown out of” Expression Engine. And, believe me, getting data out of EE, in comparison to Kirby, is nearly impossible (all data is saved in 1 (one!) table where for each record, each column could be used for something different. It’s a mess.).
The website handles about 20’000 items + some static informational pages. Kirby stores the informational pages and renders all the templates, while the bigger amount of data is stored in an external database that serves data to kirby via a REST API.
I’ve also managed (to a certain extent) to let the Panel build the forms for my “pages that are stored in a database” and I plan to release this plugin as soon as I get it working right.
The biggest difficulty I’m encountering is actually the tight coupling between the field types and the concept of having data stored in a textfile. For example the structure field, for some reason, somewhere, does a scandir for a directory that of course doesn’t exist (being that the data comes from a database).
Maybe in a few major releases we could think about decoupling the Panel from the data storage… (?)
3 Likes
I’m currently digging for ways to use Kirby’s panel to manage data stored in a database.
Clearly a DB is the way to go for thousands of entries with complex querying requirements. I prefer Kirby and if there’s a practical way to use a database with Kirby - while allowing the client to manage the data in the panel - I’d be very interested in learning more.
1 Like
I don’t understand why you would you want kirby/panel to work with db. I mean there are good things in php like laravel and slim or bunch of database driven cmeses.
With databases you get into one important problem that filebased solution doesn’t have - migrations. The beauty of kirby is that it handles txt files in way that you can very easily append, delete and update all the data. This is not always possible with databases. You often have to migrate the db to new version with new rows and different structure of the data - that means basically write little programs that will do the migration. This can be real headache and forementioned framework already have lot of tools to help with that exact problem.
If you use document based database like mongo or couchdb this isnt so big issue but then you are getting to much more advanced technologies and something like kirby is not really interesting anymore.
Thanks for the reply.
Kirby provides a database class in its Toolkit so even the folks that build Kirby understand that a DB is best in some cases. In almost all of those cases - where a database is best - the client will need to manage the content in the database. I would prefer if the client could use Kirby’s panel as opposed to jumping to some other interface. As far as the clients knows, it’s just plain odd.
Straight up… I need 1000 pages under 1 page. Is this an issue or not? Yeah… it’s gonna be on a SSD drive. No. the client will never organize things into any years (come on - what do they know or care about such things?). I don’t necessarily want to use a db, but I’m just trying to do what’s best for my clients. Every site grows over time.
Well, the Panel obviously only works with files, not with a database. The only way you could maybe achieve this, is with a plugin that connects to the database and then create custom Panel views where the user can edit the database data.
Is 1000 that many? We run page with 500 pages where there is lot of relation between pages (looking for multiple related pages so kirby has to go trough lot of tags stuff like that). It was always very fast. I the only problem is usualy the search - where db can do lot of work for you and does things that would be impossible just in php.
What i am trying to say - are you sure that its the filesystem that makes your site slow? People underestimate the filesystem but they are damn fast and DBs are actualy working in very similar ways. Don’t forget once files are used they end up in memory - you dont read it from disk all the time. Its same like with databases. Also how big is the real content (the text files) - 50MB? Thats nothing in memory.
Most of the time websites are slow because of bad frontend code. Maybe you have too little ram on your server (or you have overcrowded shared server). Or you have little bug in your template that is making things slow.
I would try to be really sure about the cause before switching to db. You might end up with site as slow as before.
If its live public site, you might aswell post url here. People might have ideas about the frontend code 
Thanks Krisa (and Sonja) for your time.
I really don’t want to use a db if Kirby can handle upwards of 1,000 “pages” without performance issues. I have personally never run any tests on massive amounts of pages. Answers in the forum here don’t seem definite regarding large amounts of pages and performance. I suppose that’s because it all depends, but I’m not creating anything crazy here. Consider 500 product pages, each with 5 product shots. I need to search the products. I need to categorize the products (note: 1 product may live in multiple categories). Pretty common stuff.
Based on your comments I’m optimistic that using Kirby is still a good option.
1 Like
I’d do a test, create 1000 pages as subpages of a single folder programmatically, including images, and see how it performs, both on the frontend with searching and filtering, as well as handling that many subpages in the Panel. With a fast server and SSD, up to 1000 pages should in fact be doable from what I’ve read here in the forum. But if it’s about product pages, I wonder if it wouldn’t make more sense to categorize them into subfolders first instead of putting them all into a single folder (yeah, even if they have multiple categories, there are usually ways to sort them into buckets in a way that makes it easier for the client to handle these products as well).
I think it will be just fine.
Like i said the problem really is the search. People are used to very good search engines and it can be quite advanced work thing to achieve that. Kirby search is “dumb”, it works for basic things but you will definitely need things like ranking and stemming and if you are not english - you will need it even more.
The way i solved it might sound counter intuitive but i am using both db and filesystem. I am basically indexing all the pages specifically for the search and use db just for the search. This way the files are always source of truth and db is just replication (besides i index only stuff that needs to be searched). I tried to go the only db route but ive found it to be worse solution. I didn’t seem to get better performance out of it and i lost the power of the filebased cms (easy to debug and backup etc.).
Yes you are duplicating content but redundancy is good. Few megs of hard drive space is cheap.
Its also easy to achieve - you either index on panel hook when somebody edits the text file or just do cron that will update the db from time to time (i do this, because the data don’t change that much).
It has huge upside if database fails which they tend to do from time to time - you are fine, your site is going you just don’t have search.
This design worked so well i am actualy considering streamline it (do plugin for it) next time i will be doing site like that.
if the created posts contain a date (which i presume is true) you could create an automated archive function which creates the archive folders and moved content from anything but current year into its proper place.
this could also be done with a hook and dir::move.
in the frontend you just have to filter by template or iterate all archives and current.
1 Like
i do not understand why the default search is “dump”? it even has scoring based on matches.
about creating a plugin… unless yours will be free the algolia plugin by kirbys @lukasbestle seems very promising and does excactly what you just describe, right?
Thanks Bnomei.
Having pages that are organized by date moved automatically to an archive, based on year for example, seems like a practical solution to keep things tidy and preforming well. I could see this method used quite often. Hmmmm.
With files moving I can imagine how that could break some queries if you’re not careful, but I guess that why you mention filtering by template as that’s pretty solid. Interesting.