I’m a recent discoverer of Kirby, and it seems truly fantastic. I’m evaluating whether I can use it for an upcoming project and have a question about “virtual users”, specifically supporting large numbers of them (20,000+).
I found a related thread and an in-progress guide about using Kirby’s great user management features for users not stored in the filesystem, but so far haven’t found a resource explaining how to do so efficiently for larger numbers of users.
In my case, being able to administer these users from the admin panel isn’t necessary and I’d be comfortable writing my own admin-only pages to handle that. Additionally, the users wouldn’t necessarily need Panel access, since their data could also live in a database and be rendered with virtual pages.
What I’m excited about is the possibility of using Kirby’s login/registration/challenge/authentication/session management for these virtual users.
I’d be very curious if this is something that’s possible with Kirby in a clean way. I know Kirby has user models, but am still wrapping my mind around how all of the pieces fit together and whether it would be possible to integrate into Kirby’s user system without having to e.g. enumerate all of the virtual users for every request.
I did a large project with kirby and extending users to a sqlite database. Actually i also ended up using virtual pages from database as well. I can just tell you what i learned out of this.
The user object is loaded at init what means all users will be loaded then at once. If you have 20k users this can take some time. I did tests and if you have around 20k users consider the page will need around 1 to 2 seconds to load the user object.
The other thing: Kirby is a flat file based CMS system. If you start using DB’s you will start “breaking” the core idea of kirby. Don’t get me wrong. All this is possible and using DB’s for some task are O.K. But if you using DB’s for users and pages i really don’t see the benefit of using kirby.
If you need a DB based setup and you don’t need the backend anyway, i would consider look into some laravel solutions.
I focused my question specifically on the the part of my project that will be database-driven but other parts are more static and smaller in scale, and would benefit from Kirby’s Panel and file-based approach.
But if there’s architecturally no way to avoid a linear scan of the entire user base, then I suspect something like a more traditional CMS is in my future, at least this time around.
Edit: Though having had a look at the other approaches, I’m considering if I can reshape the project to fit better with Kirby’s strengths, since then I could focus my time on making the product better rather than on infrastructure…
my suggestion would be to create and remove the users dynamically. this way only the concurrent users are loaded. that work well if you just need them for authentification. all other user related stuff like counting etc can be done directly in your database.
user tries to login with email
get user dataset from sqlite based on email
create user object
do auth with that user
on logout and on session timeout (you have to create that logic) delete the user from kirbys filesystem
I’ve made a starter kit that has virtual users. I’ve extended the user area for the panel to load the users effectively from the database. Feel free to create a PR or leave a comment here if you see improvements.
I wrote a recipe for virtual users from database quite a while ago. The main reason we did not publish it, is that using a database doesn’t really solve all performance problems without proper lazy loading. The table always has to be loaded at once.
Looks like you solved this with pagination, to always only load a given number of users? Does this also work when searching for users?
Hi Sonja,
I’ve solved only the panel performance with database pagination. I agree I need to implement lazy collections for this or some other solution.
More a FP approach, functions first, data-last.
I will create a issue for this in the appkit repo.