Large numbers of "virtual users"

Hello!

I’m a recent discoverer of Kirby, and it seems truly fantastic. I’m evaluating whether I can use it for an upcoming project and have a question about “virtual users”, specifically supporting large numbers of them (20,000+).

I found a related thread and an in-progress guide about using Kirby’s great user management features for users not stored in the filesystem, but so far haven’t found a resource explaining how to do so efficiently for larger numbers of users.

In my case, being able to administer these users from the admin panel isn’t necessary and I’d be comfortable writing my own admin-only pages to handle that. Additionally, the users wouldn’t necessarily need Panel access, since their data could also live in a database and be rendered with virtual pages.

What I’m excited about is the possibility of using Kirby’s login/registration/challenge/authentication/session management for these virtual users.

I’d be very curious if this is something that’s possible with Kirby in a clean way. I know Kirby has user models, but am still wrapping my mind around how all of the pieces fit together and whether it would be possible to integrate into Kirby’s user system without having to e.g. enumerate all of the virtual users for every request.

I did a large project with kirby and extending users to a sqlite database. Actually i also ended up using virtual pages from database as well. I can just tell you what i learned out of this.

The user object is loaded at init what means all users will be loaded then at once. If you have 20k users this can take some time. I did tests and if you have around 20k users consider the page will need around 1 to 2 seconds to load the user object.

The other thing: Kirby is a flat file based CMS system. If you start using DB’s you will start “breaking” the core idea of kirby. Don’t get me wrong. All this is possible and using DB’s for some task are O.K. But if you using DB’s for users and pages i really don’t see the benefit of using kirby.

If you need a DB based setup and you don’t need the backend anyway, i would consider look into some laravel solutions.

1 Like

I focused my question specifically on the the part of my project that will be database-driven but other parts are more static and smaller in scale, and would benefit from Kirby’s Panel and file-based approach.

But if there’s architecturally no way to avoid a linear scan of the entire user base, then I suspect something like a more traditional CMS is in my future, at least this time around.

Edit: Though having had a look at the other approaches, I’m considering if I can reshape the project to fit better with Kirby’s strengths, since then I could focus my time on making the product better rather than on infrastructure…

my suggestion would be to create and remove the users dynamically. this way only the concurrent users are loaded. that work well if you just need them for authentification. all other user related stuff like counting etc can be done directly in your database.

  • user tries to login with email
  • get user dataset from sqlite based on email
  • create user object
  • do auth with that user
  • on logout and on session timeout (you have to create that logic) delete the user from kirbys filesystem

that should keep kirby fast.

2 Likes