StaticBuilder - Kirby as a static site generator


#1

Hello there.

For a work project I need to export a site as static HTML files (+ page files and assets). I have more requirements than what @bastianallgeier’s statify.php script does so I started building a plugin that should ultimately do a bit more.

The first bits are at https://github.com/fvsch/kirby-staticbuilder and it’s very early work, right now it doesn’t copy files (images, PDF etc.) or assets. It’s working for a single page and the action for exporting all pages worked in my case (files are generated) but didn’t show the intended HTML response (Apache and/or PHP gave up at some point).

Update: currently requires Kirby 2.3 (beta). It should be easy enough to make it work on 2.2 but by the time the plugin would be stable then 2.3 will be stable too. ^^

There was a previous effort by @noclat here: Static Kirby website generator but the repo is gone, perhaps because it had nuked some pour soul’s system. :smiley: I’ll try to find a way to prevent that, e.g. enforcing that the output folder (which will be emptied when rebuilding the full site, I don’t intend to do incremental stuff and diffing) has a specific, unchangeable name (like static: you can configure the path to put it anywhere, but the folder itself must me named static).

It’s very early work so any feedback would be appreciated: feature requests, ideas to improve the code or behavior, etc. And as it is right now, I would suggest that you only try running it if you can read and understand the code.


#2

I’m very interested in this, unfortunately I can’t help much with code.

A question I often see raised is: how can I mix static and dynamic content? Because not every site can be 100% static.
Going full static brings lots of constraints, a lot of convenient Kirby features can no longer be used. Displaying 3 random items? Easy with Kirby, but once it’s exported to static pages it won’t be random anymore. Search engine, lists with filters, there are way to do them, but it’s a lot of work.
I wonder if there is a smart way to statify only some pages, while keeping others handled by Kirby.

Another idea is using static generation as a light content staging solution: the normal Kirby site (visible only to logged-in users) would be the staging area where editors can modify and preview. When everything is ready and green-lighted, they would push a “Publish” button that would generate or update the static site.


#3

I’ve added your two remarks in the issues:


#4

i got this error

Notice: Undefined variable: kirby in C:\xampp\htdocs\starterkit\site\plugins\staticbuilder\staticbuilder.php on line 9

Fatal error: Uncaught Error: Call to a member function set() on null in C:\xampp\htdocs\starterkit\site\plugins\staticbuilder\staticbuilder.php:9 Stack trace: #0 C:\xampp\htdocs\starterkit\kirby\kirby.php(460): include_once() #1 C:\xampp\htdocs\starterkit\kirby\kirby.php(433): Kirby->plugin(‘staticbuilder’, ‘dir’) #2 C:\xampp\htdocs\starterkit\kirby\kirby.php(745): Kirby->plugins() #3 C:\xampp\htdocs\starterkit\index.php(16): Kirby->launch() #4 {main} thrown in C:\xampp\htdocs\starterkit\site\plugins\staticbuilder\staticbuilder.php on line 9


#5

Sorry, I forgot to say that it currently requires Kirby 2.3! (At least for setting the routes. I could do that in a Kirby 2.2-compatible way but since it’s very early work I’m not sure I want to invest time in that.)


#6

After quite a few updates:

  • Plugin is now feature complete (compiles pages, copies assets).
  • It should protect you against writing files outside of the “static” folder (whose location can be configured but it MUST be called “static”, sorry but I don’t want to allow people setting it to e.g. /Users/myusersname/).

I have one outstanding bug to squash before I can tag a 1.0. And the next steps will be:

  • Working on multilingual support.
  • Generating URLs that actually work (it currently works with my test project but its URLs are not standard).

#7

Stable release: 1.0

  • A simple HTML interface that previews what is going to be generated
  • Copies assets (assets, content, and thumbs dirs, or whatever config your want)
  • Multilang support
  • Optional relative URLs

Not there yet:

  • Controllers (known bug)
  • Routes (we can’t make any route work in a static context, but there are a few possibilities)

#8

This looks great and is such a killer feature. Improving and integrating this into Kirby should be on the official roadmap IMHO.

How would one set up this plugin to build every time a user saves a page?

This would allow people to deploy this plugin on projects where they’re just delivering the final site/CMS to a non-technical client.


#9

Not sure it makes sense as an official feature. On one hand there are many technical constraints and expectations that come with static HTML export; on the other hand when it comes to rendering a page Kirby is a rather light PHP solution, so it’s not like the speed gain would be that significant (almost negligible with Kirby’s default caching). I built this plugin mostly for a use case where we needed to host the website as static HTML both online and offline (think thumb drives, HTML documentation distributed in a ZIP file, that kind of thing).

There’s no API for that yet in the plugin, though it would be possible to call the main class and ask it to build just one page.

But there are other constraints tied to “saving a page”:

  • Does the page have static assets to copy over?
  • Is it listed on some archive page? You have to rebuild the archive page too.
  • Is it listed on the home page? You have to rebuild the home page too.
  • Is it listed in a RSS or Atom feed? You have to rebuild that “page” too.

Because Kirby and its APIs where not build with the “static HTML export” constraints in mind, like say Jekyll, it offers many features that make it trivial to display content on as many pages you like and however you want, which makes it difficult to do single-page builds or even to do a full-site static build. You have follow the Best practices for static sites and perhaps have some understanding of how PHP, HTTP and HTML work to make it work.

So it’s difficult to go from this current reality to “built-in Kirby feature that works easily for clients without technical supervision” (most likely: it’s not happening).


#10

I agree with @fvsch. Even if this was an official feature, it would still be inside a plugin to keep it modular. But it is correct that the plugin would need to rebuild the entire site on every change in the Panel. It’s the same with Kirby’s cache because of menus etc.


#11

Thanks a ton for the clarification! Makes sense.


#12

Have somebody tried this Plugin togeter with Lukas Bestles & Sascha Lacks “Module Plugin” (https://github.com/getkirby-plugins/modules-plugin)?

The build process stops after the first page with modules in it without giving any error and forwards me to a non existing site URL: mysitefolder/STATICBUILDER_URL_PREFIX/projects/project-a

This is my config:

c::set([
    'plugin.staticbuilder.enabled'    => true,
    'plugin.staticbuilder.outputdir'  => 'static',
    'plugin.staticbuilder.baseurl'    => '/localtest/kirbymodule/static/',
    'plugin.staticbuilder.assets'     => ['assets', 'content', 'thumbs'],
    'plugin.staticbuilder.filter'     => null,
    'plugin.staticbuilder.filename'   => 'index.html',
    'plugin.staticbuilder.uglyurls'   => false,
    'plugin.staticbuilder.pagefiles'  => true,
    'plugin.staticbuilder.catcherror' => false
]);

any idea whats wrong?

greetings,
Sven


Performance of Kirby one page compared to static HTML
#13

I imagine that your modules are, for Kirby itself, pages, and those pages use a template or controller that redirects requests to the parent page. So the redirection your seeing is that: a redirection to the parent page.

The StaticBuilder plugin will call Kirby’s render method for each page. If when rendering a page (including a “module” page) you set HTTP headers, they might end up in the final response and redirect you or create other such issues. There are some provisions to NOT activate those page-specific HTTP headers, but they require you to set headers in the c::set('headers', …) config:
https://getkirby.com/docs/cheatsheet/options/headers
(We still try to flush HTTP headers after building each page but sometimes it doesn’t work.)

You could try:

  • If you’re doing the redirect yourself in a template or controller, try to move the redirect to a function in the c::set('headers', …) config.
  • Try excluding the module pages from the built HTML pages, by defining a function for the plugin.staticbuilder.filter option. Something like:
c::set('plugin.staticbuilder.filter', function($page) {
  // Default page exclusion filter.
  // Change to also return false if $page is a module page.
  return file_exists($page->textfile());
}):

Not 100% sure this will fix your issue, but it’s a start. :slight_smile:


#14

Oh man, it works like a charm!
Thank you very much! :kissing_heart:

best Regards,
Sven

c::set('plugin.staticbuilder.filter', function($page) {

	if ( strpos( $page->template(), "module." ) === 0 ) {
		return false;
	}

  return file_exists($page->textfile());
});

@lukasbestle: maybe you should mention this in your documentation or module plugin related forum thread.


#15

@Svnt + @lukasbestle:
I’ve included this check by default in the 2.0.0 release. I figured the Modules plugin is popular enough and the check for module.* template precise enough to add it by default.

2.0 release


#16

That’s great! However I’d recommend using the modules.template.prefix option for this. Hardcoding will lead to issues if users change the option. :wink:


#17

Hello,

i have installed V2 and some pages are excluded… i don’t know why.
Maybe because they use the “default”-template?

edit: changed to another template: same result :frowning:


#18

Hi. This is the default function used to exclude “pages” (which can be empty folders of assets or other binary data in your content folder) from the build:

So the two reasons they might get excluded:

  1. if they’re “module” pages, with a template file starting with module..
  2. if the “page” has no text file. For a folder with no .txt file, the value of $page->textfile() will be /path/to/content/pagename/pagename.txt, a theoretical path which doesn’t exist, so we check that.

If that default filter is not giving you good results, you could write your own, for instance if you want to accept EVERYTHING:

// in your config.php
c::set('staticbuilder.filter', function($page) {
    return true;
});

#19

Ok, thank you. I got it now.

The problem was:

The excluded pages only have one (of two) textfiles (one per language).
So the export only works if default.de.txt and default.en.txt exist.

Idea for the 2.1.: some kind of ‘auto generate’/‘copying’ missing text files. :wink:

best regards,
Sven


#20

Nope, writing to your content folder in any way is outside the scope of this plugin.

But I’m looking into ways to report more meaningful messages when a page is excluded.