ImageKit - An Asynchronous Thumbs API

If Kirby’s built-in thumbs component does not exceed the capacities of your server, neither should ImageKit, because the plugin also uses the Thumb component of Kirby’s toolkit.

If your live server is running Apache, also check if the mod_spelling is enabled. In an earlier version of my plugin (never released, worked a little differently …), I had problems with with this Apache feature. Try to add CheckSpelling Off to your .htaccess file. I would like to hear from you if this helps.

Good luck! :slight_smile:

Yay! Beta2 is here! :tada:

ImageKit 1.0.0-beta2

Changelog:

  • Changed Job-File Suffix: Pending thumbs aka placeholder files aka job files now have a suffix of -imagekitjob.php instead of .imagekitjob.php. This fixes errors with Apache’s MultiViews feature (read explanation). You should clear your thumbs folder after upgrading.
  • Error Handling: ImageKit now tries it’s best to show you if there was an error in the thumbnail creating process. The widget is now able to display errors and if thumbnail creation failed, an error image is returned instead of nothing.
  • Discovery Feature: The widget now scans your whole site for thumbnails, so you don’t have to open every page manually.
  • Widget Code: The widget logic has been improved on both the server and the client side for better extensibility.
  • Widget UI: Added text underneath the progress bar to give the user a better understanding of what the widget is currently doing. Added animation while the progress bar is visible. If an operation is cancelled, Widget UI is now blocked until another operation can be started.
  • Permissions: The widget now shows an error message when the user has been logged out. The widget is now accessible for all logged-in panel users by default.
  • Refactoring: The whole plugin has been refactored here and there …

Get it from GitHub: https://github.com/fabianmichael/kirby-imagekit

5 Likes

Great update, Fabian!
I just installed the latest beta and and it now works like a charm.

Thanks for the effort!

ImageKit 1.0.0 is here

It took longer, than expected, but I finally released ImageKit 1.0.0! There are not many changes from beta2, so the plugin works pretty stable now.

Pricing & Licensing

The plugin is priced $ 25 excluding VAT (the final price will vary a bit, depending on country and currency). A license is valid for one website, bulk discounts are available. TL;DR: The licensing model works very similar to that of Kirby.

Where can I get it?

  • Plugin code and install instructions are available at GitHub
  • Licenses are available via FastSpring (purchase link is also included in readme.md)

I really hope this little Tool will help you, to build even greater websites with Kirby. Stay tuned for autumn/fall, as I am still working on a companion plugin for ImageKit, intended for generating responsive, lazy-loading images using HTML5 and JavaScript (i.e. doing the frontend work).

Have a great day and a nice weekend
Fabian


Changelog

  • 1.0.0 (2016/08/19)
    • Release! Initial version of the plugin is now final. Licenses are availabel at my store.
    • Bugfix: Fix handling of images that are located at the top-level of the content directory.
3 Likes

Next Step: Optimization

I’m currently thinking about the next step for ImageKit. Generating thumbnails asynchronously gives us a little more room for time-consuming operations, i.e. image optimization using command-line utilities. Although this increases thumbnail processing time, it can make a huge difference on image-heavy pages, when loaded over a slow connection.

I’ve been testing the following tools so far:

Do you use any image optimization tools? If so, which ones would you prefer?

1 Like

I’ve tried lots of png compression tools but no one of them made the images near the tiny size of:

The only thing I don’t like about is that it’s a service.

And yes, I’ve made a plugin for that as well:

https://github.com/jenstornell/Kirby-tinyThumb

I’m not sure if it still work.

Anyway, my point is if you will add png compression, try to find something that compress as good as tinypng if possible. Then I might even buy a copy, only for the compression thing.

TinyPNG uses posterization and converts 24-bit PNG files to 8-Bit PNGS. I have been able to achieve similiar results by using the Desktop apps ImageAlpha and ImageOptim. However, this requires some manual input. I think this process could be automated though. I’m just not sure, is posterization (basically limiting the image to a palette 256 colors) is great for every PNG your have, especially when dealing with photography. But who uses PNG images for serving photographs anyway?

JPEG images also profit from posterization, there is a great article on Smashing Magazine on this topic. However, this would require a special thumb driver, because I had to implement my own ImageMagick driver. Though this is not rocket science, I need to considerate carefully, which features are really needed. I might investigate into this at a later point, but for now, I think running tools like Jpegtran or OptiPNG on a thumbnail file after it has been generated is the next and most important step. I really would like to keep ImageKit as compatible with other plugins and the Kirby API as possible. Doing some lossless post-processing on thumb files is rather easy to implement and can still make a big impact on filesizes.

@jenstornell: Maybe TinyPNG could also be added as an optional optimizer in ImageKit. do they support image transforms/resizing etc. or should I upload my resized thumbnails to the service?

Might be a bit off topic, but there was a great talk by Tobias Baldauf at btconf DUS 2016 which is still available on Vimeo. Beside HTTP2 it covers JPG compression as well. It’s worth watching!

1 Like

I’m just not sure, is posterization is great for every PNG your have

I can only tell from my own experience and TinyPNG have never let me down. I’ve used it for screenshots and they included images quite often.

Then there is lossyless PNG compression but that kind of compression does not compress the images very much. Compare it to jpg files where they always are saved with lossy compression (if you don’t save it as 100%).

But who uses PNG images for serving photographs anyway?

That would be “wrong”, except for screenshots of sites including photographs.

Maybe TinyPNG could also be added as an optional optimizer in ImageKit. do they support image transforms/resizing etc. or should I upload my resized thumbnails to the service?

Nope, just compression and there is a limit and you need to pay some bucks for every compression after that.

Maybe start with lossyless compression and build it like component for example and later on you can add some lossy compression optimizers. As long as the user can choose how much the images should be compressed it should be fine. I like lossy, someone else might like lossyless.

Maybe relevant

Google is using artificial intelligence to compress images better than JPEG

Hey @flokosiol, thanks for your links. I never played around with mozjpeg before (it’s what the author of this talk recommonds), but results are pretty impressive and sometimes, generated JPEGs are ever smaller than those created by the TinyPNG/JPEG service at a comparable quality. I did not use custom parameters for the progressive encoding, as Tobias Baldauf did, although it wouldn’t be too hard to use his settings for ImageKit. But the number of scans you want to have in your JPEGs depends on the use-case, so mozjpeg’s defaults should be fine for most people.


The only problem for including it into ImageKit is, that for efficient processing, it does not really make sense to use is as some kind of post-processing filter on the JPEGs, which are generated by Kirby’s thumb drivers. When combining it with the ImageMagick driver, it is possible to pass-through image data without saving it to a temporary file, making the process much faster. For example, one of my test-images commes from a digital SLR camera and has about 10 MP (3888 x 2392 px). To avoid double-compression, it needs to be passed to mozjpeg in an uncompressed format. I achieved the best performance by using both programms it like so:

convert [input file] -resize […] pnm:- | /usr/local/bin/mozjpeg -quality 76 -progressive > [output file]

My first attempt was to write a temporary file (I’m not a command-line hero), but this can take very long. Using an uncompressed TGA file (a format, that mozjpeg can take as input) creates a 30 MB file, that has to be encoded and written to disk. Even on my Retina MBP (which has a fast SSD), this takes much longer than piping the file directly to mozjpeg. Encoding the temporary image as PNM is also much faster, than TGA.

This kind of optimization needs a custom thumb driver for best performance and only works efficently together with the ImageMagick CLI, as SimpleImage (used for image manipulation with PHP’s GD Library) can only save images as PNG, GIF or JPEG (and some other formats, that are not interesting at this point). Saving a 24-Bit PNG takes too much processing time, so saving the image as a high-quality JPEG and then re-encoding it using mozjpeg seems to be the only feasable option for supporting GD library. But hey, if someone can manage to install mozjpeg on his/her webspace, that person should also be able to install imagemagick. :wink:

Conclusion: For best performance and lossy compression of JPEG images, you need both imagemagick and mozjpeg and a custom driver. If a little performance penalty is acceptable, the default driver could be instructed to save a temporary PNM file (by changing the output file’s extension), which can then be passed to mozjpeg. The latter solution also wouldn’t break other plugins (like the focus field) from working. The optimizer should also be able to apply lossless compression when mozjpeg is not available. Tools like jpegtran sometimes come pre-installed on shared hosting, mozjpeg probably not. I’ll do some tests with temporary files to see how this will perform.

@flokosiol: If I need to implement a custom driver, could I use the focus driver as a starting-point? I would really like to keep ImageKit compatible with your wonderful plugin. :slight_smile:


For PNG images however, the case is a little more complicated. I think pngquant (used by ImageAlpha) is the most accessible option for lossy compression. A quick test with a screenshot showed, that quality is just good enough with 256 colors, if compressed with pngquant. My test screenshot had a lot of dock icons and favicons in Firefox’ bookmark bar, so it had a lot of different colors overall. These icons and a photograph openeed in preview had noticable color shifts, but the screenshot file (2880x1800 px) went down from 4.2 MB to 1.1 MB, so for most use cases these color shifts in color should be acceptable. Also, it takes PNM as an input format, so we don’t need to encode it with ImageMagick first (this is good news).

For lossless compression of PNG files, OptiPNG seems to be a good option, if you aim for a good compression ratio. My screenshot went down to 2.6 MB, but it took a while. As I know from using ImageOptim, compression large PNGs takes very long, but the result seems to be worth the effort. With smaller input files, the process of optimization becomes much faster, so this should work in theory. OptiPng also takes PNM images as input, so this tool also fits in with our image processing pipeline :slight_smile:


Okay, now the path seems to be clear how optimization can work. I only need to implement it. The hardest part of that will be to find the right way between simplicity and extensibility. @all: Thanks for your suggestions, it helped a lot!

3 Likes

Sounds great, really looking forward to it!

Yes, of course. Would be a pity if ImageKit and Focus wouldn’t be compatible :wink:

2 Likes

Alright, optimization is (almost) here! :smiley:


ImageKit 1.1.0-beta1

  • Optimization: ImageKit is now capable of applying several optimizations to your images, using popular command-line tools.
  • Better Error Handling: The ComplaingThumb class now handles out-of-memory errors more reliable.

So they don’t get compressed on the fly when adding them or using that widget of yours? What I will probably look for is a way to have all my images compressed without the need to even think about it. Just like Kirby does with thumbs. In my templates/snippets I add code that there is a thumbnail and what size it has. Then when I upload images and use the website I don’t need to think about it anymore.

I want the same thing with compressed thumbnails. A nobrainer. If it’s this plugin or some future plugin, I don’t know. Hopefully I will NOT be the author. :slight_smile:

Thumbnails are optimized as they are generated. This happens automatically as they are processed, if ImageKit is configured to do so. The only thing you have to do is to enable it in your configuration. So in fact, once set up you should clear your thumbs directory and run the thumbnail generation process again from the widget.

Unfortunately I cannot provide the binaries for pngquant etc. because some of them are licensed under GPL and most Open-Source licenses are not compatible with a commercial license like that of ImageKit. Also, this would increase the filesize of the plugin up to many megabytes, because I would have to include binaries for every operating system.

Does this answer your question? :wink:

I’m not sure. You don’t use binaries for it? Do you use the built in quality settings then?

https://getkirby.com/docs/templates/thumbnails

Like:

echo thumb($image, array('width' => 300, 'quality' => 80));

For me the thing is this:

Google still complains about my images, that they are too big:

https://developers.google.com/speed/pagespeed/insights/?url=gratis-smslån.se&tab=desktop

I moved from PNG to JPG (60% quality I think) and saved like 100kB but that did not get Google happier.

For example this image. The image as JPG is only 13kB.

Google says:

You can save 10,2 kB (78 % reduce).

I don’t know if the images can be compressed more or if Google is just a pain in the *** right now. :confused:

RTFM. :slight_smile:

ImageKit does not ship with binaries, but it can make use of the following tools if you have them installed on the server: mozjpeg, jpegtran, pngquant, optipng, gifsicle

4 Likes

Good work!

It would be nice if the plugin also generated WebP images. The browser support is quite good, especially with mobile browsers.

A WebP image will likely have an even smaller file size than an optimized JPG / PNG / GIF.

Using the <picture> element, you can load WebP images in supported browsers:

<picture>
    <source type="image/webp" srcset="image.webp">
    <img src="image.jpg" alt="…">
</picture>

Thanks for your suggestion. I was already condidering this, because adding cwebp as an optimizer is not too hard. It’s a little tricky though, because Kirby’s Thumb class is not intended to encode images in multiple formats. I could generate the corresponding file, but it would not be accessible through Kirby’s API. As far as I know, every browser that supports WebP sends an additional header to the server to indicates support for WebP. You could check for WebP-Support via .htaccess and automatically deliver the webp version instead of the jpeg if the browser supports it without changing the JPEG. However, this would probably not work with CDNs and would introduce other problems, as the same URL could return different files based on different headers in the request.

I fear that this has to wait a little longer, because it requires additional steps and includes more than just generating the image files. In my opinion, this is not too urgend, because mozjpeg can come very close to the filesizes of WebP, unfortunately not for images with alpha transparency. But I’ll keep an eye on this feature.

If you have any ideas, how this could be implemented, please let me know.

Cheers,
Fabian

I don’t think that CDNs will be a problem for ImageKit. As the files are “virtual”, all requests need to go through Kirby anyway, or am I wrong?

Kirby 2.4 will have visitor::acceptance, which may be used to detect if a browser likes WebP more than JPG/PNG/GIF.
You can set the Vary header to Accept to tell proxies and caches to cache the result only if the browser sends the same Accept header.