Error thrown when resizing series of images via hook in panel

Hi there,

Iā€™ve been having some issues with resizing images on upload which have been really driving me crazy.

Basically all I want to acheive is this: when images are uploaded, resize each one so I can generate a src-set of images to be used in the frontend. I have acheived this with a hook (originally found here), implemented in my config as follows:

<?php

return [
  'thumbs' => [
    'driver' => 'gd',
  ],
  'hooks' => [
    'file.create:after' => function($file) {
      $widths = [25, 600, 900, 1200, 1600, 2000];
      if ($file->isResizable()) {
        foreach($widths as $width) {
          try {
            $resizedImage = $file->resize($width, null, 80)->save();
          }
          catch (Exception $e) {
            throw new Exception($e->getMessage());
          }
        }
      }
      else {
        throw new Exception('the file is not resizable');
      }
    }
  ]
];

?>

My problem is this: my client will likely upload multiple images at a time (approx. 10-20), and on my server attempting to do always throws the following error when so many images (all between 1-2MB) are uploaded, this usually occurs on the same set of images:

The file could not be uploaded

As I said, the images are roughly the same size in memory. The only strange thing I noticed is this typically happens to the images with the largest pixel dimension, for now I can only assume this is a coincidence, as I have already set max dimensions in my file blueprint (both height and width) to be way above the size of any of the images.

When large images are uploaded individually, there is no problem and all resized images are generated. There is also no issue when attempting to upload in bulk on my local server.

Of course, the first thing I thought this could be would be server time, but even after setting the following variables to be ridiculously generous in my php.ini (see below) I still have the same problem.

I am running php 7.2 on apache, below are the relevant values in my php.ini:

max_execution_time = 5000M
max_file_uploads = 30
max_input_time = -1
memory_limit = 1000M
post_max_size = 500M
upload_max_filesize = 20M 

GD library is installed, as are all other necessary modules for Kirby. It is important for my client this works on upload, so I would like to stick with this rather than them being generated on the fly in the frontend.

Any help is much appreciated!

Nick

Are this your localhost ini settings on the ones on your clientā€™s server?

Setting execution time etc. to phantasy values is of little value if not supported on the server.

This was the .ini on my server. I have just tried updating it to match my localhost exactly:

max_execution_time = 30
max_file_uploads = 20
max_input_time = -1
memory_limit = 128M
post_max_size = 8M
upload_max_filesize = 5M

And the issue still prevails. I have also noticed that Kirby is generally slower on my server than my local machine, not sure how significant that is for now as I am not getting anything my error log.

Itā€™s hard to tell but my first guess would be that you run into a max_execution_time issue. Most shared hosting services are quite limited in that respect.

I had thought the same thingā€¦however as Iā€™m running a $5 p/m droplet from digitalocean, it seems like there is no limit on this variable. I would have also thought if this was the case I would get some kind of timeout error in the log itself?

Yes, thatā€™s true.

Itā€™s very strange.

Right now Iā€™m wondering if there is a way to solve this in the loop which saves resized images. As processing each image individually works OK, perhaps there is a way to synchronously create and save() each new size before attempting the other one (i.e. there would be a php equivalent of a callback for the ā€˜saveā€™ function). I would have an idea how to do this in javascript, but in PHP and with Kirbyā€™s api it is beyond meā€¦

I just did a test myself on shared hosting (Uberspace) with 25 images in one go all between 2.4 and 4.5 MB and dimensions of 4368x2912 pixels without any issues.

They have these ini settings:

max_execution_time: 600
max_file_uploads = 20
max_input_time = 600
memory_limit = 256M
post_max_size = 2048M
upload_max_filesize = 2048M

The things is that the error message is very unspecific. ā€œThe file could not be uploadedā€ is thrown in line #747 in /kirby/src/api/Api.php. That line checks if the returned error is part of the following

 $errorMessages = [
            UPLOAD_ERR_INI_SIZE   => t('upload.error.iniSize'),
            UPLOAD_ERR_FORM_SIZE  => t('upload.error.formSize'),
            UPLOAD_ERR_PARTIAL    => t('upload.error.partial'),
            UPLOAD_ERR_NO_FILE    => t('upload.error.noFile'),
            UPLOAD_ERR_NO_TMP_DIR => t('upload.error.tmpDir'),
            UPLOAD_ERR_CANT_WRITE => t('upload.error.cantWrite'),
            UPLOAD_ERR_EXTENSION  => t('upload.error.extension')
        ];

ā€¦and if not, it throws this default error message. I actually donā€™t know if there are any other possible error messages, it might be worth a test to see what $upload[ā€˜errorā€™] actually returns by temporarily modifying this line.

That seems to be a good pace to start, although replacing:

$errorMessage = $errorMessages[$upload['error']] ?? t('upload.error.default');
throw new Exception($errorMessage);

with the following:

$errorMessage = $errorMessages[$upload['error']] ?? t('upload.error.default');
throw new Exception($upload['error']);

Still returns the same ā€˜defaultā€™ exception (The file could not be uploaded)ā€¦

Maybe then there is nothing more specific to get out of these error messages, after all.

Iā€™m runnning out of ideas. How many images can you actually upload in one go without running into this issue? Only one? Itā€™s a lot of stuff you are trying to do there, uploading 20 images and creating 6 thumbs for each.

Could you try with ImageMagic instead of GD library?

Perhaps youā€™re right.

Trying with ImageMagick inconviently throws another error:

The imagemagick convert command could not be executed: convert ā€œ/var/www/drf-dev/public_html/cms/media/pages/home/test-test/2632827851-1583677644/bindeglied-185x155cm-2000x-q80.jpgā€ -strip -auto-orient -resize 2000x2384! -quality 80 -limit thread 1 ā€œ/var/www/drf-dev/public_html/cms/media/pages/home/test-test/2632827851-1583677644/bindeglied-185x155cm-2000x-q80.jpgā€

For this one, in my error log I simply get

Killed

Which of course isnā€™t particularly useful.

Maybe you donā€™t have permissions to execute that command from PHP? Have you made sure that ImageMagick is actually installed and enabled? Does it work on the command line?

It works from the command line fine. Actually I found that yet again it works when it is only one image - Iā€™m now trying to work out if there is a limit of images it can handle at once. At this point I can only really assume that the server can not handle it.

Seeing as I may need to make a compromise here, is it possible to limit the amount of uploads at a time in Kirby itself? I know I could change the php.ini file, but would rather control as much as possible in the cms itself.