No media files are generated, when media folder is in a different location

[andio]

This particular problem has haunted me since Kirby 3 and unfortunately into Kirby 4. I have already described it in this ticket (at the very end): How to debug media files not beeing generated? - #24 by andio but unfortunately there is still no solution or explanation for it.

That’s why I need help again today!

Problem: Media files uploaded via the panel (images) are not created in the media folder as soon as the media folder is in a location other than the default.

In my case, the media folder is in a different place in the directory structure and was also set correctly via kirby->roots() in index.php and made accessible by the web server as with an Alias or RewriteRule.

The panel files are also created correctly and the rest of Kirby works flawlessly. That means its no problem of permissions or anything like that. Also, gd library is installed and working with other kirby installations (with media folder in the default location).

Only uploaded images are not created and are therefore displayed in the panel as broken images. However, the .jobs files are created, but no images. I suspect that there is an old error somewhere with the wiring of the location of the media folder and the image creation process is aborted.

I am willing to debug this, but I need a starting point. Unfortunately, moving the media folder to the default location is not an option for me due to multi-site setup and each site has its own media folder which is individually set via index.php.

How can I debug the problem that uploaded images are not generated in media folder?

[texnixe]

What does the custom folder setup look like?

[andio]

The structure looks like this:

|-storage
 | |-site1
 | | |-cache
 | | |-content
 | | |-sessions
 | | |-accounts
 | | |-media
 | | | |-panel
 | | | |-plugins
 | | | |-compiled
 | | | | |-styles.css
 | | | |-site
 | | | | |-b32c5ffa87-1723583028
 | | | | | |-.jobs
 | | | | | | |-logotellieblack-76x.png.json
 | | | | | | |-logotellieblack-38x.png.json
 | | | | | | |-logotellieblack-960x-q90.webp.json
 | | | | |-fc03371610-1723582877
 | | | | | |-.jobs
 | | | | | | |-logotelliewhite-352x.png.json
 | | | | | | |-logotelliewhite-864x.png.json
 | | | | | | |-logotelliewhite-76x.png.json
 | | | | | | |-logotelliewhite-960x-q90.webp.json
 | | | | | | |-logotelliewhite-1408x.png.json
 | | | | | | |-logotelliewhite-38x.png.json
 |-sites
 | |-sites.txt
 | |-site1
 | | |-snippets
 | | | |-modules
 | | |-blueprints
 | | | |-site.yml
 | | | |-modules
 | | | |-pages
 | | |-css
 | | |-config
 | | | |-config.php
 | | |-languages
 | | |-templates
 | | |-assets
 |-public
 | |-index.php
 | |-.htaccess
 |-global
 | |-plugins

The files are divided in two categories, one category for the site-specific files like blueprints, templates, snippets, config and so on; and the other category with the storage files which need to be persisted, like content, sessions, cache, media etc.

The plugins are all the same for every site, therefore they live in a global directory.

The index.php looks like this:

$site = "site1"; // this will be set through a mapping table from the host name in the request
$storage = realpath(__DIR__ . '/../storage/' . $site);

$roots = [
    'index'         => __DIR__,
    'site'          => realpath(__DIR__ . '/../sites/' . $root),
    'plugins'       => realpath(__DIR__ . '/../global/plugins'),
    'storage'       => $storage,
    'content'       => $storage . '/content',
    'accounts'      => $storage . '/accounts',
    'cache'         => $storage . '/cache',
    'sessions'      => $storage . '/sessions',
    'media'         => $storage . '/media',
];

Output of command dump(kirby()->roots());:

Kirby\Cms\Ingredients Object
(
    [kirby] => /var/www/kirby
    [i18n] => /var/www/kirby/i18n
    [i18n:translations] => /var/www/kirby/i18n/translations
    [i18n:rules] => /var/www/kirby/i18n/rules
    [index] => /var/www/public
    [assets] => /var/www/public/assets
    [content] => /var/www/storage/site1/content
    [media] => /var/www/storage/site1/media
    [panel] => /var/www/kirby/panel
    [site] => /var/www/sites/site1
    [accounts] => /var/www/storage/site1/accounts
    [blueprints] => /var/www/sites/site1/blueprints
    [cache] => /var/www/storage/site1/cache
    [collections] => /var/www/sites/site1/collections
    [commands] => /var/www/sites/site1/commands
    [config] => /var/www/sites/site1/config
    [controllers] => /var/www/sites/site1/controllers
    [languages] => /var/www/sites/site1/languages
    [license] => /var/www/sites/site1/config/.license
    [logs] => /var/www/sites/site1/logs
    [models] => /var/www/sites/site1/models
    [plugins] => /var/www/global/plugins
    [sessions] => /var/www/storage/site1/sessions
    [snippets] => /var/www/sites/site1/snippets
    [templates] => /var/www/sites/site1/templates
    [roles] => /var/www/sites/site1/blueprints/users
    [storage] => /var/www/storage/site1
)

media files made accessible through Apache rewrite rules:

RewriteMap dynamicmap txt:/var/www/sites/sites.txt

# RewriteEngine einschalten
RewriteEngine On

# Entferne die Portnummer aus dem Hostnamen, falls vorhanden
RewriteCond %{HTTP_HOST} ^([^:]+)
RewriteRule ^ - [E=HOST_WITHOUT_PORT:%1]

RewriteCond %{REQUEST_URI} !^/media/plugins
RewriteCond ${dynamicmap:%{ENV:HOST_WITHOUT_PORT}} ^(.+)$
RewriteRule ^/media/(.*)$ ${APACHE_DOCUMENT_ROOT}/../storage/%1/media/$1 [L]

RewriteCond ${dynamicmap:%{ENV:HOST_WITHOUT_PORT}} ^(.+)$
RewriteRule ^/assets/(.*)$ ${APACHE_DOCUMENT_ROOT}/../sites/%1/assets/$1 [L]

Like said before the rest of the media files work fine, for example the auto-generated panel files. It’s only the uploaded images in the panel which don’t work.

Hopefully that helps.

[texnixe]

@andio

Like said before the rest of the media files work fine

What do you mean? Do thumbs/images work in the frontend work?

Is that the complete index.php, what about the urls property that needs to be defined for a custom folder setup as well, see Multisite configuration | Kirby CMS

[andio]

No, i meant that the rest of the files like media/panel for the css, js, img etc. work fine, for example when i remove the contents of the media folder, it will be regenerated as expected with all those files on the next page load.
The problem only occurs when uploading images via the panel.

I have not changed the URLs because it is still ‘/media’ and is made accessible via a RewriteRule from Apache (see above). Access to files in the /media folder also works, i.e. I can also reach the other files (e.g. vue files of the panel). It’s really only the images which are not generated after uploading via the panel and saving in the content folder (but the .jobs files are there):

[andio]

Update: I found something out.
The image seems to be generated on the first access, not after upload when the .jobs files are generated. I was able to find a solution in adding another Rewrite Condition like this:

    RewriteCond %{REQUEST_URI} ^/media/site/
    RewriteCond ${dynamicmap:%{ENV:HOST_WITHOUT_PORT}} ^(.+)$
    RewriteCond ${APACHE_DOCUMENT_ROOT}/../storage/%1%{REQUEST_FILENAME} -f
    RewriteRule ^/media/(.*)$ ${APACHE_DOCUMENT_ROOT}/../storage/%1/media/$1 [L]

That excludes the /media/site files from directly loading from the media folder, when the file doesn’t exist, but instead it routes the through the index.php to get the file created. On the second request, when the file is there, then it will directly serve the file from the media folder.

The approach for multi-site published on the Kirby website you mentioned earlier has one major disadvantage (and security issue from my point of view): the media folder must be located within the DocumentRoot and is therefore publicly accessible, as are all site-specific subdirectories!

The consequence of this is that files can be accessed from other sites, something like cross-site access, for example site1.de/media/site2/file and that could be a security issue.

In my approach the media folders are outside of DocumentRoot and only accessible per site. If you are interested I can provide the whole approach based on the modified index.php and the RewriteRules based on a RewriteMap.