Links to pages (created with link-field plugin) use page ID rather than page URL, breaking translations

Kirby 3 Questions 🤔
,
1

I have a page with links in it.

In the backend, these links are represented as fields of type pages (thanks to the link-field plugin), so Kirby gives me a dialog to go through my directories and find the page I want to link to.

Another thing I have is a custom route which fetches the above page with code that looks like this:

$pages = $this->pages();
if ($pages->count() > 0) {
  $template = $pages->findByIdRecursive($ref);
  if (isset($template)) {
    $content = $template->readContent();
    …

Here, $content contains the content of my page full of links, including the links themselves, which look something like this:

{
  "type": "page",
  "value":"PAGEID"
}

I can use this in my non-Kirby-powered frontend to produce HTML links in the form: <a href="HOST/LANG/PAGEID" />

This doesn’t work for translated pages with translated slugs, though, because the URL for these pages is NOT based on the page ID, but rather based on those translated slugs!

How can I get the links to be saved using their actual translated URL slugs rather than their ID?

2 Likes
2

Reading the content is not the right way here. You would have to transform your IDs to URLs, like Kirby does it if you create standard PHP templates (use a custom method that only returns the path if the domain is different).

3

I don’t understand. How can transform the IDs to URLs without first getting the IDs from the content?

4

I have a similar issue. Not sure to understand neither. Can you point to docs or specific examples @texnixe ?

5

What I meant is to not simply return the content, but actually return the data you need. So you need some transformation on that data before returning it to the frontend.

6

So every time I load a page I have to run through the whole content and fix the links manually?

This is going to slow things down tremendously.

Is there not a way to just save the correct data to the content in the first place?

7

Well, you don’t have to do this manually, check out this plugin method for an example: https://github.com/robinscholz/better-rest/blob/eb778f475ef3dd522acea14d8798ee0a10b69499/classes/Betterrest.php#L119

So depending on field type, you could automate this.

8

Well, manually may be the wrong word, but even in the case of this plugin, they are traversing the entire content recursively to fix things…

Again, is there not a way to just get the correct data in there AT SAVE TIME?

9

Maybe you can do it with a page model that overrides the writeContent method to store something different for non-default languages.

In any case, it needs additional work.

10

Wow, I really get the feeling that if I were to fix all the problems I’m seeing, nothing would be left of the original Kirby code.

Listen, I certainly appreciate your help here, but I do feel cheated.
Kirby was advertised as headless-ready and multilingual-ready CMS and neither of these feel remotely true.

Now I’m just stuck having to program all of this stuff myself in an extremely poorly documented codebase. Very frustrating. This whole project has been a HUGE hassle every time Kirby is involved and I just feel like I should be warning everyone I meet against using it.

I know none of this is your fault and you have been very responsive on here, @texnixe , but for anyone coming across this, the only advice I can give is: Kirby CMS is most definitely NOT ready to be used headless.

11

Hello :wave:

Sorry, I didn’t get why you can’t do something like this:

//get the page with the links
if($linksPage = page('page-with-links')) {
  //assuming you used a structure field to store your links
  $urls = $linksPage->links()
              ->toStructure()
              ->toArray(fn($item) => $item->linkField()->toLinkObject()->href());

  return array_values($urls);
}

throw new Exception("Links page not found");

just curiosity…

12

I could indeed.

In fact, I did end up with something like this, but it’s incredibly wasteful to have to traverse the whole content like this on every page load. Page load slowness is what my website visitors feel. I’d much rather do the heavy lifting when saving the data and not having to transform it on page load.

Also, like you said, this may depend on the links being in a structure (the docs don’t make this clear), which is not always the case.

For the curious, this is the code I ended up with. To be clear I don’t recommend using this. I’m not very good at PHP and the docs make the Kirby APIs extremely hard to use correctly. Also, I did try 100 simpler things, but this is the only thing I found that works.

[...]
$data['templates'] = array();
if (isset($pageData['templates_ref'])) {
  $refs = explode("\n", preg_replace('/(- >\n {2}|- )/', '', $pageData['templates_ref']));
  foreach ($refs as $ref) {
    $pages = $this->pages();
    if ($pages->count() > 0) {
      $template = $pages->findByIdRecursive($ref);
      if (isset($template)) {
        $content = $template->readContent();
        $content = walk($pages, $languageCode, $template, $content);
        $data['templates'][$ref] = $content;
      }
    }
  }
}
[...]
function walk($pages, $languageCode, $template, $content, $previousKey = null, $previousContent = null)
{
  if (is_array($content) || is_object($content)) {
    $newContent = array();
    foreach ($content as $key => $value) {
      if ($key === 'type' && $value === 'page') {
        $page = $pages->findByIdRecursive($content->value);
        if ($page) {
          $translations = $page->translations();
          if (isset($translations)) {
            foreach ($translations as $k => $v) {
              if ($k === $languageCode) {
                $translatedUrl = (gettype($v) === 'string')  ? $v : $v->parent()->urlForLanguage($languageCode);
                $matches = array();
                preg_match('/(https?:\/\/)?[^\/]*\/(.*)/', $translatedUrl, $matches);
                $content->value = $matches[2] ? $matches[2] : $content->value;
              }
            }
          }
        }
      }
      [...]
    }
    return $newContent;
  } else [...]
};

Also, keep in my I’m using Kirby as a headless CMS, so that also changes things.

13

Could you share the blueprint of the page containing the links and the output you’re expecting from your route (I guess it’s some json)?

Well, it depends on your needs, it’s your choice to put them in a structure field or not. If your page ever contains only one link, putting it under a structure field wouldn’t make much sense.

14

Basically, when I want a link somewhere I extend this field blueprint:

label: Links
type: group

fields:
  label:
    label: Label
    type: writer
    inline: true
  link:
    type: link
    label: Link
    options:
      - page
      - url
      - email
      - tel
    pages:
      limit: 99
      image: false
      multiple: false
      translate: true
      query: site.children().unlisted().add(site.children().listed()).filterBy('intendedTemplate', 'not in', ['list', 'of', 'exclusions'])

The problem occurs right here, before any routes are involved. This pages thing is saving the links to pages as page IDS, forcing me to do tons of extra work on every page load:

  • find all occurrences of this link field
  • find the page associated to each ID
  • find the URL for those page in the language requested by the user
  • modify the content to use this translated URL instead of the ID before passing that along to the frontend

My route is actually a problem too: it shouldn’t be needed. My custom frontend gets the data for a page by making a request to the Kirby backend at KIRBY/pages/slug+but+with+plus+signs. Unfortunately, this doesn’t give me half the data I need to build the page, so I made a custom route to handle everything that’s missing: information on available locales, the URL for the frontend which I provide in the Kirby config with the siteUrl option, all the translations of the page and their URLS, the content of reusable templates for which all I’m getting is a reference, etc. All things a real headless CMS would know to include.

In any case I’m expecting (and getting) JSON

15

Yeah, fascinating :slight_smile:

As far as i know, the REST API is kind of tailored to the panel, so it only returns what is necessary for that use.
For headless setups I think the KQL interface would be somewhat better suited, it takes some inspiration from GraphQL and adds its own twist based on the Kirby query language. Another option would have been content representations, they would let you return a view of your content based on its type (intendedTemplate).

I guess it’s late for switching now, so the route approach will be just fine, I think.

But since we’re here and before I could try to answer you, I’d love to pick your brain about two questions I had while reading your response:

  1. “find all occurrences of this link field”.
    What do you mean? Does this mean you need to return every instance of any link field in the whole content base to respond to the route? Because otherwise, based on the blueprint, it would only be a single instance.
  2. “the content of reusable templates for which all I’m getting is a reference”?
    What do you mean by “template”? Sorry, I’m not very versed in this computery stuff… You mean the source code of the files in site/templates?

About the “tons of work” for the URL lookup, I think the pages field stores ids, rather than urls, for data normalization reasons. I’m no expert at anything, but I guess that Kirby takes any chance it gets to simplify data integrity. In fact, I think that in future Kirby will go even further in that direction by adopting completely content agnostic universal Ids, instead of urls, to identify pages (it’s currently the most requested feature). I think that looking up a page by its id currently already manages to be rather fast by taking advantage of the natural filesystem indexing.

PS:

query: site.children().unlisted().add(site.children().listed())

could also be written as site.children()

16

This is all very interesting, but very hard to discover due to the extremely poor quality of the documentation…

Unfortunately, it is a bit late to change things at the moment, and also I have absolutely zero intention of ever using Kirby again if I can avoid it.

That being said, I do appreciate your help here and I’ll be bookmarking this answer to read again if I’m ever forced into using Kirby.

So to answer your questions:

  1. I fetch the content for all my pages the same way. In other words, there isn’t a single problematic blueprint here, they all are. So for every page, I have to go through all the content recursively, find the links, fetch their translated URL, replace the IDs with the URLs, and finally return the transformed content. That’s what the code posted earlier does: in particular, notice the walk recursive traversal function.
  2. A lot of pages need to reuse bits of content (header, footer, promotional banner, etc.) so instead of duplicating that data in each page, we save it once in a separate page and just “refer” to it (save the ID as a reference) in order to find it again at page load. This is basically data normalization, to use your terminology. I am misusing the terms “reusable template” here, so maybe that’s why you’re confused.

Finally, I understand the need for unique IDs for data normalization, but page IDs aren’t unique! All the translations share the same URL! Each translated page knowing about its own URL wouldn’t violate data normalization at all. Also, data normalization is one thing you might want to optimize, but there are others! Page load performance is a big since it has huge impact on conversion, ecological impact, and perceived professionalism (in well-connected areas like the western world). When you optimize both data normalization and performance at once, you end up with a database, not a file system. So in Kirby land, this two-factored optimum will never be obtained. Kirby will quite simply never be optimally performant. And I won’t even get into the problems associated with PHP…

The other thing to keep in mind when talking about performance is that you’ll often want to choose where you can tolerate decreased performance. I’d much rather the slowness happened at save time instead of page load time. Even to the point of breaking data normalization, yes, as long as I can fully automate myself out of my denormalized data problems. Indeed, I wouldn’t want to force my users to make a change in multiple places, but my backend could be organized to handle this seamlessly, atomically, consistently. In fact, this is exactly how databases achieve their increased performance! There are indexes which, yes, duplicate some of the data, but in a way that still allows having a single source of truth and is always in perfect sync thanks to the very well tested synchronization techniques put in place on the database side (as opposed to being implemented on the app side where a developer could make a mistake). Notice how different this is from Kirby-land where the developer basically has to do everything themselves in PHP, which makes for a high risk of error.

P.S. thanks for the tip about site.children(), however that is not what I observed in the past, which is why I ended up with all this mess… Of course, this might just be a difference between Kirby v3.5.8 (which I use) and v3.6.x…

By the way, the fact that these versions are incompatible is a violation of the semantic versioning pattern, which I highly recommend. It’s just so much easier when you can look at the numbers and immediately know if things are going to break or not, especially when the docs are so bad.

18

So, for the people that still are forced into using Kirby :roll_eyes:, here’s some general guidance on how to use Kirby, completely opinionated and based solely on my experience:

  1. Kirby is file based. Other than the one provided by the filesystem, it doesn’t use an index. Therefore looking up pages by id is fast. Looking up pages by url is slow(er). This is because id’s map to file paths, urls map to properties saved in files. If however you need to index pages by their content, there are plugins that do this.

  2. If you use Kirby, use the Kirby API. For example; if you want to read data from a content file, use $page->fieldname(). If you want to find a page by id, use page('id'). If you have a pages field and want to get to the selected page, use $field->toPages(), etc.
    Kirby caches file contents in memory, writing your own logic to parse content files just circumvents this cache.

  3. Prefer caching strategies over denormalization strategies. As pointed out by shawninder, Kirby doesn’t provide built-in mechanisms to keep your redundant data in sync. This is a consequence of being file based and therefore not having the luxury of having the possibility of scanning potentially tens of thousands of files on each save (see §1).
    Normally you request only what you need, realistically therefore if you request the data for one page, you don’t need to go over all pages in your content folder, but (for example) only over those linked by the requested page. Since the lookup of those pages is done by file name, the realistic impact on response time is negligible. If for some reason it is not, caching the result is the best option.

  4. The versioning of Kirby goes like this:
    GENERATION.MAJOR.MINOR.PATCH.

    1. The sole purpose of GENERATION is the license. You buy a Kirby 3 license. It will be valid for all “3.x.x.x” versions
    2. MAJOR versions contain breaking changes
    3. MINOR versions fix stuff and add functionality
    4. PATCH versions are there only for urgent fixes of regressions or security issues, that cannot wait for the normal release cycle.

update: fixed version naming

19

Everything you write is correct, only one note regarding the naming of the versioning parts:

We use GENERATION.MAJOR.MINOR.PATCH with an optional patch number (as explained on the 3.5 release page in the “Breaking changes” section). So 3.6.3 is generation 3, major version 6 and minor version 3. This lines up with semantic versioning with only the 3 for the generation added in front.

As you write, the reason we do this is licensing. To bring Kirby forward, we need to make breaking changes from time to time (otherwise our codebase would get dusty and messy pretty quickly). But we don’t want to charge users for a new license every few months. :slight_smile:

1 Like
20

I’ve tried to read through all your topics to understand the problems you are facing. We always try to be as helpful as possible and spend a lot of time with onboarding new users. I think you saw that in tons of replies from @texnixe and various other people in the community over the last 6+ months. That’s why we also offer an unlimited free trial period. We want you to be happy with the system first. We are aware that not all systems work for all projects. We are also aware that some parts of the docs can be improved. Yes, there’s room for improvement for our API docs.

To blame it all on Kirby is just not fair in your case though. I seriously tried to understand your points but in pretty much all your topics you try to do something that we never advertised, that are hacks or simply bad practice:

  • You tried to put Tailwind into the admin panel that does not use Tailwind and asked for Tailwind support in our forum
  • You tried to use React components in a Vue.js application and asked for support in our forum
  • You ignored better options to create custom tailored-endpoints with content representations or KQL and you use internal Kirby methods instead to do something with code that I seriously don’t understand, even after reading it twice.
  • You come up with a weird hack for site.children() and you claim it didn’t work in previous versions when this is a method that has been around since 2012.
  • You complain about us not using semver, when obviously not even checking our versioning system.
  • You blamed network config issues first on Kirby and then on PHP and then on our docs.
  • You admit that you are not very good at PHP, but you also blame PHP for being PHP and Kirby for using PHP.

I’m really sorry. We have a lot of patience in here. We are always willing to admit when things are not perfect and we try to fix them. We are also happy to help far beyond what would normally be part of the typical technical support. We often even help out with personal technical consultancy in urgent situations when someone gets completely stuck.

It’s pretty obvious that Kirby is not fitting your project and/or your mental model. I think that’s absolutely fine btw. I even get your frustration for being stuck. But I seriously wonder why this is they way you choose to deal with it. Just get in contact: support@getkirby.com and I’ll send you a refund

2 Likes
21

Kirby is advertised as headless-ready and multilingual-ready, which it isn’t. That’s the source of most of my issues I believe. That and the docs.

By the way, if you advertise your CMS as having support for headless, then going headless is not supposed to be hacky or bad practice…

It’s my choice, in a headless situation, to use any CSS libraries or frontend frameworks that I want, including Tailwind. For the previews to look like the frontend, I need to bring these technologies into the panel, a process that Kirby makes very difficult. That’s not very headless at all.

You mention you can’t understand some code of mine even after reading it twice. Well, I wouldn’t have had to write that code if the CMS was truly headless-ready. Also, the code looks like it does because of YOUR Models and API! Apart from using your API, all I’m doing there is a recursive traversal function. So unless you’ve never seen recursive traversal functions, the complexity is due to Kirby itself! I wish I had found a better solution, but I didn’t.

Now, I can also look back at my posts in this forum and remind us of all the difficulties I’ve had that are NOT due to a hack or non-standard approach on my part:

  • The inability to use localhost for development, despite that not being indicated in the docs
  • Multiple, different things sharing the same name and causing confusion (Authentication vs Authentication, $page->url va $page->url, this list is actually enormous)
  • The difficulty of having to do programming work in a wild-stab-in-the-dark fashion using yml + a limited query language without the modern programming tools like IDE integrations, syntax highlighting, types, docs, examples, logging, error management, environment variables, etc.
  • The untestable nature of anything related to the query language
  • The inability to get all pages from a single non-custom API endpoint (basic requirement for a true headless CMS).
  • The inability to import third-party scripts or styles into the panel View.
  • The mysterious use of an undocumented function this.field in Vue computed functions in many examples.
  • Needing to use a hack to get the alt text for my images
  • The lack of basic capabilities in the query language, like ternaries, OR operators, etc.
  • The inability to get the translated URLs for links without having to traverse the content
  • The search on getkirby.com that doesn’t find things like $page when searching for page
  • The broken writer options marks and inline that don’t behave how the docs suggest and in some cases simply don’t work at all (I forget the details here, I just stopped using them)
  • The inability to see the docs for older versions of Kirby from the website
  • Kirby destroys my variable names (by converting everything to lowercase) between the backend and the frontend, preventing me from using my preferred capitalization strategy: camelCase.

I could go on all day like this. I understand that I can write custom API endpoints to resolve some of these issues, but what’s the point of a CMS if I have to do everything myself and don’t even get to choose my programming language?

We can also discuss best practices if you want:

  • semver is a best practice, and nothing is forcing you to have your licensing strategy collide with best practice versioning.
  • The way config works makes it impossible for my production config to survive a URL change at the DNS level
  • Docs should have examples, just saying that a function returns $mixed args is pretty much useless.
  • Everything being a page in Kirby makes simple things hard to do because non-page items keep cropping up in page-related situations.
  • scrolling a full-height <main> element rather than the body itself, which breaks expectations for any scroll-related features like scroll-snapping in the panel.
  • wrapping single images in arrays (and other things) for no obvious reason: just makes it harder to access the data from the frontend
  • Getting locked out of my LOCAL DEV instance of Kirby because of rate-limiting is definitely not best practice
  • The lack of any kind of migration strategy for content structure updates is surprising for structured content management
  • The amount of bad UI/UX that can be found in the panel fields is astonishing and beyond the scope of this superficial “review”

There is a LOT more I could say here too, but I have to stop somewhere.

To be clear, I’m not trying to start a flame war here, only to justify why I do blame Kirby for wasting months of my time.

I have also been demonstrating tremendous patience here dealing with all of the above and so much more. But since Kirby is by far the biggest reason why my project is late, over budget, of poor quality, not reusable and a continuous source of frustration, I do consider Kirby harmful and will avoid it as much as possible in the future, for myself and anyone else who asks for my advice.

I appreciate the offer to send me a refund, but no one can refund the months of pain I’ve been going through trying to make a Cathedral out of chopsticks and destroying my schedule and happiness in the process.

To be clear, the free trial would indeed have steered me away from Kirby on the first minute of day one as a cursory glance at the docs is enough to turn me away. However, I was involved in this project too late to change that decision.

Ok, let’s bring this back to things we can all do to improve our lives. On my end, I will be moving away from Kirby as soon as possible and that will solve the vast majority of my problems with this project. In the meantime, I have sacrificed performance and used all sorts of hacks everywhere and I have something that sort of works…
On your end, you seem to be satisfied with Kirby as it is, so I guess you don’t have to do anything, but here is what I would propose based on my experience:

  • Don’t claim to be headless-ready
  • Don’t claim to support multilingual setups out-of-the-box
  • Fix + improve your documentation