Virtual Pages with API and Caching

Hi!

I create virtual pages following this guide Content from an API | Kirby CMS
and it works very well.

Since the contents are so many, I try to save in the kirby cache the content of the json as suggested here: Caching | Kirby CMS

ans I set cache variable is valid for 12h.

my config.php file

return [
    'debug' => true,
    'panel.install' => true,
    'cache' => [
        'api' => true
    ]
];

my page models

class ReviewsPage extends Page
{
    public function children()
    {
        $corsi = [];
        $pages   = [];
        $apiKey  = '';
        $user = 'myUser';
        $password = 'MyPassword';

        $kirby = kirby();


        $apiCache = $kirby->cache('api');
        $apiData  = $apiCache->get('apiData');


        if ($apiData === null) {
            $apiData= Remote::get('https://api.website.com/thefile.json', [
              'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
              ]
            ]);
            $apiCache->set('apiData', $apiData->content(), 720);
        }


        if ($apiData->code() === 200) {
            $corsi = $apiData->json(false)->corsi;
        }

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }
}

obviously this model doesn’t work but I can’t figure out what I’m doing wrong, it seems the error is on the line
if ($apiData->code() === 200)

can you help me to create the virtual pages every 12 hours without reloading all the data via api every time?
I hope I made myself clear, thank you very much!

In your cache, you store $apiData->content(). So when $apiData is not null, $apiData->code() will not exist. The check for the response code should go into your if statement.

After that, you just work with the $apiData variable.

something like this?

The page works, but the foreach loop is empty but there should actually be 200 elements.

I check the cache/api/apiData.cache file which has been generated and inside there is all the json data.

class ReviewsPage extends Page
{
    public function children()
    {
        $corsi = [];
        $pages   = [];
        $apiKey  = '';
        $user = 'myUser';
        $password = 'MyPassword';

        $kirby = kirby();


        $apiCache = $kirby->cache('api');
        $apiData  = $apiCache->get('apiData');


        if ($apiData === null) {


            $apiData= Remote::get('https://api.website.com/thefile.json', [
              'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
              ]
            ]);

            

            $apiCache->set('apiData', $apiData->content(), 720);

            if ($apiCache->code() === 200) {
                $corsi = $apiCache->json(false)->corsi;
            }

        }
        

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }
}

No, unfortunately that is a similar mistake, now $corsi is only defined when $apiData is null. And you are using a wrong variable:

public function children()
    {
        $corsi = [];
        $pages   = [];
        $apiKey  = '';
        $user = 'myUser';
        $password = 'MyPassword';

        $kirby = kirby();


        $apiCache = $kirby->cache('api');
        $apiData  = $apiCache->get('apiData');


        if ($apiData === null) {


            $response = Remote::get('https://api.website.com/thefile.json', [
              'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
              ]
            ]);

          
            if ($response->code() === 200) {
                  $apiData = $response->content();
                  $apiCache->set('apiData', $apiData, 720);
            }

        }
        
     
       $corsi = $apiData->json(false)->corsi;

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }

it seems to work partially, now it generates the list but it doesn’t create the virtual pages. in the backend there are no pages and if I try to open one, the line with $courses = $apiData->json(false)->courses gives me error: Call to a member function json() on string

What does this actually return?

A page with a Basic Auth, with the credentials returns a json file. I tested it with Postman and it works.

The original code (content from an API), without the chaching, works very well, but evrey time I open the page it take 6 seconds to load the contents (there are a loto of courses in the json)

content() returns a string.
Therefore you can’t call ->json() on it. You would have to call ->json() on the $response (an instance of the Remote class); but actually, since both the cache and the API call give you strings, it’s easier to just use something like PHPs json_decode on the result of either one.

If you are using Kirby 3.8 or later, we can use $cache->getOrSet() to make the code a bit more concise.
I’d refactor the code like this:

    public function children() {
        $pages = [];

        $apiData = kirby()
            ->cache('api')
            ->getOrSet('apiData', fn() => $this->fetchData(), 720);
     
        $apiData = json_decode($apiData);
        $corsi = $apiData->corsi;

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }

    public function fetchData(): string {
        $apiKey  = '';
        $user = 'myUser';
        $password = 'MyPassword';

        $response = Remote::get('https://api.website.com/thefile.json', [
            'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
            ]
        ]);

        if ($response->code() === 200) {
            return $response->content();
        }

        //@todo: handle error case better than this
        throw "Could not fetch data";
    }

4 Likes

for completeness I’ll show you how I solved it with the help of Chat GPT :face_with_hand_over_mouth:

class ReviewsPage extends Page
{
    public function children()
    {
        $corsi = [];
        $pages   = [];
        $apiKey  = '';
        $user = 'myUser';
        $password = 'myPassword';

        $kirby = kirby();


        $apiCache = $kirby->cache('api');
        $apiData  = $apiCache->get('apiData');


        if ($apiData === null) {


            $response = Remote::get('https://api.website.com/thefile.json', [
              'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
              ]
            ]);

          
            if ($response->code() === 200) {
                  $apiData = $response->content();
                  $apiCache->set('apiData', $apiData, 720);
            }

        }
        
     
        if ($apiData && is_string($apiData)) {
            $jsonData = json_decode($apiData);
            if ($jsonData !== null) {
                $corsi = $jsonData->corsi;
            }
        }

       

        echo count($corsi) . ' courses retrieved from ' . ($apiData ? 'cache' : 'API') . '<br>';

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }

}

this solution seems to work but yours is much more compact. I will do some tests in the next few days to see if everything is ok.

Thanks a lot Rasteiner!

you might consider adding a cache to the virtual pages children. its not in the guide yet but thats how the core kirby logic works as well.