Virtual Pages with API and Caching

Reluma · March 14, 2023, 3:01pm

Hi!

I create virtual pages following this guide Content from an API | Kirby CMS
and it works very well.

Since the contents are so many, I try to save in the kirby cache the content of the json as suggested here: Caching | Kirby CMS

ans I set cache variable is valid for 12h.

my config.php file

return [
    'debug' => true,
    'panel.install' => true,
    'cache' => [
        'api' => true
    ]
];

my page models

class ReviewsPage extends Page
{
    public function children()
    {
        $corsi = [];
        $pages   = [];
        $apiKey  = '';
        $user = 'myUser';
        $password = 'MyPassword';

        $kirby = kirby();


        $apiCache = $kirby->cache('api');
        $apiData  = $apiCache->get('apiData');


        if ($apiData === null) {
            $apiData= Remote::get('https://api.website.com/thefile.json', [
              'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
              ]
            ]);
            $apiCache->set('apiData', $apiData->content(), 720);
        }


        if ($apiData->code() === 200) {
            $corsi = $apiData->json(false)->corsi;
        }

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }
}

obviously this model doesn’t work but I can’t figure out what I’m doing wrong, it seems the error is on the line
if ($apiData->code() === 200)

can you help me to create the virtual pages every 12 hours without reloading all the data via api every time?
I hope I made myself clear, thank you very much!

moonwalk · March 14, 2023, 3:22pm

In your cache, you store $apiData->content(). So when $apiData is not null, $apiData->code() will not exist. The check for the response code should go into your if statement.

After that, you just work with the $apiData variable.

Reluma · March 14, 2023, 4:28pm

something like this?

The page works, but the foreach loop is empty but there should actually be 200 elements.

I check the cache/api/apiData.cache file which has been generated and inside there is all the json data.

class ReviewsPage extends Page
{
    public function children()
    {
        $corsi = [];
        $pages   = [];
        $apiKey  = '';
        $user = 'myUser';
        $password = 'MyPassword';

        $kirby = kirby();


        $apiCache = $kirby->cache('api');
        $apiData  = $apiCache->get('apiData');


        if ($apiData === null) {


            $apiData= Remote::get('https://api.website.com/thefile.json', [
              'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
              ]
            ]);

            

            $apiCache->set('apiData', $apiData->content(), 720);

            if ($apiCache->code() === 200) {
                $corsi = $apiCache->json(false)->corsi;
            }

        }
        

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }
}

moonwalk · March 14, 2023, 4:44pm

No, unfortunately that is a similar mistake, now $corsi is only defined when $apiData is null. And you are using a wrong variable:

public function children()
    {
        $corsi = [];
        $pages   = [];
        $apiKey  = '';
        $user = 'myUser';
        $password = 'MyPassword';

        $kirby = kirby();


        $apiCache = $kirby->cache('api');
        $apiData  = $apiCache->get('apiData');


        if ($apiData === null) {


            $response = Remote::get('https://api.website.com/thefile.json', [
              'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
              ]
            ]);

          
            if ($response->code() === 200) {
                  $apiData = $response->content();
                  $apiCache->set('apiData', $apiData, 720);
            }

        }
        
     
       $corsi = $apiData->json(false)->corsi;

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }

Reluma · March 15, 2023, 8:58am

it seems to work partially, now it generates the list but it doesn’t create the virtual pages. in the backend there are no pages and if I try to open one, the line with $courses = $apiData->json(false)->courses gives me error: Call to a member function json() on string

pixelijn · March 15, 2023, 9:09am

What does this actually return?

Reluma · March 15, 2023, 9:53am

A page with a Basic Auth, with the credentials returns a json file. I tested it with Postman and it works.

The original code (content from an API), without the chaching, works very well, but evrey time I open the page it take 6 seconds to load the contents (there are a loto of courses in the json)

rasteiner · March 15, 2023, 1:41pm

content() returns a string.
Therefore you can’t call ->json() on it. You would have to call ->json() on the $response (an instance of the Remote class); but actually, since both the cache and the API call give you strings, it’s easier to just use something like PHPs json_decode on the result of either one.

If you are using Kirby 3.8 or later, we can use $cache->getOrSet() to make the code a bit more concise.
I’d refactor the code like this:

    public function children() {
        $pages = [];

        $apiData = kirby()
            ->cache('api')
            ->getOrSet('apiData', fn() => $this->fetchData(), 720);
     
        $apiData = json_decode($apiData);
        $corsi = $apiData->corsi;

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }

    public function fetchData(): string {
        $apiKey  = '';
        $user = 'myUser';
        $password = 'MyPassword';

        $response = Remote::get('https://api.website.com/thefile.json', [
            'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
            ]
        ]);

        if ($response->code() === 200) {
            return $response->content();
        }

        //@todo: handle error case better than this
        throw "Could not fetch data";
    }

Reluma · March 16, 2023, 8:51am

for completeness I’ll show you how I solved it with the help of Chat GPT

class ReviewsPage extends Page
{
    public function children()
    {
        $corsi = [];
        $pages   = [];
        $apiKey  = '';
        $user = 'myUser';
        $password = 'myPassword';

        $kirby = kirby();


        $apiCache = $kirby->cache('api');
        $apiData  = $apiCache->get('apiData');


        if ($apiData === null) {


            $response = Remote::get('https://api.website.com/thefile.json', [
              'headers' => [
                'Authorization: Basic ' . base64_encode($user . ':' . $password)
              ]
            ]);

          
            if ($response->code() === 200) {
                  $apiData = $response->content();
                  $apiCache->set('apiData', $apiData, 720);
            }

        }
        
     
        if ($apiData && is_string($apiData)) {
            $jsonData = json_decode($apiData);
            if ($jsonData !== null) {
                $corsi = $jsonData->corsi;
            }
        }

       

        echo count($corsi) . ' courses retrieved from ' . ($apiData ? 'cache' : 'API') . '<br>';

        foreach ($corsi as $key => $review) {
            $pages[] = [
                'slug'     => Str::slug($review->titolo),
                'num'      => $key+1,
                'template' => 'review',
                'model'    => 'review',
                'content'  => [
                    'title'    => $review->titolo,
                    'headline' => $review->sottotitolo,
                    'byline'   => $review->codice,
                    'summary'  => $review->sedeCorso,
                    'date'     => $review->datainizio,
                    'link'     => $review->allegati,
                    'linkText' => $review->destinatari,
                    'cover'    => $review->urlImmagine
                ]
            ];
        }

        return Pages::factory($pages, $this);
    }

}

this solution seems to work but yours is much more compact. I will do some tests in the next few days to see if everything is ok.

Thanks a lot Rasteiner!

bnomei · March 17, 2023, 4:34pm

you might consider adding a cache to the virtual pages children. its not in the guide yet but thats how the core kirby logic works as well.

github.com/getkirby/getkirby.com

virtual pages guide: uuid and cache

opened 10:59AM - 21 Feb 23 UTC

bnomei

https://discord.com/channels/525634039965679616/1030095535236980746/103011678544…7522344 https://discord.com/channels/525634039965679616/1077497021902114847/1077525554678480917 ```diff class AnimalsPage extends Page { public function children() { + if ($this->children instanceof Pages) { + return $this->children; + } $csv = csv($this->root() . '/animals.csv', ';'); $children = array_map(function ($animal) { return [ 'slug' => Str::slug($animal['Scientific Name']), 'template' => 'animal', 'model' => 'animal', 'num' => 0, 'content' => [ + 'uuid' => Str::slug($animal['Scientific Name']), 'title' => $animal['Scientific Name'], 'commonName' => $animal['Common Name'], 'description' => $animal['Description'], ] ]; }, $csv); - return Pages::factory($children, $this); + $this->children = Pages::factory($children, $this); + return $this->children; } } ```

Topic		Replies	Views
Caching virtual pages based on content from an API Questions 🤔	6	905	July 25, 2019
Cache api response data for a virtual page programatically Kirby 3	4	314	May 26, 2023
Caching in Snippet Solved ✅	4	337	January 29, 2019
Caching API requests via template.json.php Questions 🤔	3	396	June 27, 2022
Rest API Cache in Kirby v3 Questions 🤔	6	758	August 5, 2020

Virtual Pages with API and Caching

Related topics