Create pages from .csv

Kirby 3 Solved ✅
1

Hi
I’ve created content from .CSV file with bunch of entries. It worked as expected. Now I would like to turn these virtual pages to real content pages, editable from the panel.

I could read from the merging content sources article :

We will extend the Content from a spreadsheet example with content we actually add in the content folder, let’s say because we want to add an image to each each animal and some additional content.

Cool, this is exactly my purpose !

To this end, we add new subpages to the /animals parent page, using the same slugs that we generated in the animals.php model.

Ouch ! I have more than 500 entries…Is there a method to create these folders (with .txt inside) automatically from these virtual pages?

This is my model file, very similar to the example from the doc :

<?php

class CollectionPage extends Page
{

    public function children()
    {
        $csv      = csv($this->root() . '/collection.csv', ';');
        $children = array_map(function ($oeuvre) {
            return [
                'slug'     => Str::slug($oeuvre['auteur']),
                'template' => 'oeuvre',
                'model'    => 'oeuvre',
                'num'      => 0,
                'content'  => [
                    'auteur'     => $oeuvre['auteur'],
                    'titre'  => $oeuvre['titre'],
                    'date' => $oeuvre['date'],
                    'medium' => $oeuvre['medium'],
                    'edition' => $oeuvre['edition'],
                    'format' => $oeuvre['format'],
                    'galerie' => $oeuvre['galerie'],
                    'certificat' => $oeuvre['certificat'],
                    'prix' => $oeuvre['prix'],
                    'stockage' => $oeuvre['stockage'],
                    'exposition' => $oeuvre['exposition'],
                ]
            ];
        }, $csv);

        return Pages::factory($children, $this);
    }

}
2

But wouldn’t it make more sense to create pages from that csv file programmatically rather than mixing content? Or do you really want to keep and update the csv file?

3

Yes my goal is to create pages from that csv, I was just trying to find some information here and there for what I’m looking for.
I would like to use this CSV file to save time on the creation of the files and their contents. Afterwards, I would like to be able to edit each page to add images, add or modify content.

4

Then the virtual pages don’t make sense. Use the csv() helper function to read your csv and then use $pages->create()to actually create the pages. You can reuse part of the code from the virtual pages docs…

5

Okay. As “virtual pages” is a new Kirby feature, I am a bit confused.
Is there any example code I can find about reusing the model code from the doc to create pages?
I don’t know if I have to make a plugin file? A model? And how to reuse this code to properly add pages?

6

I don’t have time now, but can provide an example sometime tonight.

7

Assuming you do this in the template of the parent page, this (using $page instead of $this) gives you an array of the items.

This example uses the animals file of the guide example, but should be no problem to adapt to your use case.

So assuming you do this in the parent page (animals in the guide example, or oevres in yours):

// the `csv` function can live in a plugin file or in the template where you want to create the children if its only a one time thing
function csv(string $file, string $delimiter = ','): array
{
    $lines = file($file);

    $lines[0] = str_replace("\xEF\xBB\xBF", '', $lines[0]);

    $csv = array_map(function($d) use($delimiter) {
        return str_getcsv($d, $delimiter);
    }, $lines);

    array_walk($csv, function(&$a) use ($csv) {
       $a = array_combine($csv[0], $a);
    });

    array_shift($csv);

    return $csv;
}

// Adapt this to your csv file name and the fields in your csv file:
// Example based on the guide example
    $csv      = csv($page->root() . '/animals.csv', ';');
    $kirby->impersonate('kirby');
    foreach ($csv as $animal) {
      $newPage = page('animals')->createChild([
        'slug' => Str::slug($animal['Scientific Name']),
        'template' => 'animal',
        'model'    => 'animal',
        'content'  => [
          'title'       => $animal['Scientific Name'],
          'commonName'  => $animal['Common Name'],
          'description' => $animal['Description'],
      ]

      ]);
    };
1 Like
8

Thank you very much ! I think this code could be very useful in the documentation. I’ve just set ->changeStatus() to make them listed

$csv      = csv($page->root() . '/collection.csv', ';');
    $kirby->impersonate('kirby');
    $i = 0; 
    foreach ($csv as $oeuvre) {
      $newPage = page('collection')->createChild([
        'slug' => Str::slug($oeuvre['titre'].'-'. $oeuvre['auteur']),
        'template' => 'oeuvre',
        'model'    => 'oeuvre',
        'draft' => 0,
          'num' => 0,
        'content'  => [
          'auteur'     => $oeuvre['auteur'],
                    'titre'  => $oeuvre['titre'],
                    'date' => $oeuvre['date'],
                    'medium' => $oeuvre['medium'],
                    'edition' => $oeuvre['edition'],
                    'format' => $oeuvre['format'],
                    'galerie' => $oeuvre['galerie'],
                    'certificat' => $oeuvre['certificat'],
                    'prix' => $oeuvre['prix'],
                    'stockage' => $oeuvre['stockage'],
                    'exposition' => $oeuvre['exposition'],
      ]

      ])->changeStatus("listed", $i++);
    };
9

Hi,
Is there is a way to deal with line break into the csv file?
I have line break into the datas of the csv:

Sentence#1 lorem ipsum

New sentence#2 lorem ipsum

Sorry to ask this question here as it is more php related but I didnt find a solution on others forums.
Thanks

10

@Hans Sorry, I’m afraid I don’t really understand your question? Where are the line breaks? What do you want to happen to these line breaks?

11

Sorry, the line breaks in the csv datas of my files, are generating an error with the code used by texnixe and Oziris.
the console says: array_combine(): Both parameters should have an equal number of elements
I’ve deleted the line breaks and the it works just fine.

But I need to keep those line breaks from the datas into the txt file generated in the kirby folder.

12

Have you passed the correct delimiter?

13

Yes the delimiter for the csv is a comma.
The line breaks are not used to separates the values.

In the datas of my csv file, there are paragraphs with line breaks, .
I would like to keep those line breaks into the text file generated via the script provided by texnixe.

14

Could you post a sample of your .csv file?

15

Here is a sample:

id,title,content
11258,Title of the item,"Sed ut perspiciatis unde omnis iste natus error sit voluptatem
accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis.

Quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam."

Thank you

16

This will not work, because CSV files are read line by line. You would have to replace the hard breaks with a new line character (\n).

17

Yes I tried this but still doesn’t work.

function csv(string $file, string $delimiter = ','): array
{
    $lines = file($file);

    $lines[0] = str_replace("\xEF\xBB\xBF", '', $lines[0]);
    $lines[0] = str_replace("\r\n", '<br />', $lines[0]);


    $csv = array_map(function($d) use($delimiter) {
        return str_getcsv($d, $delimiter);
    }, $lines);

    array_walk($csv, function(&$a) use ($csv) {
       $a = array_combine($csv[0], $a);
    });

    array_shift($csv);

    return $csv;
}
18

This will replace a carriage return (\r) followed by a line feed (\n) with <br /> and only in the first line of the file ($lines[0]). You need to replace line feed and possibly carriage return in your whole file.

To make the replace work regardless if your file contains line feed only (typically on UNIX-like OS) or the combination of carriage return and line feed (typically on Windows) see the following example from the PHP documentation:

// Order of replacement
$str     = "Line 1\nLine 2\rLine 3\r\nLine 4\n";
$order   = array("\r\n", "\n", "\r");
$replace = '<br />';

// Processes \r\n's first so they aren't converted twice.
$newstr = str_replace($order, $replace, $str);
19

For a possible solution see: Read multi-line csv

20

Thanks Texnixe, can you explain me how to call this method on the create pages code above?