Import from CSV


#1

Hello,

I have a potential Kirby project. It will hold hundreds of users and upwards of 500 product pages. I need to build a feature that allows a content manager to import users via a CSV. They also want an import for products, but I’ll focus on users. I can imagine how to do this from a template, but I need this to be a feature available from the panel. Maybe its a widget on the dashboard with the ability to upload a CSV file? That seems to be the most logical solution. Despite reading the docs a few times, I have absolutely no clue on how to add functionality to the widget. It seems like you can only display data.

Is a widget the way to go or is there a better road to take (plugin, custom field)?


#2

if you need to do it just once to transition from your other database, you could just hardcode to load/read the file and separate each value and run a few lines of code to create users with all your files…

https://getkirby.com/docs/cheatsheet/users/create

http://www.w3schools.com/php/func_filesystem_fgetcsv.asp

really depends on your csv file, shouldn’t be too hard. I might hardcode one myself on my needs in the future but i have other priorities at the moment.


#3

I’d go for a plugin with a widget. I created a handler for .csv data a while ago, but then did not have the time to do the widget, although I had planned it. But it creates pages, structures and users from csv. If you don’t need it regularly, you can use it as is without the widget.

Feel free to use it as a basis for your own.


#4

The client will use the import feature say once a month so I do need some tidy solution that’s friendly for non-tech folks.

Thanks. I’ll look this over.


#6

Hello Texnixe, I would love to use your handler for a .csv file, but I’m more of a font-end developer. I want to take a CSV file and your script to populate a structure field. Can you offer a step by step? I don’t know where to start? So I have a .csv file. Now what? :blush:


#7

One way of using it is with two lines of code:

  1. create a new Object
$csv = new SonjaBroda\CsvHandler($filepath, $parseHeader, $delimer, $length);
// example using the example file in plugins/examples
$csv = new SonjaBroda\CsvHandler(kirby()->roots()->plugins() . '/kirby-csv-handler/examples/products.csv', true, true);

Set $parseHeader to true if the file contains a header, $delimiter default is a comma

  1. Create structure field
$csv->createStructure($uri, $fieldname, $append = true)
// example create a new structure field in the projects page with name of structure field products
$csv->createStructure('projects', 'products', false);

As an alternative, you can use the provides route, then you have to set the path to the file, the filename and the append option in your config.


#8

Thanks for the quick response.

This is Impressive, but I think I need to take a couple steps back regarding setup. Where do I put the class file you created (which is over there at GitHub)? Then where do I put these 2 lines of code? I’m sure there are multiple ways to go about it but just lookin’ for basics.

Your time is very much appreciated.


#9

For installation, follow the steps from the readme, under installation. Like any plugin, put it into site/plugins, if you downloaded the repo, rename the folder from “kirby-csv-handler-master” to “kirby-csv-handler” (so you end up with site/plugins/kirby-csv-handler. Afterwards, the class is available in any template/controller.

What you do next depends on what you want to do with the code. If you just want to do this one time, you can just put those two lines from above into any template, load the page, and the structure field will get created. If you want, use a fresh starterkit, install the plugin, and put the code from above as it is into the projects or home template for testing. Once done, check out the projects.txt file.

As I said above, you can also use the routes, then you don’t need to put any code anywhere, just put the settings into your config and call the route with the page uri.


#10

Oh. I do apologize. I somehow skipped the documentation you clearly included on GitHub. I’ll give this a go.


#11

Let me know if you got it to work. I think I need to update the readme a bit and add a function for easier access to the class.


#12

In a - may be - similiar project, I´ve done a one-time-import for users (about 300) and projects (about 100) with 2 simple shell-scripts, that read the data from a csv-file and wrote them to folders/text-files ready for Kirby. The site is up and running now, new users and projects are now added via the panel. If you´re interested, I can post an example for such a shell script.


#13

When I find the time, I’ll add the widget, so that it will be possible to add content from a csv file via the panel.


#14

Hi steenweg
I’m really intrested in your shell scripts, especially the one for creating projects. So I get a starting piont for an import script.
Many thanks in advanced.


#15

Hi Jonathan,

starting point was a CSV file (projects.csv) with the following structure (simplified, the actual file contained more fields)

001;2001;Elephants in Africa;http://www.elephants.com;elephants-africa
002;2003;Tigers in India;http://www.tigers.com;tigers-india
...

This is the script, to read the CSV

#!/bin/bash
OLDIFS=$IFS
IFS=";"
while read number year title website slug
do
  if [ ! -d "$number-$slug" ]; then
    mkdir ./$number-$slug
  fi
  echo -e "Year: $year\n----\nTitle: $title\n----\nWebsite: $website" > $number-$slug/project.txt
done < projects.csv
IFS=$OLDIFS

After running the script you get this folder structure

001-elephants-africa/project.txt
002-tigers-india/project.txt
...

to use in Kirby.

The fields “Year”, “Title” and “Website” were actual project fields. I added the fields “number” and “slug” to create unique folder names.

Hope, this helps!


#16

Thanks Thomas
That’s awesome. Works great.


#17

You´re welcome, always happy to help.


#18

Sonja, This plugin is excellent. It worked great. Thank you. This will save me tons of time.


#19

Glad to hear that it is useful :slight_smile:


#20

Hello sonja, is there anything new about your panel for creating pages by reading csv-file, which include HTML texts.
I am new to kirby. This feature interests me because my content is organized in html format in a csv-file. Thank you for a short feedback, if it is planed.


#21

@parviz I never created the widget, but you can either create pages from the CSV file programmatically in a template, or you can call one of the routes provided by the plugin.

An alternative would be the bash script mentioned above.

However, if you want to convert your HTML to Markdown, my plugin does not cover such conversions (and was never intended to).