Testing Email Forms with Kirby

The CMS Questions
1

Hey everyone,

I’ve set up a rather lengthy application form for a website and can’t figure out a proper testing environment for it now that it has gone live.
I never managed to setup a test environment where I could send emails through the localhost and ended up uploading changes directly to the webserver. Since none of the involved scripts were in use, this was never a problem.

Now that the form is live, I’m in a bit of a bind - how do I make adjustments without potentially inflicting damage to users? I can’t test the way I did before - the application form is being used almost on a daily basis and I don’t want to impair running applications (it’s a lot to fill out, after all).

I figured, either I duplicate all the involved files, re-name them, move them somewhere else, test on the server and once I’m done copy all the code back to the original files… which already sounds like a nightmarish hassle just typing it out.

Or, I finally do manage to figure out a way to send emails to a private address from the localhost - which I have spent 2 hours on this morning and couldn’t get to work, because it looks like emails are more complicated than I thought.

I’m probably not the first one with this problem, but couldn’t find much on specifics online. What does your test environment look like for these cases? Do you have any recommendations? Are there alternative options? I’d appreciate any pointers!

Thanks!

2

What is your local dev environment?

Testing with private email from localhost can be difficult, as mails often don’t get through. Consider using a mail provider for testing.

If you can’t get it working on local, you can set up a copy of your project on a staging server and do your tests there.

1 Like
3

Thanks for the swift response!
My default local dev environment is running on macOS with AMPPS for the localhost, running PHP 7.1
I reset the email configurations I experimented with, except that Mailhog is still installed, when I tried to get it working with MAMP Pro, as described in this guide.

I feel like the localhost should be my staging server, but at this point, setting up a copy sounds like the faster solution. I will give it a shot! Thanks for the input.

4

I don’t have a working mail solution since I switched from Mamp to Valet. If you deploy with Git or @jimbobrjames’ favorite rsync, that’s a lot faster than using SFTP.

A real staging server with the same configuration as your target host is a more reliable testing environment than your local setup, anyway.

5

rSync is extremely fast, after the first transfer, it will only push the files that have changed going forward. I normally set up a dev.yourdomain.com subdomain and work off that as a staging server until the site is ready to go live.

As for the emails, should be easier enough to use a real SMTP for that, or alternately, you can use the webhook part of Uniform to send the email via something like Postmark. I actually did that it on a recent project, so I can help you there of you go that way.

6

I knew you’d come out of your hole…:upside_down_face:

You should really create a text boilerplate for rsync…

7

Its a nice hole :slight_smile:

Yes, ill get “Just rSync It” printed on a tshirt if you get one that says “Check it exists first!”…

@Rostiger Jokes aside, this is rysync command i use to push up to a server (you can pull the other way by reversing the two paths at the end of the command).

rsync -azLK --force --delete --progress --exclude-from=rsync_exclude.txt -e "ssh -p22" ./public/ user@12.34.5.67:/home/yoursite/subdomains/dev.domain.com

You can list files out in rsync_exclude.txt that you want it to skip over in the same way you do with .gitignore, an item per line. You can do wildcards and patterns too.

Please please make sure your project is under source control - rSync is brutal and will delete stuff if your not careful. Try it out on some test files until you are confident about it.

There is an excellent rSync crash course over here.

Pro tip: if you throw that command in the scripts section of your package.json file, then you can deploy just by typing yarn deploy instead of having to remember the whole thing.

8

rsync sounds cool once you have set it up, but figuring out how to get an SSH public key onto the server at which port is currently outside of my scope.
I might go down that rabbit hole another time - for now, moving a few files around manually will have to do.
Thanks for the input!

9

Is the sever Plesk or cPanel powered? This is usually pretty trivial in their, just a couple of clicks. From there you can add it to your keychain locally if your on a mac (see the first answer on that page).

10

I always test emails with MailDev. Just a quick npm install -g maildev and you have an SMTP server that won’t actually send your emails anywhere. Instead it’ll collect all of the emails that your application tries to send and give you an inbox to look at them.

With Kirby I use localhost config to set MailDev as the SMTP server for my dev environment and the production config file to set the production SMTP server.

I posted a bit more info about how my dev environment and deployment fit together in this thread.

2 Likes
11

Hey Jayshua,

that looks very promising, thank you for the answer!
Once I have a little more time on my hands, I’ll try to mimic your setup and report back.
Thanks again!

12

So I have tried to re-create your setup and since I promised to report, here’s my experience:

I created a Docker account, downloaded and installed the client, installed MailDev, browser-sync and added the docker-compose.yml file to the project root folder.
I was worried that Docker would force me to be online all the time, but once the docker-compose file was executed, the necessary containers seemed to be stored locally - yay!

So far so good!

However, when I ran browser-sync command I got an index list of the ‘site’ subfolder of the project root.

I’d suspected it had to do with this part of the docker-compose file:

volumes:
 - ./site:/app

I changed it to

volumes:
 - .:/app

and to my genuine surprise, it worked!

As a small side-note: when running browser-sync the command can be shortened to a more convenient

browser-sync start --proxy localhost:8080

The setup works as far as testing the site locally. However, I still can’t quite figure out how to proceed with the email setup.

I’m currently not using a specific SMTP setup that I consciously configured. The form is an adaption of the Email contact form tutorial. That means, all I ever did in terms of email configuration was to add the specific address (in my case a Google Groups Address) to the controller that handles the data and verification checks.

This seemed sufficient and works in most cases, except for some occurrences where people keep receiving “Undelivered Mail Returned to Sender” emails and I have absolutely no clue why or how this happens.

I figured Kirby’s PHPMailer would take care of everything, but maybe I do need more sophisticated configuration?

For the testing setup locally I added this part to my config.localhost.php file:

‘email’ => [
‘transport’ => [
‘type’ => ‘smtp’,
‘host’ => ‘localhost’,
‘port’ => 1025,
‘security’ => false
]
],

However, now I’m receiving an ‘unexpected end of file’ error in the email body template file. It works with the live version, but not with this setup it seems. I’ll still have to bang my head against this issue, but I’d be happy for any pointers and hunches.

13

If you don’t have any SMTP configuration in your config file, it will use the default PHP mail config. This is usually fine since most webhosts will have something setup for that automatically. There is a distinction between the PHPMailer config, and the PHP config. The PHP config will be mostly controlled by your webhost and they will setup the mailing system if they choose to provide that service. (Not all hosts will, but most will.)

This sounds like you have an unclosed bracket or missing comma somewhere. If you are deploying the exact same files to production, the missing piece is probably in your config.localhost.php file since that wouldn’t be loaded in production.

Haven’t checked, but I’m pretty sure that with this command you won’t get automatic reloading when you change a file, or style injection - which is the only reason I personally use browser-sync. The php-apache-dev docker container comes with its own server that works fine if you aren’t interested in the automatic reloading or style injection.

14

I just realized that I forgot about a key detail… It’s actually kinda important :confused:

Maildev doesn’t support TLS encryption. PHPMailer enables TLS by default, and Kirby doesn’t expose the configuration option to disable it. I had intended to open an issue or pull request back when I worked the problem out last year, but I guess I forgot. There’s two ways to fix it: The easy way, and the slightly-more-correct-but-still-modifying-kirby-files-so-pretty-bad way.

Easy way

(please do not deploy this change. It could accidentally disable TLS if you are using an SMTP server): Change line 286 in kirby\vendor\phpmailer\phpmailer\src\PHPMailer.php from

    public $SMTPAutoTLS = true;

to

    public $SMTPAutoTLS = false;

slightly-more-correct-but-still-modifying-kirby-files-so-pretty-bad way

Add this line to kirby\src\Email\PHPMailer.php just after line 67:

            $mailer->SMTPAutoTLS = $this->transport()['tls'] ?? true;

Then add this line to your config.localhost.php in the transport section underneath the security line:

			'tls' => false,

I posted this in the feature repo: https://github.com/getkirby/ideas/issues/430

15

If you use Mailhog instead of Maildev, you don’t have to fiddle with the source code.

16

I checked out Mailhog but couldn’t figure out how to embed it into my current setup.
All this configuration stuff is pretty new and heavy for me, so I decided to go with the quick and dirty way and modified the Kirby source code to disable TLS.

Unfortunately I’m still not receiving emails on the Maildev web UI and I’m a bit stumped on why.
I double checked all the configurations and the setup and couldn’t find any deviations from the instructions.

I was able to fix the unexpected end of file error (which seems to have been ignored by the server in the current live version). Now, when I send the form data locally, I get the expected success message, but nothing shows up in the web UI of Maildev, even though it tells me it can receive mails on port 25 now (which is what I changed the port configuration to in config.localhost.php). Here’s what that currently looks like:

      'email' => [
        'transport' => [
          'type' => 'smtp',
          'host' => 'localhost',
          'port' => 25,
          'security' => false,
          'tls' => false
          ]
      ],

The compose file still looks like this:

version: "3"

services:
  server:
    image: webdevops/php-apache-dev:7.2
    volumes:
      - .:/app
    ports:
      - "8080:80"

  maildev:
    image: djfarrelly/maildev
    ports:
      - "1080:80"
      - "1025:25"

Does it make a difference if I load the website through localhost:3000 or localhost:8080?
Both work, but they don’t seem to make a difference in sending the form.
Is it important what email address is defined to field of the $kirby->email() function?
My guess is no, as all emails sent through the localhost should end up in the maildev inbox, right?

Thank you for all your help so far. I feel like I’m pretty close, but I must be missing a detail somewhere.

17

Bumping this again, because I still have no working solution. I banged my head against this for days and I just can’t get it to work.

I discarded all the source code changes and checked out MailHog instead. I got it to run, but wasn’t able to configure it properly so Kirby could connect to the SMTP. I read all the docs and it looks like it should just work. But it doesn’t and I’m running out of ideas.

18

Does Mailhog actually start without error messages?

19

Yes, Mailhog is up an running:

56
56870×900 19.5 KB
08
081602×912 42.4 KB

Kirby is configured to send mails to the localhost via port 25. It seems to connect successfully to the local SMTP, at least there is no error thrown when submitting the form. It shows the successfully sent message, but alas, the inbox stays empty.

20

Hm, that looks good. And your email settings in config are the same as you posted above?