Having different robots.txt info based on subdomain

Are any other routes or templates also affected? I’ve seen this before, but not in routes.

I always use this:

But I dont think it works because of a possible route collision somewhere.

Thanks @jenstornell but i actually used that first, and got the same result. It’s actually why i tried this method :slight_smile:

1 Like

Nope… everything else is working fine. The route for this is the last one declared. There are others before it.

If you set $html to a simple string like $html = 'this is a test';, does it work?

Nope… I get:

<this is a test

:frowning:

@lukasbestle I think ive figured it out. The staging server (which is a subdomain) forces www and so does the live server. If i disable the the www forcing, the robots files comes out properly.

So the question issss… how can I leave the .htaccess rule in to force the www AND get a weirdness free robots file.

to be clear…

This works:

http://sub.mydomain.com/robots.txt

This gets the < on the first line:

http://www.sub.mydomain.com/robots.txt

Which i dont think is strictly legal, but the site does work. As i understand it, www is a subdomain on its own, so i think technically the above is a subdomain of a subdomain.

Why do you force www in the .htaccess not in the httpdconf file? Or redirect www to non-www in DNS settings?

The answer there is it wasnt me :slight_smile: This is a project I have inherited, I didnt set it up or configure the server. It is a VPS though and i have access. Is that a better way?

That’s always the better way. And in this case, you don’t run into strange side effects like forcing www on a subdomain.

What’s interesting though is that forcing www has an effect on the output of your route (and only that route). That’s very strange.

Sure is. If i get a minute i might try and replicate it on my own server which happens to be basically identical to my clients, but with a different webhost entirely (VPS, CentOs 7, PHP 7.2, Apache, WHM/cPanel).

Ill look into setting it via httpdconf, it’s not something i can just remove because the sites been live for a couple of years and ill i have to clean up Googles index if i do (I think).

Have you tried what I wrote above (replacing the call to f::read() with a simple string)? The result could help with debugging.

@lukasbestle I did, yes…I got the same result.

Ah. Sorry, I missed that post. Becomes stranger and stranger to be honest. :frowning:

Just to narrow it down. Do you use a cache or a minifier plugin on top? I know my html minifier plugin has caused similar issues in the past.

:open_mouth::open_mouth::open_mouth: It was the @jenstornell minifier plugin! Switched it off in the config, and now issue is gone, even with forcing the www.

So we forgot to do the most important thing: Check if it happens in a fresh Starterkit as well :roll_eyes:

Yes lol… the amount of times I have told other people to try that. I have had no trouble from the plugin elsewhere though, so didn’t occur to me.

So it was my fault after all… crap! :frowning:

I wonder what the best way is to prevent issues like this in my plugin. It does not know it’s a robots file right now. Somehow it needs to know that it should be skipped from minification.

Any ideas @texnixe @jimbobrjames @lukasbestle ?