I have created a robots.txt
and sitemap.xml
files using Magento UI but when trying to access www.mydomain.com/robots.txt or www.mydomain.com/sitemap.xml it gives me 404 not found
.
I found that robots.txt
and sitemap.xml
file are created in mydomain/http/
, however, the pub
directory is set to be my root so moving the file to the pub
fixes the issue.
Why Magento created the files in this location?
If I want to make changes to the robots.txt
file, I can't use the native Magento feature?
What is the best solution to this?
I am using NginX and added this to my configuration file which "solves" the issue, however I am not sure if this is the best solution:
# robots
location = /robots.txt {
alias /var/www/example.com/html/robots.txt;
}
# sitemap
location = /sitemap.xml {
alias /var/www/example.com/html/sitemap.xml;
}
I am using Magento 2.1.8.
Best Answer
This seems to be fixed in Magento 2.2.1, see https://github.com/magento/magento2/pull/9094.