I'm using Apache/2.4.16 on FreeBSD 10.2. I have two instances of Apache running, each serving several domain names. The server has two network interfaces, one for the public Internet, one only accessible through my internal LAN. All of the virtual host configuration files are nearly identical except for the domain names and directories. All of the sites work as expected, except for one: If I try to go to adultemployment.net, I get the server's default page, but if I specify any of the machine names configured in the DNS (www, ftp, mail, etc.) I get the site's page as expected, e.g., www.adultemployment.net displays correctly.
This is the virtual host configuration file for adultemployment.net:
<VirtualHost 100.0.193.102:80 192.168.14.202:80>
ServerName AdultEmployment.net
ServerAlias *.AdultEmployment.net
ServerAdmin webmaster@AdultEmployment.net
DocumentRoot "/path.to/adultemploy/www"
<Directory "/path.to/adultemploy/www">
Options All MultiViews
AllowOverride All
Require all granted
</Directory>
ErrorDocument 404 /cgibin/badurl.pl
ErrorLog "|/usr/local/sbin/rotatelogs /path.to/adultemploy/logs/error_log 86400"
CustomLog "|/usr/local/sbin/rotatelogs /path.to/adultemploy/logs/access_log 86400" combined
<IfModule mod_alias.c>
ScriptAlias /cgibin/ "/path.to/adultemploy/cgibin/"
<Directory "/path.to/adultemploy/cgibin">
AllowOverride None
Options None
Require all granted
</Directory>
</IfModule>
</VirtualHost>
This is the virtual host configuration file for another site (which works correctly), bidsadult.com:
<VirtualHost 100.0.193.102:80 192.168.14.202:80>
ServerName BidsAdult.com
ServerAlias *.BidsAdult.com
ServerAdmin webmaster@BidsAdult.com
DocumentRoot "/path.to/bidsadult/www"
<Directory "/path.to/bidsadult/www">
Options All MultiViews
AllowOverride All
Require all granted
</Directory>
ErrorDocument 404 /cgibin/badurl.pl
ErrorLog "|/usr/local/sbin/rotatelogs /path.to/bidsadult/logs/error_log 86400"
CustomLog "|/usr/local/sbin/rotatelogs /path.to/bidsadult/logs/access_log 86400" combined
<IfModule mod_alias.c>
ScriptAlias /cgibin/ "/path.to/bidsadult/cgibin/"
<Directory "/path.to/bidsadult/cgibin">
AllowOverride None
Options None
Require all granted
</Directory>
</IfModule>
</VirtualHost>
The default and virtual host configurations are set in httpd.conf:
# 'Main' server configuration
#
# The directives in this section provide defaults for any
# <VirtualHost> containers you may define later in the file.
#
# All of these directives may appear inside <VirtualHost> containers,
# in which case these default settings will be overridden for the
# virtual host being defined.
#
ServerAdmin webmaster@occupant591.net
ServerName generic.occupant591
#
<Directory />
AllowOverride none
Require all denied
</Directory>
#
<IfModule dir_module>
DirectoryIndex index.php index.cgi index.shtml index.htm index.html
</IfModule>
#
<Files ".ht*">
Require all denied
</Files>
#
LogLevel info
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
<IfModule ssl_module>
LogFormat "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b" ssl_request
</IfModule>
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
</IfModule>
<IfModule cgid_module>
Scriptsock logs/cgisock
</IfModule>
<IfModule mime_module>
TypesConfig conf/mime.types
AddType application/x-httpd-php .php
AddType application/x-tar .tgz
AddType text/plain .xml
AddType image/x-icon .ico
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddHandler cgi-script .cgi
AddHandler cgi-script .pl
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
</IfModule>
<FilesMatch "\.php$">
SetHandler application/x-httpd-php
</FilesMatch>
<FilesMatch "\.phps$">
SetHandler application/x-httpd-php-source
</FilesMatch>
<Location /cgi-bin/phf*>
Deny from all
ErrorDocument 403 http://phf.apache.org/phf_abuse_log.cgi
</Location>
<Location /cgibin/phf*>
Deny from all
ErrorDocument 403 http://phf.apache.org/phf_abuse_log.cgi
</Location>
# 'Main' server configuration
#
# The directives in this file set up the values used by the 'main'
# server, which responds to any requests that aren't handled by any
# other <VirtualHost> definition.
#
Include /path.to.apache.config/conf/generic.site
# Virtual hosts
Include /path.to.apache.config/conf/ActiveSites/*.conf
The default site configuration is set by generic.site:
<VirtualHost 100.0.193.102:80 192.168.14.202:80>
ServerAdmin webmaster@occupant591.net
DocumentRoot "/path.to/generic/www"
<Directory "/path.to/generic/www">
Options All MultiViews
AllowOverride All
Require all granted
</Directory>
ErrorDocument 404 /cgibin/badurl-occ.pl
ErrorLog "|/usr/local/sbin/rotatelogs /path.to/generic/logs-occ/error_log 86400"
CustomLog "|/usr/local/sbin/rotatelogs /path.to/generic/logs-occ/access_log 86400" combined
<IfModule mod_alias.c>
ScriptAlias /cgibin/ "/path.to/generic/cgibin/"
<Directory "/path.to/generic/cgibin">
AllowOverride None
Options None
Require all granted
</Directory>
</IfModule>
</VirtualHost>
I've tried to fix this by copying a functional configuration file onto the one for adultemployment.net and updating the domain name and directory entries. That didn't change anything. I've tried several different sets of ServerName and ServerAlias entries, such as
ServerName www.AdultEmployment.net
ServerAlias *.AdultEmployment.net AdultEmployment.net
and
ServerName www.AdultEmployment.net
ServerAlias *.AdultEmployment.net
I thought the problem might be because adultemployment.net was alphabetically first in the ActiveSites directory, so I added another domain to precede it – and that site works as expected, but adultemployment.net still doesn't return its page without a machine name.
At first I thought it might be a browser cache issue, but I get the same behavior accessing the site from Firefox and IE on my w2K machine, Firefox on my w7 laptop, and lynx ssh'd into an outside hosting company's shell. I also tried clearing the arp entry for the domain on the server, to no avail.
I've tried tweaking the DNS entries, and looked for symlinks as suggested here or .htaccess entries as suggested here for similar problems, but nothing seems to fix this problem where one site on the server fails, but all of the others work correctly.
I've been trying to fix this problem for over a week, intermittently when I first discovered it, and almost continuously for the past two days. Any hints or pointers would be greatly appreciated!!
Best Answer
I stumbled across a "solution" (a work-around, actually, the fundamental cause is still unknown):
For this configuration (the second Apache configuration loaded at boot time), I had to add a
ServerName
entry duplicating the information in httpd.conf to the generic.site configuration file:After adding the "default"
ServerName
entry, all of the sites served by this Apache configuration work correctly. If a similar entry is added to the other configuration (the first one loaded at boot time), the sites matching the "default"ServerName
entry fail.TL;DR;
If a visitor attempted to access a non-existent page on an unconfigured domain (i.e., one properly being served through the generic.site configuration) such as mature.bid/videos, a
[core:info]
entry was written into the error log with the messageFile does not exist: /web/generic-occ/www/videos
because the generic site doesn't have a file or directory namedvideos
and so Apache has nothing to serve to answer the request. While the error log entries illuminate the fact non-existent pages are being requested, they provide no domain information about the requests and are therefore really just noise.I was using the stock Apache "combined" log format to generate the access logs. Again, the access logs indicated the pages being accessed, but not the domain they were being requested from. For normal configurations where the logs are recording information for a single domain, that's not a problem, but the generic.site configuration is designed to answer for all domains that are not otherwise configured, so logging the domain information is required in this instance to be useful. Fortunately, Apache provides options for logging domain info, and
%v
for "the canonical ServerName of the server serving the request" seemed just what I needed. I created a new LogFormat statement:I restarted Apache, tried to load http://adultemployment.net again, and found the access log contained an entry including
AdultEmployment.net "GET / HTTP/1.1"
Great, I thought, that worked! - until I noticed the same text was logged when an unconfigured domain was accessed. Perhaps I should be using%V
for "the server name according to the UseCanonicalName setting to get the domain information? I changed my LogFormat command tothen restarted Apache - and found the domains with non-existent pages were being logged as I expected. Now I have a record of which of the unconfigured sites are being accessed, and what pages visitors expect them to have.
This left me wondering why the LogFormat
%v
string logged "AdultEmployment.net" as "the canonical ServerName of the server serving the request" - none of the configuration files are written such that AdultEmployment.net should be the default server name. (For the other Apache configuration on this machine, the%v
string logged is "Dreamer" - the O/S machine name for the server.) I can't find the page now, but during my earlier research I came across a situation where a similar problem was fixed by changing the domain name to not be the same as the operating system's server name. I tried ssh'ing in to the IP address for this Apache configuration to see if AdultEmployment.net was showing up as part of the operating system configuration - but it wasn't, as I expected.As an experiment I expected to cause problems, I tried adding a
ServerName
entry to the generic.site configuration file:I thought this might fix adultemployment.net (it did), but I expected it would also break occupant591.net, causing it to display the unconfigured site page. However, it did not break the "host" site - in fact, I haven't been able to find anything failed now.
Since
apache2ctl -S
(as mentioned here) isn't available with Apache 2.4 on FreeBSD, I read thehttpd
man page, and found-S
is an option I can pass directly to the daemon. With theServerName
entry in the generic.site configuration, thehttpd -S
report starts withWhen I remove the
ServerName
entry from the generic.site configuration, thehttpd -S
report starts withNote that without the
ServerName
entry, the default server is still supposedly being set atgeneric.site:6
even though there is no ServerName entry in the file, at line 6 or any other line in the file. Clearly something is broken but I don't have time to chase this ghost any more right now.When I was initially setting up this Apache configuration with two instances on the server, I had a
ServerName
entry in the other instance's configuration that included the full domain name. That caused the "host" site to break, and I fixed that problem by removing the top-level domain component from theServerName
entry in httpd.conf, and removing theServerName
entry altogether from the generic.site configuration file. I made the same changes to this configuration, which seemed to work until approximately when I reconfigured my network to get rid of my ISP's router. The next time I tried to access adultemployment.net, I found the problem described in my question.Now I'm puzzled by why the two instances of Apache behave differently - but I'm going to have to put off that investigation for when I have nothing to do and a staff to do it with...