Apache2 disable catch all virtual host

It doesn't matter what address you use. If the IP address is the same, you could use site3.com and still access site1. The first VirtualHost will be served for all hostnames that don't match any ServerName or ServerAlias. You need to symlink the site2.com.conf file (or use a2ensite) to be able to access it.

Your WSGIScriptAlias directive is saying to send everything except for explicit Alias matched static files, through to Django. Thus, if you are trying to access index.html explicitly it will not use the static one and if Django doesn't know what to do with it, it will return a 404. What is the actual URL you are using? For that URL, do you have matching configuration in Django urls.py for it so Django knows what to do?

Alternatively, issue may be because you don't actually have a /srv/site2/wsgi/django.wsgi file, or because you have not set up an Allow directive for /srv/site2/wsgi like Django/mod_wsgi documentation says you should although lack of that would normally result in 403.

Suggest you comment out WSGIScriptAlias and see what happens first. You would then need to properly explain how you want things to work with actual examples. Reading Django integration guide on mod_wsgi site may also be a good idea.