Ubuntu reinstall apache

Apache has been corrupted somehow and now it just displays the following error when going to any page.

Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.

This is nothing to do with .htaccess, everything was working fine on multiple virtual hosts before my virtual machine screwed up some files.

I think the best idea is to reinstall the apache2 package.

I have tried apt-get --purge remove apache2 and apt-get --reinstall install apache2. Nothing seems to work, it just marks it as removed but doesn't actually remove the files.