Apache: How to restrict access to a directory while the whole server has lower restrictions

One of our publicly reachable web servers is restricting access to all of itself using a <Location /> block.

I have a particular directory that needs stricter access restrictions than provided by <Location />.

However, using <Directory /path/to/dir> is not possible, as <Directory> is overwritten by <Location>.

What can I do to restrict access to a specific directory while maintaining a less strict access restriction on the whole web server?

_{It is not possible to change <Location /> to <Directory /path/to/docroot> as this Apache is acting as a reverse proxy for a lot of other web servers on internal servers that cannot be directly reached from outside the network.}

By request, a little detail about our configuration:

All of our authorization and authentication is done with LDAP.

Since this Apache is acting as a reverse proxy for a lot of internal web servers we need to restrict access to all of it:

<Location />
     Order deny,allow
     Deny from all
     AuthName [...]
     AuthType Basic
     AuthLDAPURL [...]
     AuthBasicProvider ldap
     Require ldap-attribute [...]
</Location>

A particular directory under the document root needs stricter access permissions. Therefore I thought of this:

<Directory /path/to/dir>
    Order allow,deny
    AuthName [...]
    AuthType Basic
    AuthLDAPURL [...]
    AuthBasicProvider ldap
    Require ldap-attribute [...]
    Require ldap-attribute [...]
    Require ldap-attribute [...]
</Directory>

However, this doesn't work as <Location> sections are merged after <Directory> sections (see here).

So even though I have written a <Directory> section for the directory that I want to have stricter access permissions for the directives are overwritten by the ones in <Location />, allowing access to more users than desired.