Nginx proxy_hide_header still caches header

I'm using nginx with proxy_cache to cache some pages, I have set up a separate refresh location block to use for refreshing a particular cache entry. I have tested this and it works fine for refreshing.

However it also caches the set-cookie header (the framework I use, Symfony, always creates a new session cookie for new users even if they are not logged in). I searched around a bit and apparently proxy_hide_header should help me here but it only hides the header for the final response that is returned to the client, the header is still cached. I also tried this for other headers and had the same result. I also tried using the headers_more_module (more_clear_headers), but that didn't help either. Does anyone know what is the problem here, or has an alternative to what I'm trying to do? I'm using nginx 1.11.2 and this is the config:

location ~ /refresh(/.*) {
    allow 127.0.0.1;
    deny all;

    rewrite ^/refresh?(/.*)$ $1 break;

    proxy_hide_header "Set-Cookie";
    proxy_ignore_headers "Set-Cookie" "Vary";

    proxy_cache FOS_CACHE;
    proxy_pass http://localhost:8080;
    proxy_set_header Host $host;
    proxy_cache_key $uri$is_args$args;
    proxy_cache_valid 200 1m;

    proxy_cache_bypass 1;

    access_log /var/log/nginx/cache_refresh.log cache_info;
}

Thanks