Nginx reverse proxy address/port already in use

I have an nginx reverse proxy, and I would like to have it forward traffic on to several sites, and listen on port 443 for all of these services. I've seen this done several places, and seems like the normal way to have a reverse proxy setup…it listens on a single port, and forwards based on URL.

However, I have some settings that I consider to be a little weird because I have a site that uses NTLM validation through an IIS site, and because of that I am using nginx Stream

Here's my current config files:

/opt/nginx/nginx.conf

stream {
    upstream backend {
        hash $remote_addr consistent;
        server mysite.domain.com:80 weight=5;
        server 192.168.0.5 max_fails=3 fail_timeout=30s;
    }

    server {
        listen 192.168.0.2:443 ssl;
        ssl_certificate /usr/local/nginx/conf/mycert.crt;
        ssl_certificate_key /usr/local/nginx/conf/mykey.key;
        ssl_session_cache shared:SSL:10m;

        ssl_session_timeout 5m;

        proxy_connect_timeout 1s;
        proxy_timeout 3s;
        proxy_pass backend;
    }
}

http {
    include mime.types;
    default_type application/octet-stream;

    sendfile on;
    keepalive_timeout 65;

    server {
        listen       80;
        server_name  localhost;

        location / {
            root   html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }
    include /opt/nginx/sites-enabled/*;
}

/opt/nginx/sites-available/default

server {
        listen 80 default;
        server_name _;
        return 301 https://$host$request_uri;
}

server {

        listen 192.168.0.2:443 ssl;
        server_name myothersite.domain.com;

        ssl_certificate /usr/local/nginx/conf/mycert.crt;
        ssl_certificate_key /usr/local/nginx/conf/mykeykey;

        ssl_session_timeout 5m;

        ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers "HIGH:!aNULL:!MD5 or HIGH:!aNULL:!MD5:!3DES";
        ssl_prefer_server_ciphers on;

        location / {
                proxy_pass http://192.168.0.6:80;
                proxy_set_header Host $host;

                proxy_redirect http:// $scheme://;

        }
}

When I go to restart nginx, I get the error

bind() to 127.0.0.1:443 failed (98: Address already in use)

I thought that under each server you could have it listen on the same port, but this seems to not be working.

If I remove the stream all together, and just use two other sites that don't use stream, this seems to work fine with multiple server sections.