Java – Login to site with HttpClient Post

authenticationcookieshttpclientjavapost

I am trying to make a program that logs into a site and performs some automated activities. I have been using HttpClient 4.0.1, and using this to get started: http://hc.apache.org/httpcomponents-client/primer.html.

On this particular site, the cookies are not set through a "set-cookie" header, but in javascript.

So far, I am unable to achieve the login.

I've tried the following things:

  1. add headers manually for all request headers that appear in firebug
   NameValuePair[] data = {
         new BasicNameValuePair("Host",host),
         new BasicNameValuePair("User-Agent"," Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7"),
         new BasicNameValuePair("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"),
         new BasicNameValuePair("Accept-Language","en-us,en;q=0.5"),
         new BasicNameValuePair("Accept-Encoding","gzip,deflate"),
         new BasicNameValuePair("Accept-Charset","ISO-8859-1,utf-8;q=0.7,*;q=0.7"),
         new BasicNameValuePair("Keep-Alive","300"),
         new BasicNameValuePair("Connection","keep-alive"),
         new BasicNameValuePair("Referer",referer),
         new BasicNameValuePair("Cookie",cookiestr)
   };

   for(NameValuePair pair : data){
      loginPost.addHeader(pair.getName(),pair.getValue());
   }

  1. creating BasicClientCookies and setting using setCookieStore. unfortunately, i can't figure out how to test if the cookies are actually being sent. also, is there a way to test what other automatic parameters are being sent? (like which browser is being emulated, etc).

The response I'm getting is: HTTP/1.1 417 Expectation Failed

I'm still new to this, so does anyone know off-hand what the problem could be? If not, I'll post more details, code, and the site.

Best Answer

You need WireShark or Fiddler. The first is a network analyser (so you'll see what's going on at a very low level); the second acts as a proxy - less transparent, but higher level.

That way you can look in detail at what happens when you log in with a browser, and what's happening when you try doing the same thing in code.