C# – Improving performance of multithreaded HttpWebRequests in .NET

chttpwebrequestmultithreadingnet

I am trying to measure the throughput of a webservice.

In order to do that, I have written a small tool that continuously sends requests and reads responses from a number of threads.

The contents of the inner loop of each thread looks like this:

public void PerformRequest()
{
  WebRequest webRequest = WebRequest.Create(_uri);

  webRequest.ContentType = "application/ocsp-request";
  webRequest.Method = "POST";
  webRequest.Credentials = _credentials;
  webRequest.ContentLength = _request.Length;
  ((HttpWebRequest)webRequest).KeepAlive = false;

  using (Stream st = webRequest.GetRequestStream())
    st.Write(_request, 0, _request.Length);

  using (HttpWebResponse httpWebResponse = (HttpWebResponse)webRequest.GetResponse())
  using (Stream responseStream = httpWebResponse.GetResponseStream())
  using (BufferedStream bufferedStream = new BufferedStream(responseStream))
  using (BinaryReader reader = new BinaryReader(bufferedStream))
  {
    if (httpWebResponse.StatusCode != HttpStatusCode.OK)
      throw new WebException("Got response status code: " + httpWebResponse.StatusCode);

    byte[] response = reader.ReadBytes((int)httpWebResponse.ContentLength);
    httpWebResponse.Close();
  }      
}

It seems to work okay, except that something seems to be limiting the tool. If I run two instances of the tool with each 40 threads, I get significantly more throughput than one instance with 80 threads.

I found the ServicePointManager.DefaultConnectionLimit property, which I set to 10000 (and it makes no difference if I set it through app.config as suggested by Jader Dias).

Are there any other settings in .NET or on my machine that can influence the performance? (I am running Vista, but I see the same problem on Windows Server 2003).

Perhaps some restrictions on how many connections a single process can make?

Best Answer

You must set the maxconnection parameter at the app.config or web.config file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.net>
    <connectionManagement>
      <add address="*" maxconnection="80"/>
    </connectionManagement>
  </system.net>
</configuration>

Values up to 100 work very well with Windows XP.

Update: I just found out that the method above is an alternative way to set the System.Net.ServicePointManager.DefaultConnectionLimit

Related Solutions

How should I unit test multithreaded code

Look, there's no easy way to do this. I'm working on a project that is inherently multithreaded. Events come in from the operating system and I have to process them concurrently.

The simplest way to deal with testing complex, multithreaded application code is this: If it's too complex to test, you're doing it wrong. If you have a single instance that has multiple threads acting upon it, and you can't test situations where these threads step all over each other, then your design needs to be redone. It's both as simple and as complex as this.

There are many ways to program for multithreading that avoids threads running through instances at the same time. The simplest is to make all your objects immutable. Of course, that's not usually possible. So you have to identify those places in your design where threads interact with the same instance and reduce the number of those places. By doing this, you isolate a few classes where multithreading actually occurs, reducing the overall complexity of testing your system.

But you have to realize that even by doing this, you still can't test every situation where two threads step on each other. To do that, you'd have to run two threads concurrently in the same test, then control exactly what lines they are executing at any given moment. The best you can do is simulate this situation. But this might require you to code specifically for testing, and that's at best a half step towards a true solution.

Probably the best way to test code for threading issues is through static analysis of the code. If your threaded code doesn't follow a finite set of thread safe patterns, then you might have a problem. I believe Code Analysis in VS does contain some knowledge of threading, but probably not much.

Look, as things stand currently (and probably will stand for a good time to come), the best way to test multithreaded apps is to reduce the complexity of threaded code as much as possible. Minimize areas where threads interact, test as best as possible, and use code analysis to identify danger areas.

C# – Sending email in .NET through Gmail

Be sure to use System.Net.Mail, not the deprecated System.Web.Mail. Doing SSL with System.Web.Mail is a gross mess of hacky extensions.

using System.Net;
using System.Net.Mail;

var fromAddress = new MailAddress("from@gmail.com", "From Name");
var toAddress = new MailAddress("to@example.com", "To Name");
const string fromPassword = "fromPassword";
const string subject = "Subject";
const string body = "Body";

var smtp = new SmtpClient
{
    Host = "smtp.gmail.com",
    Port = 587,
    EnableSsl = true,
    DeliveryMethod = SmtpDeliveryMethod.Network,
    UseDefaultCredentials = false,
    Credentials = new NetworkCredential(fromAddress.Address, fromPassword)
};
using (var message = new MailMessage(fromAddress, toAddress)
{
    Subject = subject,
    Body = body
})
{
    smtp.Send(message);
}

Additionally go to the Google Account > Security page and look at the Signing in to Google > 2-Step Verification setting.

If it is enabled, then you have to generate a password allowing .NET to bypass the 2-Step Verification. To do this, click on Signing in to Google > App passwords, select app = Mail, and device = Windows Computer, and finally generate the password. Use the generated password in the fromPassword constant instead of your standard Gmail password.
If it is disabled, then you have to turn on Less secure app access, which is not recommended! So better enable the 2-Step verification.

Best Answer

Related Solutions

How should I unit test multithreaded code

C# – Sending email in .NET through Gmail

Related Topic