HTTP Server – How to Write an HTTP Server

httplanguage-agnosticserver

As the title says, I would like to write a HTTP server. My question is this, how do I do this? I know this sounds VERY general and too "high level", but there is a method to my madness. An answer to this question should be, I believe, language agnostic; meaning, no matter what language I use (e.g., C, C++, Java, etc.) the answer should be the same. I have a general idea of how this is supposed to work:

Open a socket on port 80.
Wait for a client to make a request.
Read the request (i.e., this person wants page "contact-us.html").
Find and read "contact-us.html".
Send an html header, then send the content of "contact-us.html"
Done

Like I said, I believe this is the process, but I am not 100% sure. This leads me to the heart of my question. How or where does a person find out this information?

What if I didn't want to write just an HTTP server, what if I wanted to write an FTP server, a chat server, an image viewer, etc.? How does a person find out the exact steps/process needed to create a working HTTP server?

A co-worker told me about the html header, so I would have NEVER know this without him. He also said something about handing off each request to a new thread. Is there some big book of how things work? Is there some manual of what it takes to be an HTTP server?

I tried googling "how does a HTTP server work", but the only answers I could find were geared towards your average Joe, and not towards a person wanting to program a HTTP server.

Best Answer

Use the RFC2616, Luke!

You read the RFC 2616 on HTTP/1.1, and you go for it.

That was actually a project in my 3rd year in engineering school, and that's pretty much the project description.

Tools

Your tools are:

basic networking stuff (socket management, binding, understand addresses),
good understanding of I/O streams,
a lot patience to get some shady parts of the RFC (mime-types are fun).

Fun Considerations

Things to consider for extra fun:

plug-in architecture to add CGI / mod support,
configuration files for, well, many things,
lots of experimentation on how to optimize transfers,
lots of experimentation to see how to manage load in terms of CPU and memory, and to pick a dispatch model (big fat even loop, single accept dispatch, multi-thread, multi-process, etc...).

Have fun. It's a very cool thing to look at.

Other (Simpler) Suggestions

FTP client/server (mostly RFC959 but there are older versions and also some extensions)
IRC client/server (mostly RFC1459, but there are extensions)

They're way easier to tackle first, and their RFCs are a lot easier to digest (well, the IRC one has some odd parts, but the FTP one is pretty clear).

Language Choice

Of course, some implementation details will be highly dependant on the language and stack you use to implement it. I approached all that in C, but I'm sure it can be fun just as well in other languages (ok, maybe not as much fun, but still fun).

Related Solutions

Why HTTP Doesn’t Have POST Redirect – Web Development Insights

In HTTP 1.1, there actually is a status code (307) which indicates that the request should be repeated using the same method and post data.

As others have said, there is a potential for misuse here which may be why many frameworks stick to 301 and 302 in their abstractions. However, with proper understanding and responsible usage, you should be able to accomplish what you're looking for.

Note that according to the W3.org spec, when the METHOD is not HEAD or GET, user agents should prompt the user before re-executing the request at the new location. You should also provide a note and a fallback mechanism for the user in case old user agents aren't sure what to do with a 307.

Using this form:

<form action="Test307.aspx" method="post">
    <input type="hidden" name="test" value="the test" />
    <input type="submit" value="test" />    
</form>

And having Test307.aspx simply return 307 with the Location:http://google.com, Chrome 13 and Fiddler confirm that "test=the test" is indeed posted to Google. Of course the further response is a 405 since Google doesn't allow the POST, but it shows the mechanics.

For more information see List of HTTP status codes and the W3.org spec.

307 Temporary Redirect (since HTTP/1.1) In this occasion, the request should be repeated with another URI, but future requests can still use the original URI.2 In contrast to 303, the request method should not be changed when reissuing the original request. For instance, a POST request must be repeated using another POST request.

Why doesn’t the HTML\DOM specification allow hyperlinks to set an accept header

Because the headers are not part of the URL. The URL should identify the resource, on it's own, and should do it uniquely. The Accept (and the more useful Accept-Encoding) headers should not affect semantics of the request. They should indicate capabilities of the client so the server can format the response accordingly.

If you do a HTTP call from JavaScript, your JavaScript code is the client and so it gets to set headers. But for HTML link the browser is the client and therefore it gets to choose.

If you want to provide CSV to a user, the linked resource should generally be in a fixed format. Imagine the user does

Copy Link Address

$ wget PasteEnter

That's a legitimate thing for them to do for various reasons. And because the link promised them CSV, that's what they'll expect. But the wget won't get any other attribute of the link. Only the URL. So the information should be encoded in the URL.

In fact there are not many uses for Accept header, because most resources just have their format and don't make much sense in another. The Accept-Encoding is useful; if the client says Accept-Encoding: gzip,deflate it means the client can do transparent decompression. But about the only use I could think of for Accept is image and video formats e.g. to provide alternate for browsers that don't support say image/svg+xml or video/webm. Except browsers don't seem to actually advertise which media formats they support, so it does not work. And HTML chose to implement different, slightly more flexible mechanism anyway. And few other kinds of data have several widely implemented alternate formats to choose from.