I'm writing a Server/Client application where clients will connect to the server. What I want to do, is make sure that the client connecting to the server is actually using my protocol and I can "trust" the data being sent from the client to the server.
What I thought about doing is creating a sort of hash on the client's machine that follows a particular algorithm. What I did in a previous version was took their IP address, the client version, and a few other attributes of the client and sent it as a calculated hash to the server, who then took their IP, and the version of the protocol the client claimed to be using, and calculated that number to see if they matched. This works ok until you get clients that connect from within a router environment where their internal IP is different from their external IP. My fix for this was to pass the client's internal IP used to calculate this hash with the authentication protocol. My fear is this approach is not secure enough. Since I'm passing the data used to create the "auth hash".
Here's an example of what I'm talking about:
Client IP: 192.168.1.10, Version: 2.4.5.2
hash = 2*4*5*1 * (1+9+2) * (1+6+8) * (1) * (1+0)
Client Connects to Server
client sends: auth hash ip version
Server calculates that info, and accepts or denies the hash.
Before I go and come up with another algorithm to prove a client can provide data a server (or use this existing algorithm), I was wondering if there are any existing, proven, and secure systems out there for generating a hash that both sides can generate with general knowledge. The server won't know about the client until the very first connection is established.
The protocol's intent is to manage a network of clients who will be contributing data to the server periodically.
New clients will be added simply by connecting the client to the server and "registering" with the server. So a client connects to the server for the first time, and registers their info (mac address or some other kind of unique computer identifier), then when they connect again, the server will recognize that client as a previous person and associate them with their data in the database.
Edit:
The approach I ended up going was this:
I ended up using an approach similar to what OpenSSH does, but reverse. Since when a client connects, I'm not sure I can trust the client.
If it is the first time a client is connecting, it will provide me with a public key and a unique identifier (In this case, I will be using a computer's service tag since the computers I'm going to be dealing with are all Dells). The server stores the id and public key in the database then sends the client a secret question encrypted using the public key. The client will then need to respond with the decrypted answer. Before a client can be accepted by the server, I may end up doing a unique id registration on the server manually before connecting the client for the first time, this way not any sly person can just generate public keys and emulate what a client does with a "new unique id".
Thanks for the help! I hope this helps someone down the line. Feel free to tear apart the security flaws of this approach 🙂
Best Answer
You can't really do authentication without sharing anything before hand. If you can have any sort of shared secret (i.e a key), you can use challenge-response authentication using an appropriate message digest algorithm.
On the topic of protocol compatibility, I would separate the concerns. You can use a traditional handshake to verify that the customer is speaking the same protocol/version the server uses (or a compatible one), but authentication is best dealt with separately.
An alternative with an acceptable tradeoff in many distributed systems is trusting the first connection from a specific client, sharing a key and then validating all subsequent connections from the same client using that. This is what OpenSSH does when connecting to a new server (prompts you whether you want to connect to an unknown host, and then store the fingerprint of that host and validate it on subsequent connections). You can find a more detailed explanation of the SSH "known hosts" mechanism on the WinSCP wiki.
Puppet uses a variant of this where they have a signed (trusted) SSL certificate on each client. When first connecting an agent with the server, the agent will issue a certificate signing request for the server that you then have to manually sign and give to the agent. This has added security but can become annoying quickly if you need to set up lots of clients, or do it without anyone interacting with the system. This is why there is a configuration option that changes the behavior to be like SSH's (trust everyone that you haven't seen before, validate identities only for known hosts). This is described in more detail at the PuppetLabs website.
In the end, what you have is an issue of trust. The question remains: how can you trust a connection from a client? The answer to this isn't technical in nature - accepting any new connection by an unknown host and trusting it blindly might work for some systems, but not for all. It really depends on the tradeoffs of the actual system you are implementing, and the capabilities granted to the client by the server.