I'm involved in a project which involves implementing a binary protocol that will be transmitted over TCP. During our early discussions we have hit a brief snag on deciding whether byte stuffing is required if we decide to include the packet length in the field header.
For example. If our packet structure was like so:
[STX] [PACKET LENGTH] [PACKET ID] ..... [ETX]
Do we need to byte stuff? One of my colleagues seems to say that given we know the packet length then even if a special character i.e. STX, ETX appears in the bytes it will not matter as we are reading [Packet length] amount of bytes anyway.
Seems to make sense but I'm not 100% sure if that reasoning is quite right.
Please note that we are not using existing protocol handler type modules such as proto-buf but designing this protocol ourselves for our own use.
To confirm what I think I'm meaning when referring to byte stuffing:
Byte stuffing is a process that transforms a sequence of data bytes
that may contain 'illegal' or 'reserved' values into a potentially
longer sequence that contains no occurrences of those value
Taken from Wiki – Consistent Overhead Byte Stuffing
Best Answer
Your colleague is correct.
If you have total packet length, and the data payload is the only variable-length piece in the packet, then you do not need to byte stuff the payload. If you have more than one variable-length piece, you will have to think about how you will disambiguate the various chunks, so you know where one stops and the next begins.
Byte stuffing is used when it is not feasible to insert the packet length into the stream before the packet data payload. USUALLY, this is because messages are of varying lengths and the message length cannot be known a priori.