When you are debugging, you are giving the port driver lots of time to receive a byte. The timeout timer doesn't start running until you step over the ReadByte() call, the driver probably already has received the byte so ReadByte() returns immediately. This doesn't happen when you run at full speed.
Increase the value of the ReadTimeout property. Also consider using the DataReceived event instead.
A receive buffer size of 100*1024*1024 is HUGE! I would seriously doubt you need this size at all, and certainly not for the actual serial port buffer.
It could be that you are overflowing the physical receive buffer of the receiver, so you may have to look into using flow control. This will effectively enable your Receiver to say to your Transmitter "Hang on, stop sending for a while, let me deal with what I have first."
Hardware flow-control is (commonly) utilised through the RTS (Request To Send) and CTS (Clear To Send) pins.
Take a look at this article, which explains a bit more about it.
I would personally recommend leaving the ReceivedBytesThreshold property of the serial port to its default value of 1, and then handling the DataReceived event. Who knows, maybe tomorrow you will need to read a message of 20 bytes, or perhaps 5 bytes. Maybe you will need to read messages of variable length in the future? Leaving the threshold at 1 will mean that you can handle any and all bytes that will be received, now and in the future.
This threshold will mean the event will fire when there is at least 1 byte in the buffer. There maybe more, and probably will be. Note that it does NOT necessarily mean it will fire for every single byte received. On each event, you need to check the BytesToRead property and read this many into your own buffer.
It is worth repeating that the event will NOT necessarily fire for every single byte received.
Also, it is best to do as little as possible within the DataReceived event handler. Read in any bytes received into your buffer, and perhaps add completed messages into a queue for further processing, but do little else.
Also note that the standard maximum spec of RS232 is a speed of 19200 baud and a length of cable of 50 feet. Anything above that is undefined. Higher speeds generally need better cable (lower capacitance) and shorter cable lengths. Make sure it is screened and not running near other 'noisy' items such as motors, motor inverters, live cables etc.
Best Answer
If the problem seems to be that you can't process the data fast enough, what you could try would be to double-buffer your data.
1) Allow one thread to read the serial port into one buffer. This may involve copying data off the port into the buffer (i'm not intimately familiar with .NET).
2) When you are ready to handle the incoming data, (on a different thread) make your program read into the 2nd buffer, and while this is happening you should write the first buffer to disk.
3) When the first buffer is written to disk, swap it back to the serial port buffer, and write the 2nd buffer to disk. Repeat process, continually swapping the buffers.