Python – Reading binary data with PySerial from serial port

pyqtpythonqprocessqtserial-port

PyQT 4.7 does not have inherited class from QIODevice that allows to talk with serial port directly (e.g. QSerialDevice). So I thought that it would be easier for me to use QProcess class and implement the actual reading/writing to serial port from a different process that will interface with my main QT application using QProcess interface.

Now the problem is that amount of bytes sent and received is not the same when I am using the code below. So my question is how to correctly read binary data from a serial port and then forward everything to the stdout?

This is an excerpt from my main QT program that creates QProcess:

        self.micromouse_socket = QProcess()
        self.micromouse_socket.start("/home/ansis/Source/Perforce-pele/Pele/tools/console/comtalker.py", "")
        self.micromouse_socket.started.connect(self.on_micromouse_socket_started)
        self.label_8.setText("Starting COM...")

And this is the Process that will talk with Serial port (comtalker.py; non blocking part is not yet finished):

#!/usr/bin/python
import serial
import sys

if __name__ == "__main__":

    ser = serial.Serial(0)

    while 1 :
        x = ser.read(1)
        sys.stdout.write(x)
        sys.stdout.flush()

P.S. It could be that problem is somewhere else and not in PySerial. On the other computer I am writing to ttyS0 with this command "./binary_data_generator > /dev/ttyS0". The same code seemed to work fine when I was sending only ASCII characters (text+numbers)

Best Answer

It seems that PySerial (or a library that Pyserial depends on) is translating a single "0x0a" (\n) character into two characters "0x0d 0x0a"(\r\n). Both communication end-points are running on Linux, so I am not sure why someone would like to even translate those line endings at all...

Here strace indicates that sender sends only \n to ttyS0:

write(1, "M\n", 2)                      = 2
write(1, "\n", 1)                       = 1
write(1, "M\n", 2)                      = 2
write(1, "\n", 1)                       = 1

While debugging PySerial output I saw that each \n is prefixed with a \r.

Before claiming that this as a Bug I will do further investigation to find out who and why adds this carriage return...

tl;dr: Because of different default settings in C++ requiring more system calls.

By default, cin is synchronized with stdio, which causes it to avoid any input buffering. If you add this to the top of your main, you should see much better performance:

std::ios_base::sync_with_stdio(false);

Normally, when an input stream is buffered, instead of reading one character at a time, the stream will be read in larger chunks. This reduces the number of system calls, which are typically relatively expensive. However, since the FILE* based stdio and iostreams often have separate implementations and therefore separate buffers, this could lead to a problem if both were used together. For example:

int myvalue1;
cin >> myvalue1;
int myvalue2;
scanf("%d",&myvalue2);

If more input was read by cin than it actually needed, then the second integer value wouldn't be available for the scanf function, which has its own independent buffer. This would lead to unexpected results.

To avoid this, by default, streams are synchronized with stdio. One common way to achieve this is to have cin read each character one at a time as needed using stdio functions. Unfortunately, this introduces a lot of overhead. For small amounts of input, this isn't a big problem, but when you are reading millions of lines, the performance penalty is significant.

Fortunately, the library designers decided that you should also be able to disable this feature to get improved performance if you knew what you were doing, so they provided the sync_with_stdio method.

Best Answer

Related Solutions

Python – pyserial – possible to write to serial port from thread a, do blocking reads from thread b

Python – Why is reading lines from stdin much slower in C++ than Python

tl;dr: Because of different default settings in C++ requiring more system calls.

Related Topic