Electronic – the best practices of placing of termination resistor

What resistor do you refer as the termination? The 1 kOhm resistor is still way to much for RS485. If the twisted pair is your media (as it should be), resistors should be around 120 Ohms (100 to 150 Ohms) between the lines. Of course, this should be arranged only at the connection endpoints. Is it possible that your ATC1000 which is working just have those resistors, and in other case they are not present (and you 1 kOhm is not a proper termination).

You migh also want to try some 10 kOhm resistors from each line to the ground. It is not termination, but helps keeping the noise on the lines down when there is no transmission (so that receivers would not get false signal triggering UARTs).

I have also had the case of going 'too fast' into reception mode (when working in half duplex), so that my firmware would cut off part of the message. It was resolved by having receiver on all the time so that my sender would actually 'see' everything that went down the line when transmitting. If that was not the case I would have to arrange for some delay before starting receiving, and delay could be to long so I could miss the start of the message.

This sounds like it should be easy to diagnose. You say the signals on the RS-485 bus look OK, so check the next point. Keep following the signal until you find it not right somewhere.

The next logical point to test after the RS-485 bus would be when it gets converted back to a single-ended signal that presumably gets presented to UART receive pin of the device. Does that look right? If not, there is a problem with the RS-485 receiver circuitry. If so, then either the device is not working or you have a protocol problem.

In general, it's good to keep dividing up the suspect parts of the system to isolate what is not working, or to test them in isolation. For example, you could either try to tap the UART line into the device and see if you get what is expected by looking at the bytes on a PC with a low level program, or you could try a program on the PC to inject what you think is the right sequence of bytes and see if the device responds as you expect.

Chances are that if the UART signal at the device looks OK (including the obvious issue of baud rate), then you have a protocol problem. In other words, somehow you're not sending the right data to the device. Early on in bringing up a system like this, I often use a low level test program on the PC that sends and receives individual bytes. I can type byte values in decimal, hex, and as ASCII character, and it displays everything it receives likewise. This can be very useful for verifying that you really do understand the protocol the device expects.

Do some of these experiments and tell us what you find. It would also help to provide some information on this device you are trying to talk to from the micro.