Your first para is correct. There is no connection, though some IP engines use source address/port and destination address/port to provide a synchronicity-based approximation to state.
Refusal is handled with ICMP, often ICMP port-unreachable (type 3 subtype 3).
Your last para is also correct; it's up to your application to ensure that UDP-transmitted data is correctly sequenced.
Your proxy server is thus doing a double NAT, masquerading (by hiding the other real peer) both ways.
It has to be configured as a router on its (single) interface or nothing will be forwarded. Let's say the interface name is eth0, and its IP is 10.1.2.50:
echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
Now because of the double NAT, the transformations in PREROUTING and/or POSTROUTING prevent matching the flow just using again the same filter. So in PREROUTING a CONNMARK is used to mark the flow which can then be reused immediately in PREROUTING (and avoiding duplicating the specific test for easier management) and later in POSTROUTING. Note -p udp is still needed because we ask SNAT and/or DNAT to alter UDP ports. After that conntrack will handle all future exchange, as long as it doesn't time out (usually 30s for first reply, then 3mn)
iptables -t nat -A PREROUTING -s 10.1.2.10 -p udp --dport 5005 -j CONNMARK --set-mark 0x1
iptables -t nat -A PREROUTING -p udp -m connmark --mark 0x1 -j DNAT --to-destination 10.1.2.20:5010
iptables -t nat -A POSTROUTING -m connmark --mark 0x1 -j SNAT --to-source 10.1.2.50
If it's really a requirement that you also alter the source port, instead of having conntrack do it only when needed, change the 3rd rule into:
iptables -t nat -A POSTROUTING -p udp -m connmark --mark 0x1 -j SNAT --random-fully --to-source 10.1.2.50:32768-60999
(usual range taken from /proc/sys/net/ipv4/ip_local_port_range, and --random-fully is needed else nothing is done because the original port is already in range).
You could also change SNAT to MASQUERADE (and not have to state server IP) but only --random is available.
UPDATE: while the above rules work as intended, usually the connmark match and the CONNMARK target are intended for interaction with mark and MARK for a more complex usage as described in this blog. They are not needed for the simple usage here. Simply replace them all with resp. mark and MARK. Here netfilter's connection tracking already takes care of the flow once the first packet is handled (creating a conntrack entry in NEW state) without having to ask it to mark it in the conntrack entry. A mark is just put in the (first) packet (the nat table sees only packets in NEW state):
iptables -t nat -A PREROUTING -s 10.1.2.10 -p udp --dport 5005 -j MARK --set-mark 0x1
iptables -t nat -A PREROUTING -p udp -m mark --mark 0x1 -j DNAT --to-destination 10.1.2.20:5010
iptables -t nat -A POSTROUTING -m mark --mark 0x1 -j SNAT --to-source 10.1.2.50
Example captures, including errors (service not running etc.):
00:28:36.476453 IP 10.1.2.10.56955 > 10.1.2.50.5005: UDP, length 5
00:28:36.476487 IP 10.1.2.50.35172 > 10.1.2.20.5010: UDP, length 5
00:28:36.476516 IP 10.1.2.20 > 10.1.2.50: ICMP 10.1.2.20 udp port 5010 unreachable, length 41
00:28:36.476522 IP 10.1.2.50 > 10.1.2.10: ICMP 10.1.2.50 udp port 5005 unreachable, length 41
00:32:28.597050 IP 10.1.2.10.35443 > 10.1.2.50.5005: UDP, length 5
00:32:28.597084 IP 10.1.2.50.36842 > 10.1.2.20.5010: UDP, length 5
00:32:32.503709 IP 10.1.2.20.5010 > 10.1.2.50.36842: UDP, length 7
00:32:32.503745 IP 10.1.2.50.5005 > 10.1.2.10.35443: UDP, length 7
00:32:41.704371 IP 10.1.2.20.5010 > 10.1.2.50.36842: UDP, length 4
00:32:41.704404 IP 10.1.2.50.5005 > 10.1.2.10.35443: UDP, length 4
00:32:41.704427 IP 10.1.2.10 > 10.1.2.50: ICMP 10.1.2.10 udp port 35443 unreachable, length 40
00:32:41.704433 IP 10.1.2.50 > 10.1.2.20: ICMP 10.1.2.50 udp port 36842 unreachable, length 40
Please note that, you can replace all of this with this simple socat line (and you don't get any 30s/3mn timeout but you lose ICMP handling):
socat UDP4-LISTEN:5005,range=10.1.2.10/32,fork UDP4:10.1.2.20:5010
Best Answer
It depends on the application. For example, the Linux kernel implemented UDP source port randomization when no source port is specified in kernel 2.6.24.
So, the behavior you are seeing in some connections must be the particular application specifying the source port to be the same as the destination port, while others are leaving it to the kernel.