Reverse PTR record lookup not working with dig

You've set it up correctly on your server, but your server is not marked as authoritative for 123.5.253.159.in-addr.arpa. Your reverse DNS is managed by your ISP (the whole 5.253.159.in-addr.arpa zone):

; <<>> DiG 9.7.3-P3 <<>> ns 5.253.159.in-addr.arpa
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43579
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;5.253.159.in-addr.arpa.        IN  NS

;; ANSWER SECTION:
5.253.159.in-addr.arpa. 86010   IN  NS  ns3.uxw.nl.
5.253.159.in-addr.arpa. 86010   IN  NS  ns4.uxw.nl.

You have 2 solutions to solve that:

1. ask your ISP to define the 123.5.253.159.in-addr.arpa record for you on their DNS (ns3.uxw.nl and ns4.uxw.nl),
2. ask your ISP to delegate the 123.5.253.159.in-addr.arpa record to your server, as per the RFC 2317, this is just a matter of defining a CNAME pointing to a new record on a new zone on your DNS (you then have to define this special zone on your DNS, your ISP will tell you how to name this new zone).

First solution is probably the quickest, but if you change your server's name in the future, you've got to ask them to change it again. Second solution is the most flexible, if your ISP is willing to support RFC 2317. If your ISP is providing you with more than a single address, reverse DNS can be delegated for all of them (the whole range) so you can be in charge. I'd recommend you to ask for that.

This similar question is worth reading.

Trivia: a few years ago (5, or 10 years back), some people were advocating against RFC-2317, because some DNS clients couldn't handle the reverse queries well. However, I don't think that is relevant anymore, RFC-2317 exists for about 15 years now.

It doesn't actually really matter what the name in reverse DNS is, as long as the use of names is consistent.

Arguably, as long as there is "forward confirmation" (there exist A/AAAA records that agree with the reverse DNS PTR), the reverse DNS in itself is "correctly set up".

Then there's the matter of what the hostname / "mailname" of the mail server host is set to. Ie, the name that a mailserver uses to refer to itself in the SMTP dialog.
This is expected to match the declared hostname according to reverse DNS, and some, but not all, services out there will care if these differ.

What does not matter is if the hostname of your mailserver has any relation to the domain names it handles mail for.
It's a fairly widespread misconception that these names should match, but them matching would actually be very strange for several reasons:

• Many mail servers handle mail for several domains, but they can just have one hostname, so they can't possibly match all.
• A hostname is expected to be something like zeus.example.com while the domain name it handles mail for, even with just a single domain name, would almost certainly be something like example.com. Certainly no match there either, even if there is sometimes a relation between the names for smaller-scale deployments.

There are of course many other aspects regarding what will cause other mail services out there to accept/reject your email, but these are the basics specific to reverse DNS.