Outlook.com DNS so slow, postfix times out

I'm trying to send e-mails to for example: postmaster@grassvalley.com and postmaster@jorssen.net.bmw.be. In both cases these are Outlook mailboxes.

Postfix replies with:

(when ipv6 enabled):

06F908112E      606 Thu Jun 19 10:20:40  root@dinosaur.rootspirit.com
(Host or domain name not found. Name service error for name=grassvalley-com.mail.protection.outlook.com type=AAAA: Host not found, try again)
                                         postmaster@grassvalley.com

(only ipv4):

6946720087D     3117 Tue Jun 17 21:41:07  root@rootspirit.be
(delivery temporarily suspended: Host or domain name not found. Name service error for name=grassvalley-com.mail.protection.outlook.com type=A: Host not found, try again)
                                         postmaster@grassvalley.com

When checking the DNS I do get a reply (I've tried Google DNS, IPv6 only DNS, Leaseweb DNS, OpenDNS, always the same):

grassvalley-com.mail.protection.outlook.com. 7 IN A 207.46.163.138
grassvalley-com.mail.protection.outlook.com. 7 IN A 207.46.163.170
grassvalley-com.mail.protection.outlook.com. 7 IN A 207.46.163.247

So there is a correct A reply.

Now, what seems to be causing the issue is the ridiculously low TTL of 7 seconds and that every DNS reply to these Outlook domains takes > 2 seconds: Query time: 2039 msec.

This causes Postfix to timeout and not send the e-mails.

If I run this command dig grassvalley-com.mail.protection.outlook.com && postfix flush the e-mail actually get sent, because it happens within the 7 second time frame (DNS being still cached).

Can I tell Postfix to timeout not as fast? Can I tell bind to ignore TTL < 7 seconds from Outlook and cache it for, say, 2 hours?