DNS: Windows Server 2008 R2 DNS fails on some queries

[ATC]

We had a strange issue where commonly used hostnames (such as www.microsoft.com) would fail but others (such as www.google.com) would work.

[ATC]

Investigating the issue, we saw that large answers would fail consistently while small ones worked.

www.microsoft.com.      1750    IN      CNAME   toggle.www.ms.akadns.net.
toggle.www.ms.akadns.net. 229   IN      CNAME   g.www.ms.akadns.net.
g.www.ms.akadns.net.    229     IN      CNAME   lb1.www.ms.akadns.net.
lb1.www.ms.akadns.net.  229     IN      A       65.55.21.250

www.google.com.         86228   IN      CNAME   www.l.google.com.
www.l.google.com.       100     IN      A       74.125.79.99
www.l.google.com.       100     IN      A       74.125.79.104
www.l.google.com.       100     IN      A       74.125.79.147

Noticing the difference in response sizes we came across the dnscmd 'AddressAnswerLimit' setting: http://technet.microsoft.com/en-us/library/cc940770.aspx

From the command line, type 'dnscmd /Config /AddressAnswerLimit 0'

[Floyd-ATC]

The problem may also be caused by Windows Server 2003 announcing EDNS0 capabilities which may not work with firewalls that by default block UDP packets larger than 512 bytes. To restore traditional DNS behaviour, use the following command:

'dnscmd /Config /EnableEDnsProbes 0'