SSAC-Evo-Reso-WP

Download

ssac-evo-reso-wp@icann.org

March 2023

2 participants
2 discussions

Fwd: dns-operations Digest, Vol 206, Issue 7
by Andrew McConachie March 30, 2023

March 30, 2023

This is a followup to the mail Rod sent to the list on March 14. Basically they increased the number of times a UDP src port can be reused from 2 to 3. Thereby reducing the probability of Chrome switching to TCP. —Andrew > Begin forwarded message: > > From: Adam Casella <acasella(a)infoblox.com> > Subject: Re: dns-operations Digest, Vol 206, Issue 7 > Date: 29 March 2023 at 23:03:28 CEST > To: "dns-operations(a)dns-oarc.net" <dns-operations(a)dns-oarc.net>, "dns-operations(a)lists.dns-oarc.net" <dns-operations(a)lists.dns-oarc.net> > > Just following up on this. This issue was narrowed down to a need to increase the entropy threshold on Chrome’s DNS source port logic on Windows 10 and 11 to prevent the built-in DNS client from falling back to TCP. This impacts all Chromium based browsers and the fix can be found here: > > Chrome Bug tracking this issue: > > https://bugs.chromium.org/p/chromium/issues/detail?id=1413620 [bugs.chromium.org] <https://urldefense.com/v3/__https://bugs.chromium.org/p/chromium/issues/det…> > > Fix (a one-liner) can be found here: > > https://chromium.googlesource.com/chromium/src/+/59d686c1417b5aea7b1d94a28b… [chromium.googlesource.com] <https://urldefense.com/v3/__https://chromium.googlesource.com/chromium/src/…> > > This looks like the fix will be added in Chrome 112 or 113. > > Thanks, > > Adam Casella | Solutions Architect > Infoblox | infoblox.com <http://infoblox.com/> > 914.953.8571 > > From: dns-operations <dns-operations-bounces(a)dns-oarc.net <mailto:dns-operations-bounces@dns-oarc.net>> on behalf of dns-operations-request(a)dns-oarc.net <mailto:dns-operations-request@dns-oarc.net> <dns-operations-request(a)dns-oarc.net <mailto:dns-operations-request@dns-oarc.net>> > Date: Friday, March 17, 2023 at 5:02 AM > To: dns-operations(a)lists.dns-oarc.net <mailto:dns-operations@lists.dns-oarc.net> <dns-operations(a)lists.dns-oarc.net <mailto:dns-operations@lists.dns-oarc.net>> > Subject: dns-operations Digest, Vol 206, Issue 7 > > !-------------------------------------------------------------------| > This Message Is From an External Sender > This message came from outside your organization. > |-------------------------------------------------------------------! > > Send dns-operations mailing list submissions to > dns-operations(a)lists.dns-oarc.net <mailto:dns-operations@lists.dns-oarc.net> > > To subscribe or unsubscribe via the World Wide Web, visit > https://urldefense.com/v3/__https://lists.dns-oarc.net/mailman/listinfo/dns… <https://urldefense.com/v3/__https:/lists.dns-oarc.net/mailman/listinfo/dns-…> > or, via email, send a message with subject or body 'help' to > dns-operations-request(a)lists.dns-oarc.net <mailto:dns-operations-request@lists.dns-oarc.net> > > You can reach the person managing the list at > dns-operations-owner(a)lists.dns-oarc.net <mailto:dns-operations-owner@lists.dns-oarc.net> > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of dns-operations digest..." > > > Today's Topics: > > 1. Re: Increase in DNS over TCP from Chrome Browser on Windows > 11 (David Zych) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 16 Mar 2023 11:57:00 -0500 > From: David Zych <dmrz(a)illinois.edu <mailto:dmrz@illinois.edu>> > To: "dns-operations(a)lists.dns-oarc.net <mailto:dns-operations@lists.dns-oarc.net>" > <dns-operations(a)lists.dns-oarc.net <mailto:dns-operations@lists.dns-oarc.net>> > Subject: Re: [dns-operations] Increase in DNS over TCP from Chrome > Browser on Windows 11 > Message-ID: <13b9d8bc-55d3-a069-d907-299b8dad9d53(a)illinois.edu <mailto:13b9d8bc-55d3-a069-d907-299b8dad9d53@illinois.edu>> > Content-Type: text/plain; charset=UTF-8; format=flowed > > On 3/15/23 11:29, Adam Casella wrote: > > It seems that Chrome is leveraging 1 TCP session per DNS query to prevent tracking of the DNS traffic, which unfortunately does not take advantage of TCP pipelining/multiplexing or out-of-order TCP DNS responses over a single TCP stream. > > Hi Adam, thanks for sharing this! > > We definitely noticed a dramatic increase in TCP DNS requests circa Mon 2022-11-07, for which I'm grateful to finally have a plausible explanation. > > The use of 1 TCP session per query is especially significant because our recursive resolvers have iptables rules designed to prevent them from being monopolized by a single misbehaving client, which includes limiting the number of parallel inbound 53/tcp connections per client IP. The sudden increase in throttling by that particular iptables rule was quite a surprise. > > Thanks, > David > > -- > David Zych (he/him) > Lead Network Service Engineer > > University of Illinois Urbana-Champaign > Office of the Chief Information Officer > Technology Services > > Under the Illinois Freedom of Information Act any written communication to or from university employees regarding university business is a public record and may be subject to public disclosure. > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > dns-operations mailing list > dns-operations(a)lists.dns-oarc.net <mailto:dns-operations@lists.dns-oarc.net> > https://urldefense.com/v3/__https://lists.dns-oarc.net/mailman/listinfo/dns… <https://urldefense.com/v3/__https:/lists.dns-oarc.net/mailman/listinfo/dns-…> > > > ------------------------------ > > End of dns-operations Digest, Vol 206, Issue 7 > **********************************************

1 0

Fwd: [dns-operations] Increase in DNS over TCP from Chrome Browser on Windows 11
by Rod Rasmussen March 14, 2023

March 14, 2023

This came in on the DNS Ops list. Seems like both an interesting potential problem, but also something that is rather pertinent to our work on this WP. Will be interesting to see how this issue evolves! > Begin forwarded message: > > > Hey Folks, > > Has anyone else seen an increase in DNS over TCP traffic in their environment? We have been seeing a steady increase since late last year and have believe we have narrowed down a major cause. After reaching out to the Chromium folks and Cricket Liu reaching out to the Microsoft folks it seems that there has been a recent behavior change that is incompatible with each other, which is causing DNS over TCP to be preferred over UDP. > > Based on my discussion with the Chromium team, It appears that for about 3 years Chrome has a bit of internal logic around falling back to TCP when there is a detection of reduced UDP port entropy being handed out by the OS. When the Chrome stack falls back to TCP, according to the Chromium folks, it will continue to use TCP until Chrome is restarted or there is a network change (port flap, IP address change, etc). The code that tracks the low entropy can be found here net::DnsUdpTracker <https://urldefense.com/v3/__https:/nam06.safelinks.protection.outlook.com/?…>. > > The Chromium folks confirmed that they are seeing an increase of TCP traffic from Windows client only. Crickey Liu reach out to the Microsoft folks and dfound that starting with Windows 11, the OS began to use socket caching due to exhaustion occurring with UDP ports. Meaning that DNS UDP port is cached when communicating with the same server and any DNS client will continue to get the same UDP src-port when connecting to that DNS server. Now starting in Chrome 105, there was a change made by the Chromium folks to leverage the internat Chrome DNS stack to to run more Windows DNS queries through the Chrome stack instead of delegating the resolutions to the OS. Due to the low UDP port entropy logic discussed above in combination to the socket caching introduced Windows 11, we are seeing DNS clients preferring TCP over UDP for what seemed like to discernable reason until these discussions with the Chromium and Microsoft folks. > > From our perspective this is and will cause a lot of issues for DNS providers as more and more Chrome + Windows clients begin to prefer TCP over UDP for DNS. And believe this has the potential to quickly become a rather large issue for DNS providers, especially at scale. > > Is anyone here seeing a seemingly unexplained increase in DNS over TCP traffic and if it is causing any issues within their network? > > For reference, Google Chrome version 105 was released on August 30th, 2022 and Windows 11 was released on October 5th, 2021. Only with the combination of the two (post August 30th, 2022) would the issue be seen. > > Thanks, > > Adam Casella | Solutions Architect > Infoblox | infoblox.com <http://infoblox.com/>

1 0