Source of the number of EBERO incidents as it relates to discussions around the pre-approved RSP program
During the course of today’s call, Donna asked me for the source of the “close to 40 EBERO triggering incidents” that ICANN had detected in the earlier stages of the new gTLD program. Below are the stats and sources. Turns out it was 32 at the time but staff reported that since they started monitoring this, they had been seeing on average 2 incidents per quarter so the number is probably closer to 40 now. You need to pair this with the revelation from Christine Willett during the same session in Bangkok that during PDT, ICANN “regularly encountered registry operators with issues that had remediation, significant remediation to do.” “We’re not talking about a handful of incidents either.” “Pre-delegation testing, registry service testing continues to regularly find issues with basic registry service delivery that require remediation. And its not just about IDN tables.” As Steve suggested in chat and seconded by Anne and also asked for by Donna during a session on RSP Approval program during the GDD Summit in Bangkok, I too think it would be good to get an update on these statistics from GDD as well as more information on PDT issues mentioned. Would be helpful to inform not only the discussions on the RSP program but also the EBERO as a whole when we get to that part of the deliberations. Both of these combined demonstrate the need to continue a testing program as part of an RSP program. Note that this write up is taken from a document circulated to the RySG RSP working team members which included some of the registry participants in Sub Pro. ICANN Data on the Current Registry Operator’s Systems During the Madrid DNS Symposium / ICANN GDD Summit in May 2017 as well as the ICANN 59 meeting in Johannesburg in June 2017, ICANN provided information about potential SLA violations that took place from 2014 through 2016. On August 31, 2017, ICANN updated that data with the most current set of measurements. The data shared by ICANN shows that there have been 32 cases where a gTLD reached one of the emergency thresholds: * 16 out of 32 cases were triggered by perceived failures in the DNS/DNSSEC services, * 16 out of 32 cases were triggered by failures in the RDDS service. Of the 32 cases, 10 occurred prior to the TLD’s Sunrise period, 8 during Sunrise, 5 before general availability, and 9 during general availability. 11 RSPs, 26 gTLDs and a total of 211.7k active names, were involved in the 32 cases. The root cause, which ICANN began tracking in 2015, can be broken down as follows: RDDS: ▪ 3 were due to IPv6 transport failure; ▪ 1 was caused by a broken chain of trust in DNSSEC; and ▪ 1 was due to the web WHOIS service not responding. ▪ For 11 of these RDDS cases, ICANN does not know the root cause as the cases occurred before ICANN began documenting the causes of each incident. DNS/DNSSEC: ▪ 5 cases exhibited issues in which either the DNS servers were not responding or if they were responding, they were returning a malformed DNSSEC response where the NSEC3 records were not included; ▪ 2 cases were caused by expired signatures followed by breakage of the chain of trust in DNSSEC; ▪ 2 cases arose when there was no response from the DNS servers (apparently a routing issue); ▪ 1 case of expired DNSSEC signatures; ▪ 1 break in the chain of trust in DNSSEC; and ▪ 1 case where there were no DS records when requesting delegation from IANA. ▪ 4 of these cases occurred in 2014, for which ICANN does not know the root cause. Links Johannesburg discussion on RSP - https://icann59johannesburg2017.sched.com/event/B49H/gnso-registries-stakeho... Madrid – video of presentation - https://www.youtube.com/watch?v=VA_fCYHAQVo&list=PLQziMT9GXafWAdedvzQYmqvW3N... Slides - https://www.icann.org/en/system/files/files/presentation-slam-13may17-en.pdf RSP discussion at GDD in Bangkok - https://icann.zoom.us/recording/play/1TfcynOARSGE5HPeicn5Bx36vZZsW2jXp5YTmTD... Jim Prendergast The Galway Strategy Group +1 202 285 3699 @jimpren
Thanks Jim for providing the information. It would be very helpful to have the statistics updated to include the past two years, and I’d also welcome more information about the issues with pre-delegation testing. Donna From: Gnso-newgtld-wg [mailto:gnso-newgtld-wg-bounces@icann.org] On Behalf Of Jim Prendergast Sent: Tuesday, May 28, 2019 1:59 PM To: gnso-newgtld-wg@icann.org Subject: [Gnso-newgtld-wg] Source of the number of EBERO incidents as it relates to discussions around the pre-approved RSP program During the course of today’s call, Donna asked me for the source of the “close to 40 EBERO triggering incidents” that ICANN had detected in the earlier stages of the new gTLD program. Below are the stats and sources. Turns out it was 32 at the time but staff reported that since they started monitoring this, they had been seeing on average 2 incidents per quarter so the number is probably closer to 40 now. You need to pair this with the revelation from Christine Willett during the same session in Bangkok that during PDT, ICANN “regularly encountered registry operators with issues that had remediation, significant remediation to do.” “We’re not talking about a handful of incidents either.” “Pre-delegation testing, registry service testing continues to regularly find issues with basic registry service delivery that require remediation. And its not just about IDN tables.” As Steve suggested in chat and seconded by Anne and also asked for by Donna during a session on RSP Approval program during the GDD Summit in Bangkok, I too think it would be good to get an update on these statistics from GDD as well as more information on PDT issues mentioned. Would be helpful to inform not only the discussions on the RSP program but also the EBERO as a whole when we get to that part of the deliberations. Both of these combined demonstrate the need to continue a testing program as part of an RSP program. Note that this write up is taken from a document circulated to the RySG RSP working team members which included some of the registry participants in Sub Pro. ICANN Data on the Current Registry Operator’s Systems During the Madrid DNS Symposium / ICANN GDD Summit in May 2017 as well as the ICANN 59 meeting in Johannesburg in June 2017, ICANN provided information about potential SLA violations that took place from 2014 through 2016. On August 31, 2017, ICANN updated that data with the most current set of measurements. The data shared by ICANN shows that there have been 32 cases where a gTLD reached one of the emergency thresholds: * 16 out of 32 cases were triggered by perceived failures in the DNS/DNSSEC services, * 16 out of 32 cases were triggered by failures in the RDDS service. Of the 32 cases, 10 occurred prior to the TLD’s Sunrise period, 8 during Sunrise, 5 before general availability, and 9 during general availability. 11 RSPs, 26 gTLDs and a total of 211.7k active names, were involved in the 32 cases. The root cause, which ICANN began tracking in 2015, can be broken down as follows: RDDS: ▪ 3 were due to IPv6 transport failure; ▪ 1 was caused by a broken chain of trust in DNSSEC; and ▪ 1 was due to the web WHOIS service not responding. ▪ For 11 of these RDDS cases, ICANN does not know the root cause as the cases occurred before ICANN began documenting the causes of each incident. DNS/DNSSEC: ▪ 5 cases exhibited issues in which either the DNS servers were not responding or if they were responding, they were returning a malformed DNSSEC response where the NSEC3 records were not included; ▪ 2 cases were caused by expired signatures followed by breakage of the chain of trust in DNSSEC; ▪ 2 cases arose when there was no response from the DNS servers (apparently a routing issue); ▪ 1 case of expired DNSSEC signatures; ▪ 1 break in the chain of trust in DNSSEC; and ▪ 1 case where there were no DS records when requesting delegation from IANA. ▪ 4 of these cases occurred in 2014, for which ICANN does not know the root cause. Links Johannesburg discussion on RSP - https://icann59johannesburg2017.sched.com/event/B49H/gnso-registries-stakeholder-group-rysg-rsp-discussion-group<https://urldefense.proofpoint.com/v2/url?u=https-3A__icann59johannesburg2017.sched.com_event_B49H_gnso-2Dregistries-2Dstakeholder-2Dgroup-2Drysg-2Drsp-2Ddiscussion-2Dgroup&d=DwMGaQ&c=MOptNlVtIETeDALC_lULrw&r=CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=Yo6OTm3VWINXRS9p90bq7UCber8ZxnHidMxp8ZsciFs&s=zXOVI6ZiqBYV5ze7P1-YJt8Tly2ctm3tZNxoFB9PMhU&e=> Madrid – video of presentation - https://www.youtube.com/watch?v=VA_fCYHAQVo&list=PLQziMT9GXafWAdedvzQYmqvW3N7SEo5_e&index=6<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3DVA-5FfCYHAQVo-26list-3DPLQziMT9GXafWAdedvzQYmqvW3N7SEo5-5Fe-26index-3D6&d=DwMGaQ&c=MOptNlVtIETeDALC_lULrw&r=CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=Yo6OTm3VWINXRS9p90bq7UCber8ZxnHidMxp8ZsciFs&s=ViPRPR-E3ibbYwvWMht1wsbeYJ57eLAWh9FDV9T9OTc&e=> Slides - https://www.icann.org/en/system/files/files/presentation-slam-13may17-en.pdf<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_en_system_files_files_presentation-2Dslam-2D13may17-2Den.pdf&d=DwMGaQ&c=MOptNlVtIETeDALC_lULrw&r=CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=Yo6OTm3VWINXRS9p90bq7UCber8ZxnHidMxp8ZsciFs&s=THaQroRb7VSxDaQA8O0V5Tn4QAyI49ARt_EP6KGJLvU&e=> RSP discussion at GDD in Bangkok - https://icann.zoom.us/recording/play/1TfcynOARSGE5HPeicn5Bx36vZZsW2jXp5YTmTD5XjJLVpQf-96AdfHleq1DJQTW?startTime=1557287172000<https://urldefense.proofpoint.com/v2/url?u=https-3A__icann.zoom.us_recording_play_1TfcynOARSGE5HPeicn5Bx36vZZsW2jXp5YTmTD5XjJLVpQf-2D96AdfHleq1DJQTW-3FstartTime-3D1557287172000&d=DwMGaQ&c=MOptNlVtIETeDALC_lULrw&r=CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=Yo6OTm3VWINXRS9p90bq7UCber8ZxnHidMxp8ZsciFs&s=dfypPFWVvoCqrvpNo5MQmPfX0U4VyQF2ixFNHfxZEPw&e=> Jim Prendergast The Galway Strategy Group +1 202 285 3699 @jimpren
participants (2)
-
Austin, Donna -
Jim Prendergast