Sorry for the delayed reply David but I do want to thank you for the detailed information. Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:cwg-stewardship-bounces@icann.org] On Behalf Of David Conrad Sent: Thursday, February 11, 2016 11:48 AM To: CWG Mailing List Subject: [CWG-Stewardship] Marc's presentation and report with some background Hi, On the 76th CWG call on February 4th (https://community.icann.org/x/nJBlAw), Marc Blanchet of Viagenie gave a presentation on the work ICANN contracted them to perform that attempted to derive approximations of the SLEs using the current RZMS and RT logs and databases. Attached is the PowerPoint deck that Marc used for his presentation as well as the final report detailing his analyses. I am circulating these reports for your review. Based on the questions raised during the CWG call, it seems there might have been some confusion around why that work was undertaken and how it relates to the implementation of the SLEs. I am providing some background here to help remove any confusion. My apologies in advance for the length of this message, however I believe it important for there to be clarity on this issue. 1. Before the SLEs were developed, ICANN staff informed the DT-A that the current RZMS and RT systems collect performance metrics as directed by NTIA and our own internal requirements. The DT-A felt these metrics were insufficient to ensure IANA performance met community requirements and that new metrics would be necessary. ICANN staff informed the DT-A that changes to performance metrics would require code changes to the RZMS. 2. After the SLEs were developed, ICANN staff informed the CWG that due to the number of simultaneous demands placed upon ICANN to safely and securely modify systems and processes to meet transition requirements, ICANN staff estimated the code changes to RZMS would be completed by the end of March 2016. Some in the CWG suggested ICANN add additional staff to the RZMS development team in an attempt to deploy the code sooner. ICANN informed the CWG that doing so would more likely result in development taking longer since any new developers would need to become familiar with the existing code base and this familiarization would necessarily involve interruptions to the existing development team, delaying their efforts. 3. During ICANN 54, ICANN staff were informed of a new requirement that SLE data must be collected for a period of 6 months before SLAs could be set. ICANN staff was further informed that the March 2016 timeframe was unacceptable since a 6-month data collection requirement would lead to insufficient time being available to incorporate the SLE-derived SLAs into the ICANN-PTI contract. A request was made to ICANN to make available RZMS and RT raw data so that SLEs could be extracted from data collected by the current RZMS/RT systems. Due to the confidential nature of the root zone change request data, which includes email discussions between ICANN staff and the requesters in which potentially business proprietary details of registry operation are disclosed, ICANN staff informed the CWG chairs and DT-A that releasing the raw RZMS/RT data would be a violation of existing IANA policy and thus, would not be possible, particularly to any organization that competes in the domain name space. ICANN staff also again explained that the RZMS/RT systems do not currently collect data the way the SLEs were defined and the primary task for ICANN was to modify RZMS in order to collect the new SLEs. 4. As a compromise to try to address the new 6-month data collection requirement, ICANN staff contracted with Viagenie as an independent and neutral third-party to explore whether the data collected by the current RZMS/RT system could be used to "seed" some portion of the SLEs, thereby reducing the data collection time requirement. 5. Viagenie completed their work and presented a summary on the Feb 4th CWG call, confirming that: (a) The current RZMS/RT systems do not collect data in accordance with the newly defined SLEs; (b) The heuristics developed provided approximations for most metrics, but some approximations were less conclusive; and (c) The RZMS/RT tool is a complex system that frequently relies on email interactions for progressing request state. 6. In parallel to the work ICANN contracted Viagenie to perform, ICANN has continued to pursue modifying the RZMS system to collect data in the way that the SLEs were defined. ICANN now expects that the RZMS modifications will be completed at the end of February, a month ahead of schedule. Data collection for the new SLEs can thus begin in March. Assuming everything goes well, this would allow for sufficient time for SLE data collection without delay of the transition in order to meet the 6-month SLE data collection requirement. The main approach to implementing the SLEs had always been to make code changes to the RZMS so that it can capture processing time data that the DT-A defined for all future change request processing. ICANN is on-track to have this work completed by the end of this month, February. If the CWG would like any additional clarification around the work performed by Viagenie, the RZMS development work, or other aspects of the technical implementation of the transition, please let ICANN staff know and we will be happy to provide clarification. Regards, -drc ICANN CTO