Dear CWG members, Jay Daley (.NZ) has prepared some excellent Q&A to update the community on the work of the SLE Working Group which may be of interest. Rest assured we are trying to progress the drafting of the SLE which should be concluded in the next few weeks so ICANN / IANA has a framework for monitoring their systems as part of the Implementation Phase. Kind regards Paul ----- Forwarded message from Jay Daley <jay@nzrs.net.nz> ----- Date: Mon, 22 Jun 2015 15:29:26 -0300 From: Jay Daley <jay@nzrs.net.nz> Reply-To: Jay Daley <jay@nzrs.net.nz> Subject: [DTA - SLE] Questions and Answers - Revised To: dt1@icann.org I had some feedback on my initial Q&As so Iâve updated them below. The changes are about a) The IANA system development; b) The plan for more data capture to inform the performance targets; c) the next steps. Now that Iâve had some feedback, I was planning to share these later today with a few people who are involved in CWG-Stewardship and who are concerned about our progress - please let me know if this is an issue. Jay Q: Why has there been so little open discussion by DT-A on the mailing list? It all looks very secretive? A: We began work some time before a DT-A mailing list was established and by the time it was, a lot of work on drafting had already taken place. Soon after the list was established the issue of how IANA would be involved was resolved with the full engagement from the IANA staff, and it was felt that direct conversations with IANA were the best way to take advantage of that rather than slow conversations on the list. In order to engage fully IANA had to get assemble a series of documents that detail their internal processes and then get permission from the NTIA to release them. Altogether this process took a long time. Q: What involvement has ICANN/IANA had and how committed are they to this? A: Initially it appeared to us that ICANN was only going to engage at a senior executive level to insist that the SLA stays unchanged from that in place currently. This was not acceptable to the DT-A team as we regard the current NTIA approved SLA as well below the minimum SLA required for a service of such global importance. Fortunately IANA staff are now involved and they are fully committed to a proper SLA that serves the need of customers and drives internal improvements. It is useful to note that an SLE (Service Level Expectation) is generally set by customers alone, whereas with this full engagement from IANA it is possible for us to agree an SLA (Service Level Agreement) which is equally supported by both customers and the providers. Q: Does the DT-A team have an agenda here, such as forcing through the automation of IANA? A: No. The team has taken the view from the outset that it is not our role to change the processes in any way. IANA have already started a process of automation of some elements of their service, as presented at various ICANN meetings, and we are trying to keep up with that but we are not introducing any agenda of our own. Q: What is the general plan for the SLA? A: Conceptually it breaks down into three parts: 1. A full service definition with each service broken down into constituent processes and stages within those processes that require individual measurement. 2. An initial set of performance targets. 3. An initial set of breach thresholds - a percentage of requests that must meet the performance target or IANA is in breach of the SLA. A set of principles has been developed to guide the work on these three parts. Q: What is being measured? A: Both time to complete a request (broken down by stage) and the accuracy of the work completed. For some processes, such as nameserver changes, the accuracy needs to be 100% but for others a lower standard is acceptable. Q: Is all that measurement necessary? A: It is the clear consensus view of DT-A that without this level of measurement it is not possible to define an SLA nor is it possible to ensure that an SLA meets the needs of the customer. The breakdown into stages of processes has been chosen to reflect the different responsibilities for those stages and the different nature of those stages. For example, if IANA ask for more information about a root change then the clock needs to stop and the responsibility for progressing that request transfers from IANA to the requestor. This level of measurement is vital to resolve, once and for all, two different perceptions that people have of IANA performance: - that IANA operates a two-tier SLA with contracted TLDs receiving a faster service than non-contracted TLDs while both are still within the overall SLA. - that IANA is the primary cause of delays. By specifying stages where the clock stops when responsibility transfers, this ensures an accurate measurement of IANA performance. Q: How were the initial performance targets set? A: It is important to note that no targets have yet been set. Some targets were initially proposed based on analysis of previous IANA performance (the full data set used for the analysis is available) but we have agreed with IANA that we need more data and are hoping for 2-3 months of data capture from IANA. The key takeaway is that the performance targets in current NTIA SLA are far looser than we as customers believe they should be and IANA as the operator clearly recognises this because it is over-performing by so much. Q: Has work been done to ensure that these targets match the importance of the processes and are not arbitrary? Isnât there a problem that if we use current performance then IANA needs to establish a base line even higher to ensure that they never breach those targets? A: The answers are yes the work has been done and no there isnât a problem. These issues are addressed by the use of breach levels, which work in conjunction with the performance targets. For example a performance target might be set for a specific type of customers request at 5 days based on evidential analysis, but the breach threshold might be set at 90%, which means that only 90% of requests of this type need to meet the 5 day target. This breach threshold serves a dual purpose: - It provides headroom for IANA above the target by allowing them to ignore the worst performing requests. - This level of headroom given reflects the customer priority of this type of request. Q: Once these targets are set are they fixed in stone? A: No, these are an initial set of targets and it is expected that they will develop over time in two ways: - ongoing process improvement by IANA leading to improvements in performance - better consultation with customers to understand our priorities better, leading to adjustments in targets to reflect that Q: How important is it that these elements are in place before transition? A: The view of the DT-A team is that a full SLE/SLA needs to be in place by the start of the transition however IANA have noted that there are some pre-requisites that must be met before they can implement it. These are: 1. The current IT system needs to have features added to extract the required measurements. This development can only begin once there is certainty that this transition is going ahead on this basis and IANA can only turn those changes on after transition given the nature of their agreement with the NTIA. It should also be remembered that IANA will need to make a case for development resources to the ICANN executive team as IANA has no development resources of its own. 2. IANA staff will need time to adapt to the changes in their working practices to implement this measurement. Q: When will the DT-A be finished? A: We are close to completing the SLA except the actual performance targets. These will be added later after a data collection period and discussion with IANA. -- Jay Daley Chief Executive NZRS Ltd desk: +64 4 931 6977 mobile: +64 21 678840 linkedin: www.linkedin.com/in/jaydaley _______________________________________________ dt1 mailing list dt1@icann.org https://mm.icann.org/mailman/listinfo/dt1 ----- End forwarded message -----