SLE update - ICANN seeks to delay SLE Accountability reporting......
Dear all I note today's CWG call has been cancelled and there is a request for a SLE progress update. As you know the IANA Service Level standards developed by members of the Community and ICANN staff was agreed by all involved in September. This Service Level Expectation (SLE) captures _current_ IANA processes and current transaction times for updates to the current Root Zone Management System and introduces thresholds for each stage of the process - thereby building a framework of accountability to the naming community. In June, as part of validating the SLE compliance thresholds for their current processes, the SLE WG proposed a test phase. The approach of having a test phase was accepted by ICANN (and NTIA) to simulate IANA's life "post" NTIA's oversight. (Copy below of email explaining this sent to the CWG Chairs below). The test phase is scheduled to start 1st January 2016. I attended a meeting in Brussels on Wednesday and met with Elise Gerish, IANA Manager who told me that ICANN does not have the resources to identify their current RZM processes in terms of time taken for the SLE. Consequently, they are proposing to delay the testing of the naming community's reporting requirements until after March/April 2016. This was news to me and I assume/hope it would be unreasonable to deduce that ICANN are delaying to potentially seize control of the IANA RZM (from NTIA's oversight) without having to adhere to the operational SLE standards agreed with the community. The SLE document is fundamental to ensuring that ICANN delivers an accountable service to each Registry operator and the naming community generally. It is essential that the community developed SLE is in place as soon as possible. A while back, I spoke with the contractor who wrote the IANA RZM system and he said most of the data is already captured, it is just that ICANN wanted the data extraction tools disabled. Having an effective transaction log of each process step is a fundamental part of Information Security Management (ISO/IEC 27001)- and is basic good practice - and it should not be a complicated task to generate the SLE reports from the transaction logs. It is essential that at the time of transition we have a tried/tested/proven SLE standard in place so ICANN is accountable to our naming community. Can I encourage every member of the CWG (and ICG) to invite ICANN to engage the necessary resources to enable the SLE accountability reporting mechanisms from the 1st January 2016 on their test (post NTIA) RZM platform ...... as they are doing for the other IANA customers (numbers and protocols)? Thanks Best Paul -------- Forwarded Message -------- Subject: DTA - got there .... Date: Tue, 09-Jun-2015 07:45 +0100 From: Paul M Kane <Paul.Kane@icb.co.uk> To: Jonathan Robinson <jrobinson@afilias.info>, lise.fuhr@difo.dk <lise.fuhr@difo.dk> I am pleased to advise ICANN/IANA and the Service Level Design team reached agreement for a proper SLE to be in place post transition. IANA are offering to capture the additional time stamps necessary to ascertain their performance. We do need the CWG (and or ICG) to sanction the continuation of our work, after the submission to ICG, as there may be moves by ICANN to kill a demanding SLE once it leaves our oversight and CWG's role is wound down. What has been agreed (privately). 1. The CWG/ICG Community needs to sanction the work of the SLE Team as a Community initiative. 2. IANA and Adam (my guy) are going to refine the SLE and then present that to the DTA for adoption (approx 4 weeks to finish). 3. Once agreed, IANA will develop an Test Phase plan (2 weeks) including an internal ICANN request for Technical Resources. 4. The Test phase needs NTIA approval - 2 weeks. 5. Once NTIA has approved, the resources to extract the times needed will be deployed.- 1 month 6. IANA will then for 2 to 3 months run a trial capturing real world transaction information and provide that to the DT 7. With real world data (that IANA is comfortable with) - we will populate the agreed thresholds for the SLE. 8. With the SLE data specified (Jan/Feb 2016), the SLE will be in place for the Implementation Phase. 9. Checking of the SLE during the Implementation Phase 10. SLE in place from the date of Transition ! I will be on the call later - estimated arrival time 5-30 UTC - sorry for being late Best Paul
Thanks for this update Paul. This is a critical part of our CWG process and it worries me to hear that there may be potential delays from ICANNs side on this incredibly important matter. If necessary short term additional staff should be provided from the ICANN Reserve Fund in order to fulfil this community expectation. As a security guy in my day job I share your concern about the apparent/potential lack of an audit trail and transaction log for this system, I cannot see this being the case in reality given ICANNs recent security issues I would think that systems closest to the root would be heavily logged and tracked. This is a troubling point and one that should be taken up by senior ICANN staff/board members if necessary and our co-chairs at the earliest possible juncture. -James Gannon On 13/10/2015, 11:31 a.m., "cwg-stewardship-bounces@icann.org on behalf of Paul M Kane - CWG" <cwg-stewardship-bounces@icann.org on behalf of paul.kane-cwg@icb.co.uk> wrote:
Dear all
I note today's CWG call has been cancelled and there is a request for a SLE progress update.
As you know the IANA Service Level standards developed by members of the Community and ICANN staff was agreed by all involved in September.
This Service Level Expectation (SLE) captures _current_ IANA processes and current transaction times for updates to the current Root Zone Management System and introduces thresholds for each stage of the process - thereby building a framework of accountability to the naming community.
In June, as part of validating the SLE compliance thresholds for their current processes, the SLE WG proposed a test phase. The approach of having a test phase was accepted by ICANN (and NTIA) to simulate IANA's life "post" NTIA's oversight. (Copy below of email explaining this sent to the CWG Chairs below). The test phase is scheduled to start 1st January 2016.
I attended a meeting in Brussels on Wednesday and met with Elise Gerish, IANA Manager who told me that ICANN does not have the resources to identify their current RZM processes in terms of time taken for the SLE. Consequently, they are proposing to delay the testing of the naming community's reporting requirements until after March/April 2016. This was news to me and I assume/hope it would be unreasonable to deduce that ICANN are delaying to potentially seize control of the IANA RZM (from NTIA's oversight) without having to adhere to the operational SLE standards agreed with the community.
The SLE document is fundamental to ensuring that ICANN delivers an accountable service to each Registry operator and the naming community generally. It is essential that the community developed SLE is in place as soon as possible.
A while back, I spoke with the contractor who wrote the IANA RZM system and he said most of the data is already captured, it is just that ICANN wanted the data extraction tools disabled. Having an effective transaction log of each process step is a fundamental part of Information Security Management (ISO/IEC 27001)- and is basic good practice - and it should not be a complicated task to generate the SLE reports from the transaction logs.
It is essential that at the time of transition we have a tried/tested/proven SLE standard in place so ICANN is accountable to our naming community.
Can I encourage every member of the CWG (and ICG) to invite ICANN to engage the necessary resources to enable the SLE accountability reporting mechanisms from the 1st January 2016 on their test (post NTIA) RZM platform ...... as they are doing for the other IANA customers (numbers and protocols)?
Thanks
Best
Paul
-------- Forwarded Message -------- Subject: DTA - got there .... Date: Tue, 09-Jun-2015 07:45 +0100 From: Paul M Kane <Paul.Kane@icb.co.uk> To: Jonathan Robinson <jrobinson@afilias.info>, lise.fuhr@difo.dk <lise.fuhr@difo.dk>
I am pleased to advise ICANN/IANA and the Service Level Design team reached agreement for a proper SLE to be in place post transition.
IANA are offering to capture the additional time stamps necessary to ascertain their performance.
We do need the CWG (and or ICG) to sanction the continuation of our work, after the submission to ICG, as there may be moves by ICANN to kill a demanding SLE once it leaves our oversight and CWG's role is wound down.
What has been agreed (privately).
1. The CWG/ICG Community needs to sanction the work of the SLE Team as a Community initiative. 2. IANA and Adam (my guy) are going to refine the SLE and then present that to the DTA for adoption (approx 4 weeks to finish). 3. Once agreed, IANA will develop an Test Phase plan (2 weeks) including an internal ICANN request for Technical Resources. 4. The Test phase needs NTIA approval - 2 weeks. 5. Once NTIA has approved, the resources to extract the times needed will be deployed.- 1 month 6. IANA will then for 2 to 3 months run a trial capturing real world transaction information and provide that to the DT 7. With real world data (that IANA is comfortable with) - we will populate the agreed thresholds for the SLE. 8. With the SLE data specified (Jan/Feb 2016), the SLE will be in place for the Implementation Phase. 9. Checking of the SLE during the Implementation Phase 10. SLE in place from the date of Transition !
I will be on the call later - estimated arrival time 5-30 UTC - sorry for being late
Best
Paul
_______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
All, Lise & I will have the opportunity to meet with senior ICANN staff to discuss their proposed planning and execution work on implementation (of the CWG's recommendations). We can take this issue up with them then and propose that they attend our Wednesday 21 Oct meeting to discuss further with the CWG. Thank-you. Jonathan -----Original Message----- From: James Gannon [mailto:james@cyberinvasion.net] Sent: 13 October 2015 11:41 To: Paul M Kane - CWG <paul.kane-cwg@icb.co.uk>; cwg-stewardship@icann.org Cc: dt1@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... Thanks for this update Paul. This is a critical part of our CWG process and it worries me to hear that there may be potential delays from ICANNs side on this incredibly important matter. If necessary short term additional staff should be provided from the ICANN Reserve Fund in order to fulfil this community expectation. As a security guy in my day job I share your concern about the apparent/potential lack of an audit trail and transaction log for this system, I cannot see this being the case in reality given ICANNs recent security issues I would think that systems closest to the root would be heavily logged and tracked. This is a troubling point and one that should be taken up by senior ICANN staff/board members if necessary and our co-chairs at the earliest possible juncture. -James Gannon On 13/10/2015, 11:31 a.m., "cwg-stewardship-bounces@icann.org on behalf of Paul M Kane - CWG" <cwg-stewardship-bounces@icann.org on behalf of paul.kane-cwg@icb.co.uk> wrote:
Dear all
I note today's CWG call has been cancelled and there is a request for a SLE progress update.
As you know the IANA Service Level standards developed by members of the Community and ICANN staff was agreed by all involved in September.
This Service Level Expectation (SLE) captures _current_ IANA processes and current transaction times for updates to the current Root Zone Management System and introduces thresholds for each stage of the process - thereby building a framework of accountability to the naming community.
In June, as part of validating the SLE compliance thresholds for their current processes, the SLE WG proposed a test phase. The approach of having a test phase was accepted by ICANN (and NTIA) to simulate IANA's life "post" NTIA's oversight. (Copy below of email explaining this sent to the CWG Chairs below). The test phase is scheduled to start 1st January 2016.
I attended a meeting in Brussels on Wednesday and met with Elise Gerish, IANA Manager who told me that ICANN does not have the resources to identify their current RZM processes in terms of time taken for the SLE. Consequently, they are proposing to delay the testing of the naming community's reporting requirements until after March/April 2016. This was news to me and I assume/hope it would be unreasonable to deduce that ICANN are delaying to potentially seize control of the IANA RZM (from NTIA's oversight) without having to adhere to the operational SLE standards agreed with the community.
The SLE document is fundamental to ensuring that ICANN delivers an accountable service to each Registry operator and the naming community generally. It is essential that the community developed SLE is in place as soon as possible.
A while back, I spoke with the contractor who wrote the IANA RZM system and he said most of the data is already captured, it is just that ICANN wanted the data extraction tools disabled. Having an effective transaction log of each process step is a fundamental part of Information Security Management (ISO/IEC 27001)- and is basic good practice - and it should not be a complicated task to generate the SLE reports from the transaction logs.
It is essential that at the time of transition we have a tried/tested/proven SLE standard in place so ICANN is accountable to our naming community.
Can I encourage every member of the CWG (and ICG) to invite ICANN to engage the necessary resources to enable the SLE accountability reporting mechanisms from the 1st January 2016 on their test (post NTIA) RZM platform ...... as they are doing for the other IANA customers (numbers and protocols)?
Thanks
Best
Paul
-------- Forwarded Message -------- Subject: DTA - got there .... Date: Tue, 09-Jun-2015 07:45 +0100 From: Paul M Kane <Paul.Kane@icb.co.uk> To: Jonathan Robinson <jrobinson@afilias.info>, lise.fuhr@difo.dk <lise.fuhr@difo.dk>
I am pleased to advise ICANN/IANA and the Service Level Design team reached agreement for a proper SLE to be in place post transition.
IANA are offering to capture the additional time stamps necessary to ascertain their performance.
We do need the CWG (and or ICG) to sanction the continuation of our work, after the submission to ICG, as there may be moves by ICANN to kill a demanding SLE once it leaves our oversight and CWG's role is wound down.
What has been agreed (privately).
1. The CWG/ICG Community needs to sanction the work of the SLE Team as a Community initiative. 2. IANA and Adam (my guy) are going to refine the SLE and then present that to the DTA for adoption (approx 4 weeks to finish). 3. Once agreed, IANA will develop an Test Phase plan (2 weeks) including an internal ICANN request for Technical Resources. 4. The Test phase needs NTIA approval - 2 weeks. 5. Once NTIA has approved, the resources to extract the times needed will be deployed.- 1 month 6. IANA will then for 2 to 3 months run a trial capturing real world transaction information and provide that to the DT 7. With real world data (that IANA is comfortable with) - we will populate the agreed thresholds for the SLE. 8. With the SLE data specified (Jan/Feb 2016), the SLE will be in place for the Implementation Phase. 9. Checking of the SLE during the Implementation Phase 10. SLE in place from the date of Transition !
I will be on the call later - estimated arrival time 5-30 UTC - sorry for being late
Best
Paul
_______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
_______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
James, On 10/13/15, 11:41 AM, "James Gannon" <james@cyberinvasion.net> wrote:
As a security guy in my day job I share your concern about the apparent/potential lack of an audit trail and transaction log for this system, I cannot see this being the case in reality given ICANNs recent security issues I would think that systems closest to the root would be heavily logged and tracked.
While I make no claim to being "a security guy" (I have a number of actual "security guys" reporting to me and I know what I don't know), I do have a bit of background in software engineering and software development management. I believe it would be a mistake to make assumptions about code based on nearly decade old third-hand hearsay about how that code has been implemented. The point here is not about having an audit trail and transaction logging, both of which the existing Root Zone Management System has. The point is that the SLE Working Group has demanded metrics on actions that, in some cases, are not logged simply because we didn't see the need. For example, despite numerous arguments against it, the SLE Working Group has demanded we measure "The time the automation system takes from when the last required confirmation is received, until the business process logic progresses the request to the next logic state." From a conceptual point of view, currently in the RZMS code base when the last confirmation is received, a while loop conditional fails (that is, "do we have to wait more?") and the next logic state is entered. To implement this metric we will have to insert code immediately after the termination of the while loop and before the _immediate next executable statement_ which is the update of the logic state. I remain a bit mystified as to what this is supposed to measure, but at this point my job is to see that it is implemented in a way that doesn't impact the security and stability of ICANN's Root Zone Management System (not too hard in that particular case).
This is a troubling point and one that should be taken up by senior ICANN staff/board members if necessary and our co-chairs at the earliest possible juncture.
Feel free. Regards, -drc
Hey David, I am really happy to hear that, I possibly read too much into the statement, if we have good audit trails from a security standpoint then much of my comment is moot. If the issue is logging more than audit trailing then yes the situation is different I would still hope that a timeline that suits all parties can be achieved without compromising any of the SSR. -JG On 14/10/2015, 2:39 a.m., "David Conrad" <david.conrad@icann.org> wrote:
James,
On 10/13/15, 11:41 AM, "James Gannon" <james@cyberinvasion.net> wrote:
As a security guy in my day job I share your concern about the apparent/potential lack of an audit trail and transaction log for this system, I cannot see this being the case in reality given ICANNs recent security issues I would think that systems closest to the root would be heavily logged and tracked.
While I make no claim to being "a security guy" (I have a number of actual "security guys" reporting to me and I know what I don't know), I do have a bit of background in software engineering and software development management. I believe it would be a mistake to make assumptions about code based on nearly decade old third-hand hearsay about how that code has been implemented.
The point here is not about having an audit trail and transaction logging, both of which the existing Root Zone Management System has. The point is that the SLE Working Group has demanded metrics on actions that, in some cases, are not logged simply because we didn't see the need. For example, despite numerous arguments against it, the SLE Working Group has demanded we measure "The time the automation system takes from when the last required confirmation is received, until the business process logic progresses the request to the next logic state." From a conceptual point of view, currently in the RZMS code base when the last confirmation is received, a while loop conditional fails (that is, "do we have to wait more?") and the next logic state is entered. To implement this metric we will have to insert code immediately after the termination of the while loop and before the _immediate next executable statement_ which is the update of the logic state. I remain a bit mystified as to what this is supposed to measure, but at this point my job is to see that it is implemented in a way that doesn't impact the security and stability of ICANN's Root Zone Management System (not too hard in that particular case).
This is a troubling point and one that should be taken up by senior ICANN staff/board members if necessary and our co-chairs at the earliest possible juncture.
Feel free.
Regards, -drc
On Tue, Oct 13, 2015 at 11:31:16AM +0100, Paul M Kane - CWG wrote:
IANA Manager who told me that ICANN does not have the resources to identify their current RZM processes in terms of time taken for the SLE.
I don't think this should be too surprising. If I understand this report correctly, it's that Elise doesn't feel like she has the staff today to put them to work developing new measures to which ICANN will be bound contractually. That's different from measurements that they happen to be able to produce. If I were in her shoes, I'd probably say something similar. This is more or less why some of us argued months ago that changing one word of the current SLE was just asking for additional complication during the transition. Instead, as some of us argued previously, there should be a schedule for the implementation of new the SLE, which should happen after the transition. My opinion remains that staff should not be distracted in the run up to the transition by making any changes to what they measure or how they measure it, _regardless of_ how easy it would be for IANA to make the targets. Re-reading Annex H, I think there's still room to make things happen this way. Best regards, A -- Andrew Sullivan ajs@anvilwalrusden.com
Andrew If I am not mistaken, this is a settled issue. Did this group decide to revise the SLE or did it not? I am under the impression that it did. It undermines the process when people try to use snafus to reinsert a non-consensus position into the outcome. --MM
-----Original Message----- Instead, as some of us argued previously, there should be a schedule for the implementation of new the SLE, which should happen after the transition. My opinion remains that staff should not be distracted in the run up to the transition by making any changes to what they measure or how they measure it, _regardless of_ how easy it would be for IANA to make the targets. Re-reading Annex H, I think there's still room to make things happen this way.
Best regards,
A
-- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
On Tue, Oct 13, 2015 at 03:42:33PM +0000, Mueller, Milton L wrote:
Andrew If I am not mistaken, this is a settled issue. Did this group decide to revise the SLE or did it not? I am under the impression that it did. It undermines the process when people try to use snafus to reinsert a non-consensus position into the outcome.
I'm not trying to insert anything. My reading of Annex H is that the text says that the plan is part of implementation and is running in parallel. Therefore, if it takes a few more months, it's not the end of the world; and we can proceed with the transition without it being completely sorted. What am I misreading? A -- Andrew Sullivan ajs@anvilwalrusden.com
Hi Andrew I think you are misreading. the IANA operator is either: a) accountable to the NTIA via the SLA, or b) accountable to the naming community via the SLE. So delay in finalising the SLE just delays the date of transition from NTIA. The concept of running in parallel scheduled to start on 1st January with a test phase of IANA operations (without NTIA oversight)...... Best Paul Quoting Andrew Sullivan <ajs@anvilwalrusden.com>:
On Tue, Oct 13, 2015 at 03:42:33PM +0000, Mueller, Milton L wrote:
Andrew If I am not mistaken, this is a settled issue. Did this group decide to revise the SLE or did it not? I am under the impression that it did. It undermines the process when people try to use snafus to reinsert a non-consensus position into the outcome.
I'm not trying to insert anything. My reading of Annex H is that the text says that the plan is part of implementation and is running in parallel. Therefore, if it takes a few more months, it's not the end of the world; and we can proceed with the transition without it being completely sorted. What am I misreading?
A
-- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
On Tue, Oct 13, 2015 at 07:11:35PM +0100, Paul M Kane - CWG wrote:
the IANA operator is either: a) accountable to the NTIA via the SLA, or b) accountable to the naming community via the SLE.
Obviously. But changing the counterparty and changing the measurements in question are two completely separate problems. One is contract-only, the other involves changes to how the data is gathered, what the thresholds are, and even what data is gathered.
So delay in finalising the SLE just delays the date of transition from NTIA.
That doesn't follow, and Annex H doesn't say that. There's nothing in Annex H by my reading that requires that the actual data gathered and the levels of service need to change at the same time the parties to the agreement change, unless I am missing something.
From a technical operations standpoint, this is the most stable way to proceed:
1. Get an agreement that the old SLE measurements and levels remain in place but that the new counterparty is ICANN to PTI. 2. Get an agreement that within n months (for some n) the new SLE measurements and levels take effect. [transition can happen after that] 3. Run in parallel the new-SLE and old-SLE measurements under ICANN stewardship. Iterate until working. 4. Switch over to new SLEs by month n. As nearly as I can tell, that approach is completely consistent with what's in Annex H and doesn't block the transition. I was not arguing that the new SLEs are not valuable or shouldn't be pursued. I argued before (and argue now) that the above approach is consistent with the goal, maximises stability, and allows the transition. Contrary to what Milton seems to be implying, I'm not trying to undo any consensus; frankly, this is what I thought people had agreed to since the SLE text wasn't even close to ready in time to submit to the ICG. If people are insistent on something else and IANA can't deliver on the timetable we want (which seems to be the report), what is the fallback plan? For it seems to me that it'd be a needless crisis if these SLEs can't be had as quickly as one would like. Best regards, A -- Andrew Sullivan ajs@anvilwalrusden.com
I personally had assumed that the new SLEs would be implemented before or at the transition, but that doesn't mean that our proposal made that clear. Whether the SLEs are implemented as I thought they would be or as Andrew suggests, we need to first agree on the SLEs. Most of them are not defined yet. The way I understand it is that the testing that the SLE WG proposed to help us define the SLEs. If I am correct on that, then we need to focus on implementing the testing. Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:cwg-stewardship-bounces@icann.org] On Behalf Of Andrew Sullivan Sent: Tuesday, October 13, 2015 3:21 PM To: cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... On Tue, Oct 13, 2015 at 07:11:35PM +0100, Paul M Kane - CWG wrote:
the IANA operator is either: a) accountable to the NTIA via the SLA, or b) accountable to the naming community via the SLE.
Obviously. But changing the counterparty and changing the measurements in question are two completely separate problems. One is contract-only, the other involves changes to how the data is gathered, what the thresholds are, and even what data is gathered.
So delay in finalising the SLE just delays the date of transition from NTIA.
That doesn't follow, and Annex H doesn't say that. There's nothing in Annex H by my reading that requires that the actual data gathered and the levels of service need to change at the same time the parties to the agreement change, unless I am missing something.
From a technical operations standpoint, this is the most stable way to proceed:
1. Get an agreement that the old SLE measurements and levels remain in place but that the new counterparty is ICANN to PTI. 2. Get an agreement that within n months (for some n) the new SLE measurements and levels take effect. [transition can happen after that] 3. Run in parallel the new-SLE and old-SLE measurements under ICANN stewardship. Iterate until working. 4. Switch over to new SLEs by month n. As nearly as I can tell, that approach is completely consistent with what's in Annex H and doesn't block the transition. I was not arguing that the new SLEs are not valuable or shouldn't be pursued. I argued before (and argue now) that the above approach is consistent with the goal, maximises stability, and allows the transition. Contrary to what Milton seems to be implying, I'm not trying to undo any consensus; frankly, this is what I thought people had agreed to since the SLE text wasn't even close to ready in time to submit to the ICG. If people are insistent on something else and IANA can't deliver on the timetable we want (which seems to be the report), what is the fallback plan? For it seems to me that it'd be a needless crisis if these SLEs can't be had as quickly as one would like. Best regards, A -- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
On Tue, Oct 13, 2015 at 8:57 PM, Gomes, Chuck <cgomes@verisign.com> wrote:
I personally had assumed that the new SLEs would be implemented before or at the transition,
Perhaps we should ask the IETF of their experience when they tried updating their SLA (which was somewhat the outcome of the IANAPLAN) during the transition.
but that doesn't mean that our proposal made that clear. Whether the SLEs are implemented as I thought they would be or as Andrew suggests, we need to first agree on the SLEs.
I had thought Paul's team passed the level of agreeing on the SLE requirements with IANA no?
Most of them are not defined yet. The way I understand it is that the testing that the SLE WG proposed to help us define the SLEs. If I am correct on that, then we need to focus on implementing the testing.
If I may, what does testing imply? does testing in this case mean working on the live system or creating a dummy setup or doing similar hypothetical test case scenario like it was done within the CCWG? Cheers!
Chuck
-----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto: cwg-stewardship-bounces@icann.org] On Behalf Of Andrew Sullivan Sent: Tuesday, October 13, 2015 3:21 PM To: cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting......
On Tue, Oct 13, 2015 at 07:11:35PM +0100, Paul M Kane - CWG wrote:
the IANA operator is either: a) accountable to the NTIA via the SLA, or b) accountable to the naming community via the SLE.
Obviously. But changing the counterparty and changing the measurements in question are two completely separate problems. One is contract-only, the other involves changes to how the data is gathered, what the thresholds are, and even what data is gathered.
So delay in finalising the SLE just delays the date of transition from NTIA.
That doesn't follow, and Annex H doesn't say that. There's nothing in Annex H by my reading that requires that the actual data gathered and the levels of service need to change at the same time the parties to the agreement change, unless I am missing something.
From a technical operations standpoint, this is the most stable way to proceed:
1. Get an agreement that the old SLE measurements and levels remain in place but that the new counterparty is ICANN to PTI.
2. Get an agreement that within n months (for some n) the new SLE measurements and levels take effect.
[transition can happen after that]
3. Run in parallel the new-SLE and old-SLE measurements under ICANN stewardship. Iterate until working.
4. Switch over to new SLEs by month n.
As nearly as I can tell, that approach is completely consistent with what's in Annex H and doesn't block the transition. I was not arguing that the new SLEs are not valuable or shouldn't be pursued. I argued before (and argue now) that the above approach is consistent with the goal, maximises stability, and allows the transition. Contrary to what Milton seems to be implying, I'm not trying to undo any consensus; frankly, this is what I thought people had agreed to since the SLE text wasn't even close to ready in time to submit to the ICG.
If people are insistent on something else and IANA can't deliver on the timetable we want (which seems to be the report), what is the fallback plan? For it seems to me that it'd be a needless crisis if these SLEs can't be had as quickly as one would like.
Best regards,
A
-- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
-- ------------------------------------------------------------------------ *Seun Ojedeji,Federal University Oye-Ekitiweb: http://www.fuoye.edu.ng <http://www.fuoye.edu.ng> Mobile: +2348035233535**alt email: <http://goog_1872880453>seun.ojedeji@fuoye.edu.ng <seun.ojedeji@fuoye.edu.ng>* Bringing another down does not take you up - think about your action!
On Tue, Oct 13, 2015 at 09:45:38PM +0100, Seun Ojedeji wrote:
Perhaps we should ask the IETF of their experience when they tried updating their SLA (which was somewhat the outcome of the IANAPLAN) during the transition.
Nothing, yet. We haven't completed. Our understanding is that it is going to be possible, but not yet because it gives people the impression we'd be going ahead without the rest of the transition. Nobody has suggested there's a problem, just that there's a problem now. I'm not sure it's exactly parallel, though, because in the IETF case there are two different agreements: the one between the IETF and ICANN and a different one between ICANN and NTIA. In the ICANN case, that's not actually a problem because the PTI doesn't exist at all yet. A -- Andrew Sullivan ajs@anvilwalrusden.com
Sent from my Asus Zenfone2 Kindly excuse brevity and typos. On 13 Oct 2015 22:10, "Andrew Sullivan" <ajs@anvilwalrusden.com> wrote:
On Tue, Oct 13, 2015 at 09:45:38PM +0100, Seun Ojedeji wrote:
Perhaps we should ask the IETF of their experience when they tried
updating
their SLA (which was somewhat the outcome of the IANAPLAN) during the transition.
Nothing, yet. We haven't completed. Our understanding is that it is going to be possible, but not yet because it gives people the impression we'd be going ahead without the rest of the transition. Nobody has suggested there's a problem, just that there's a problem now.
SO: This is actually what I wanted an IETFer to highlight and I hope it will also guide CWG as we determine what we want ICANN to commit to.
I'm not sure it's exactly parallel, though, because in the IETF case there are two different agreements: the one between the IETF and ICANN and a different one between ICANN and NTIA.
SO: Sure that's the case with numbers as well. What I just wanted to point out is that no formal commitment should be expected before transition
In the ICANN case, that's
not actually a problem because the PTI doesn't exist at all yet.
SO: Yeah that's right, so noting to commit on since the IFO is yet to emerge. I guess this would have implications on all the OCs SLA/SLE since the new IFO will operate for the triplet and ICANN would still need to do a subcontracting terms for numbers/IETF Cheers!
A
-- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
Seun, It is probably best if Paul answers your questions but I take a stab at them below, understanding that I was not on the SLE WG. Chuck From: Seun Ojedeji [mailto:seun.ojedeji@gmail.com] Sent: Tuesday, October 13, 2015 4:46 PM To: Gomes, Chuck Cc: Andrew Sullivan; cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... On Tue, Oct 13, 2015 at 8:57 PM, Gomes, Chuck <cgomes@verisign.com<mailto:cgomes@verisign.com>> wrote: I personally had assumed that the new SLEs would be implemented before or at the transition, Perhaps we should ask the IETF of their experience when they tried updating their SLA (which was somewhat the outcome of the IANAPLAN) during the transition. but that doesn't mean that our proposal made that clear. Whether the SLEs are implemented as I thought they would be or as Andrew suggests, we need to first agree on the SLEs. I had thought Paul's team passed the level of agreeing on the SLE requirements with IANA no? [Chuck Gomes] No. Most of them were not defined. Most of them are not defined yet. The way I understand it is that the testing that the SLE WG proposed to help us define the SLEs. If I am correct on that, then we need to focus on implementing the testing. If I may, what does testing imply? does testing in this case mean working on the live system or creating a dummy setup or doing similar hypothetical test case scenario like it was done within the CCWG? [Chuck Gomes] I believe it meant working on a parallel system. Cheers! Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org<mailto:cwg-stewardship-bounces@icann.org> [mailto:cwg-stewardship-bounces@icann.org<mailto:cwg-stewardship-bounces@icann.org>] On Behalf Of Andrew Sullivan Sent: Tuesday, October 13, 2015 3:21 PM To: cwg-stewardship@icann.org<mailto:cwg-stewardship@icann.org> Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... On Tue, Oct 13, 2015 at 07:11:35PM +0100, Paul M Kane - CWG wrote:
the IANA operator is either: a) accountable to the NTIA via the SLA, or b) accountable to the naming community via the SLE.
Obviously. But changing the counterparty and changing the measurements in question are two completely separate problems. One is contract-only, the other involves changes to how the data is gathered, what the thresholds are, and even what data is gathered.
So delay in finalising the SLE just delays the date of transition from NTIA.
That doesn't follow, and Annex H doesn't say that. There's nothing in Annex H by my reading that requires that the actual data gathered and the levels of service need to change at the same time the parties to the agreement change, unless I am missing something. From a technical operations standpoint, this is the most stable way to proceed: 1. Get an agreement that the old SLE measurements and levels remain in place but that the new counterparty is ICANN to PTI. 2. Get an agreement that within n months (for some n) the new SLE measurements and levels take effect. [transition can happen after that] 3. Run in parallel the new-SLE and old-SLE measurements under ICANN stewardship. Iterate until working. 4. Switch over to new SLEs by month n. As nearly as I can tell, that approach is completely consistent with what's in Annex H and doesn't block the transition. I was not arguing that the new SLEs are not valuable or shouldn't be pursued. I argued before (and argue now) that the above approach is consistent with the goal, maximises stability, and allows the transition. Contrary to what Milton seems to be implying, I'm not trying to undo any consensus; frankly, this is what I thought people had agreed to since the SLE text wasn't even close to ready in time to submit to the ICG. If people are insistent on something else and IANA can't deliver on the timetable we want (which seems to be the report), what is the fallback plan? For it seems to me that it'd be a needless crisis if these SLEs can't be had as quickly as one would like. Best regards, A -- Andrew Sullivan ajs@anvilwalrusden.com<mailto:ajs@anvilwalrusden.com> _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org<mailto:CWG-Stewardship@icann.org> https://mm.icann.org/mailman/listinfo/cwg-stewardship _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org<mailto:CWG-Stewardship@icann.org> https://mm.icann.org/mailman/listinfo/cwg-stewardship -- ------------------------------------------------------------------------ Seun Ojedeji, Federal University Oye-Ekiti web: http://www.fuoye.edu.ng Mobile: +2348035233535 alt email: <http://goog_1872880453> seun.ojedeji@fuoye.edu.ng<mailto:seun.ojedeji@fuoye.edu.ng> Bringing another down does not take you up - think about your action!
Hello Chuck, Kindly find inline Sent from my Asus Zenfone2 Kindly excuse brevity and typos. On 13 Oct 2015 22:12, "Gomes, Chuck" <cgomes@verisign.com> wrote:
Seun,
It is probably best if Paul answers your questions but I take a stab at
them below, understanding that I was not on the SLE WG.
Chuck
From: Seun Ojedeji [mailto:seun.ojedeji@gmail.com] Sent: Tuesday, October 13, 2015 4:46 PM To: Gomes, Chuck Cc: Andrew Sullivan; cwg-stewardship@icann.org
Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE
Accountability reporting......
On Tue, Oct 13, 2015 at 8:57 PM, Gomes, Chuck <cgomes@verisign.com> wrote:
I personally had assumed that the new SLEs would be implemented before or
at the transition,
Perhaps we should ask the IETF of their experience when they tried
updating their SLA (which was somewhat the outcome of the IANAPLAN) during the transition.
but that doesn't mean that our proposal made that clear. Whether the
SLEs are implemented as I thought they would be or as Andrew suggests, we need to first agree on the SLEs.
I had thought Paul's team passed the level of agreeing on the SLE
requirements with IANA no?
[Chuck Gomes] No. Most of them were not defined.
SO: Okay maybe I read too much meaning to Paul's mail in august: http://mm.icann.org/pipermail/cwg-stewardship/2015-August/004216.html I extract relevant parts below: "I am delighted to attach the agreed Service Level Expectation document, containing the specifics of what ICANN/IANA has agreed to monitor and report..........ICANN/IANA will now plan the scope of work to implement this SLE, operating the service in a test phase/parallel mode to start the necessary real-world data capture."
Most of them are not defined yet. The way I understand it is that the
testing that the SLE WG proposed to help us define the SLEs. If I am correct on that, then we need to focus on implementing the testing.
If I may, what does testing imply? does testing in this case mean working
on the live system or creating a dummy setup or doing similar hypothetical test case scenario like it was done within the CCWG?
[Chuck Gomes] I believe it meant working on a parallel system.
SO: Okay that would be major activity then which I don't think can be compared to the SLA development process/requirement for numbers and IETF. Regards
Cheers!
Chuck
-----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:
cwg-stewardship-bounces@icann.org] On Behalf Of Andrew Sullivan
Sent: Tuesday, October 13, 2015 3:21 PM To: cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting......
On Tue, Oct 13, 2015 at 07:11:35PM +0100, Paul M Kane - CWG wrote:
the IANA operator is either: a) accountable to the NTIA via the SLA, or b) accountable to the naming community via the SLE.
Obviously. But changing the counterparty and changing the measurements in question are two completely separate problems. One is contract-only, the other involves changes to how the data is gathered, what the thresholds are, and even what data is gathered.
So delay in finalising the SLE just delays the date of transition from NTIA.
That doesn't follow, and Annex H doesn't say that. There's nothing in Annex H by my reading that requires that the actual data gathered and the levels of service need to change at the same time the parties to the agreement change, unless I am missing something.
From a technical operations standpoint, this is the most stable way to proceed:
1. Get an agreement that the old SLE measurements and levels remain in place but that the new counterparty is ICANN to PTI.
2. Get an agreement that within n months (for some n) the new SLE measurements and levels take effect.
[transition can happen after that]
3. Run in parallel the new-SLE and old-SLE measurements under ICANN stewardship. Iterate until working.
4. Switch over to new SLEs by month n.
As nearly as I can tell, that approach is completely consistent with what's in Annex H and doesn't block the transition. I was not arguing that the new SLEs are not valuable or shouldn't be pursued. I argued before (and argue now) that the above approach is consistent with the goal, maximises stability, and allows the transition. Contrary to what Milton seems to be implying, I'm not trying to undo any consensus; frankly, this is what I thought people had agreed to since the SLE text wasn't even close to ready in time to submit to the ICG.
If people are insistent on something else and IANA can't deliver on the timetable we want (which seems to be the report), what is the fallback plan? For it seems to me that it'd be a needless crisis if these SLEs can't be had as quickly as one would like.
Best regards,
A
-- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
--
------------------------------------------------------------------------
Seun Ojedeji, Federal University Oye-Ekiti web: http://www.fuoye.edu.ng Mobile: +2348035233535 alt email: seun.ojedeji@fuoye.edu.ng
Bringing another down does not take you up - think about your action!
Andrew, Are you suggesting that the transition happen without revised SLEs? Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:cwg-stewardship-bounces@icann.org] On Behalf Of Andrew Sullivan Sent: Tuesday, October 13, 2015 12:04 PM To: cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... On Tue, Oct 13, 2015 at 03:42:33PM +0000, Mueller, Milton L wrote:
Andrew If I am not mistaken, this is a settled issue. Did this group decide to revise the SLE or did it not? I am under the impression that it did. It undermines the process when people try to use snafus to reinsert a non-consensus position into the outcome.
I'm not trying to insert anything. My reading of Annex H is that the text says that the plan is part of implementation and is running in parallel. Therefore, if it takes a few more months, it's not the end of the world; and we can proceed with the transition without it being completely sorted. What am I misreading? A -- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
On Tue, Oct 13, 2015 at 06:26:00PM +0000, Gomes, Chuck wrote:
Andrew,
Are you suggesting that the transition happen without revised SLEs?
It seems to me that, if the SLEs are left exactly the same as they are with only a change in who monitors them, it's not a change that anyone at IANA could possibly buck. If instead at the same time the transition happens IANA also has to make different commitments as to what it will monitor and what the targets will be, they have a reason to be cautious. That's all I was saying. From the response, it sounds like I'm in the rough, but it seems to me that if IANA has a problem achieving these new requirements then we do too. A -- Andrew Sullivan ajs@anvilwalrusden.com
Thanks for the clarification Andrew. My understanding is that the purpose of testing was to validate that proposed SLEs are reasonably achievable. I suppose that could occur after the transition but what is to prevent it from taking a very long time or not happening at all. Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:cwg-stewardship-bounces@icann.org] On Behalf Of Andrew Sullivan Sent: Tuesday, October 13, 2015 2:40 PM To: cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... On Tue, Oct 13, 2015 at 06:26:00PM +0000, Gomes, Chuck wrote:
Andrew,
Are you suggesting that the transition happen without revised SLEs?
It seems to me that, if the SLEs are left exactly the same as they are with only a change in who monitors them, it's not a change that anyone at IANA could possibly buck. If instead at the same time the transition happens IANA also has to make different commitments as to what it will monitor and what the targets will be, they have a reason to be cautious. That's all I was saying. From the response, it sounds like I'm in the rough, but it seems to me that if IANA has a problem achieving these new requirements then we do too. A -- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
On Tue, Oct 13, 2015 at 06:45:16PM +0000, Gomes, Chuck wrote:
Thanks for the clarification Andrew.
My understanding is that the purpose of testing was to validate that proposed SLEs are reasonably achievable. I suppose that could occur after the transition but what is to prevent it from taking a very long time or not happening at all.
You have an oversight function and it's written into the contract. If the contractual terms aren't followed, then the contract is terminated by the oversight body. This doesn't seem to me like the sort of thing that needs some other kind of lever. A -- Andrew Sullivan ajs@anvilwalrusden.com
I agree with that Andrew once we have the SLEs defined and in the contract. But they first need to be defined and it seems to me that they would need to be defined before the contract could be developed and executed. Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:cwg-stewardship-bounces@icann.org] On Behalf Of Andrew Sullivan Sent: Tuesday, October 13, 2015 3:04 PM To: cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... On Tue, Oct 13, 2015 at 06:45:16PM +0000, Gomes, Chuck wrote:
Thanks for the clarification Andrew.
My understanding is that the purpose of testing was to validate that proposed SLEs are reasonably achievable. I suppose that could occur after the transition but what is to prevent it from taking a very long time or not happening at all.
You have an oversight function and it's written into the contract. If the contractual terms aren't followed, then the contract is terminated by the oversight body. This doesn't seem to me like the sort of thing that needs some other kind of lever. A -- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
-----Original Message-----
It seems to me that, if the SLEs are left exactly the same as they are with only a change in who monitors them, it's not a change that anyone at IANA could possibly buck.
If instead at the same time the transition happens IANA also has to make different commitments as to what it will monitor and what the targets will be, they have a reason to be cautious.
But this is what you argued, unsuccessfully, during the proposal development process. (Don't change the targets) It is not what we chose to do. Had we chosen to do that, we would not have created the WG that Paul led. But we did.
That's all I was saying. From the response, it sounds like I'm in the rough
Yep.
Let me answer my own question after reading Andrew's earlier message. I believe he is suggesting that the revised SLEs be implemented after the transition. My question then is this: how can we be ensured that they will be implemented? It seems to me that if the revised SLEs need to be implemented before the transition happens, that provides some leverage that will disappear after the transition occurs. Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:cwg-stewardship-bounces@icann.org] On Behalf Of Gomes, Chuck Sent: Tuesday, October 13, 2015 2:26 PM To: Andrew Sullivan; cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... Andrew, Are you suggesting that the transition happen without revised SLEs? Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:cwg-stewardship-bounces@icann.org] On Behalf Of Andrew Sullivan Sent: Tuesday, October 13, 2015 12:04 PM To: cwg-stewardship@icann.org Subject: Re: [CWG-Stewardship] SLE update - ICANN seeks to delay SLE Accountability reporting...... On Tue, Oct 13, 2015 at 03:42:33PM +0000, Mueller, Milton L wrote:
Andrew If I am not mistaken, this is a settled issue. Did this group decide to revise the SLE or did it not? I am under the impression that it did. It undermines the process when people try to use snafus to reinsert a non-consensus position into the outcome.
I'm not trying to insert anything. My reading of Annex H is that the text says that the plan is part of implementation and is running in parallel. Therefore, if it takes a few more months, it's not the end of the world; and we can proceed with the transition without it being completely sorted. What am I misreading? A -- Andrew Sullivan ajs@anvilwalrusden.com _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship _______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
Paul, On 10/13/15, 11:31 AM, "dt1-bounces@icann.org on behalf of Paul M Kane - CWG" <dt1-bounces@icann.org on behalf of paul.kane-cwg@icb.co.uk> wrote:
The test phase is scheduled to start 1st January 2016.
I am unaware of any commitment by my development team to a "1st January 2016" start date for SLE data collection. If you believe ICANN staff have made such a commitment, I'd very much appreciate a pointer to the recording, transcript, email, or other documentary evidence. My recollection was that ICANN staff went on record to repeatedly state that we can NOT make any commitment about when we would deliver the code until we had a chance to actually scope the development necessary. We have done a preliminary scoping of the work involved in revising the existing Root Zone Management System. Based on that scoping, we feel confident deploying the revised code by the beginning of 2Q2016. This provides for up to 6 months of data collection for the establishment of the actual contractual SLAs. If you do not believe this sufficient time, perhaps you could explain why you believe additional time will be necessary?
I attended a meeting in Brussels on Wednesday and met with Elise Gerish, IANA Manager who told me that ICANN does not have the resources to identify their current RZM processes in terms of time taken for the SLE.
Not being party to that particular meeting, I suspect confusion here. ICANN does, of course, have the resources to identify Root Zone Management System processes for the purposes of establishing the SLEs. As stated during many conversations with the SLE Working Group and/or sub-team, the granularity to which the SLE Working Group has required metrics to be collected goes beyond the detail that is logged today. As such we have to insert new code into the existing RZMS code base and make sure we don¹t break anything when we do so. We cannot, of course, comment on current Root Zone Maintainer (RZM), that is, Verisign processes -- I'll presume this to be a typo.
Consequently, they are proposing to delay the testing of the naming community's reporting requirements until after March/April 2016.
As I stated repeatedly on the SLE Working Group calls, there are a number of changes that are required to the RZMS software in order to support operation without NTIA in a way that can be verified to not impact the security and stability of changes to the root zone. Implementation of the various code points within the RZMS software to support the SLEs is one component of those changes. There are also changes to support the removal of NTIA from their authorization role as well as changes to the interfaces between the front end and the back end of the RZMS software to facilitate the parallel testing necessary to ensure we are not breaking anything. This is not a "delay" since we did NOT commit to a timeframe in which the development would be done (other than before the transition).
This was news to me and I assume/hope it would be unreasonable to deduce that ICANN are delaying to potentially seize control of the IANA RZM (from NTIA's oversight) without having to adhere to the operational SLE standards agreed with the community.
It would indeed be unreasonable to make such a deduction, particularly given the repeated assertions from members of the SLE Working Group "no SLEs, no transition." In fact I might even call such a deduction offensive given the efforts ICANN staff have expended on this particular topic.
A while back, I spoke with the contractor who wrote the IANA RZM system and he said most of the data is already captured, it is just that ICANN wanted the data extraction tools disabled.
Could you clarify to whom you have spoken? This is both simply incorrect and suggests malfeasance on the part of ICANN staff and I would like to understand from that individual directly why they believe this, particularly given the contractor who was part of the team that initially did the work with us (nearly a decade ago) was a she, not a he.
Having an effective transaction log of each process step is a fundamental part of Information Security Management (ISO/IEC 27001)- and is basic good practice - and it should not be a complicated task to generate the SLE reports from the transaction logs.
As mentioned previously, the granularity to which the SLE Working Group has required metrics to be collected goes beyond the detail that is logged within our transactions logs. For example, the SLEs require the RZMS code to keep track of "Time for authorization contacts to be asked to approve change request after completing previous process phase". Since there is literally nothing that occurs between the point in time at which the previous process phase has completed and sending the approval request, we're going to have to insert code and new logging statements to allow us to meet the SLE Working Group requirements even though the time measured for this SLE will likely be in the nanoseconds since it does not involve any human interaction nor branching logic. There are a number of these types of metrics that the SLE Working Group decided were necessary and as a result, there is a number of code modifications that need to be made and tested before we can put the code into production.
It is essential that at the time of transition we have a tried/tested/proven SLE standard in place so ICANN is accountable to our naming community.
Indeed, but that is not what the SLE Working Group has demanded. The SLE Working Group demanded a new set of metrics from those that NTIA had defined and which had been tried/tested/proven. In keeping with that demand, ICANN staff has now undertaken to modify the RZMS code. The inclusion of new code into the RZMS software requires us to do extensive testing. It would be quite unfortunate if due to a rush to deploy code that measures ICANN's performance, we break root management.
Can I encourage every member of the CWG (and ICG) to invite ICANN to engage the necessary resources to enable the SLE accountability reporting mechanisms from the 1st January 2016 on their test (post NTIA) RZM platform
To quote Fredrick Brooks: "The bearing of a child takes nine months, no matter how many women are assigned." -- Fredrick Brooks, Mythical Man-Month, pg. 17 Pragmatically speaking, the issue is that in order to implement the SLEs, a fairly in-depth knowledge of the underlying RZMS code base is required. If we were to attempt to throw new additional resources at implementing the SLEs within the timeframe you assert (approximately 9 work _weeks_ from today), it would require time to find those resources and bring them up to speed on the code base. Bringing those resources up to speed would, of course, impact the productivity of those who already know the code base, resulting in delays. Further, since people new to the code base would likely make mistakes, additional testing and debugging would necessitate even more delays.
...... as they are doing for the other IANA customers (numbers and protocols)?
I'm sorry, what? As far as I understand, the SLAs for the numbers community are still being discussed with people at the RIRs and I'm unaware of any discussions relating to the routine updating of the SLAs with the IETF (which last I heard for this go around, didn't involve any code development and certainly not code that is critical to the management of the root zone of the DNS). Why would you assert we are doing something for either of those communities by 1 January 2016? Regards, -drc
Ah okay I guess I should have gone through the entire thread. David's detailed response seem to clarify a lot. That said, perhaps it will be good to establish a specific staff contact for different issues just so everyone is in sync as I sense some communication issues in David's mail. Thanks Sent from my Asus Zenfone2 Kindly excuse brevity and typos. On 14 Oct 2015 02:11, "David Conrad" <david.conrad@icann.org> wrote:
Paul,
On 10/13/15, 11:31 AM, "dt1-bounces@icann.org on behalf of Paul M Kane - CWG" <dt1-bounces@icann.org on behalf of paul.kane-cwg@icb.co.uk> wrote:
The test phase is scheduled to start 1st January 2016.
I am unaware of any commitment by my development team to a "1st January 2016" start date for SLE data collection. If you believe ICANN staff have made such a commitment, I'd very much appreciate a pointer to the recording, transcript, email, or other documentary evidence. My recollection was that ICANN staff went on record to repeatedly state that we can NOT make any commitment about when we would deliver the code until we had a chance to actually scope the development necessary.
We have done a preliminary scoping of the work involved in revising the existing Root Zone Management System. Based on that scoping, we feel confident deploying the revised code by the beginning of 2Q2016. This provides for up to 6 months of data collection for the establishment of the actual contractual SLAs. If you do not believe this sufficient time, perhaps you could explain why you believe additional time will be necessary?
I attended a meeting in Brussels on Wednesday and met with Elise Gerish, IANA Manager who told me that ICANN does not have the resources to identify their current RZM processes in terms of time taken for the SLE.
Not being party to that particular meeting, I suspect confusion here. ICANN does, of course, have the resources to identify Root Zone Management System processes for the purposes of establishing the SLEs. As stated during many conversations with the SLE Working Group and/or sub-team, the granularity to which the SLE Working Group has required metrics to be collected goes beyond the detail that is logged today. As such we have to insert new code into the existing RZMS code base and make sure we don¹t break anything when we do so.
We cannot, of course, comment on current Root Zone Maintainer (RZM), that is, Verisign processes -- I'll presume this to be a typo.
Consequently, they are proposing to delay the testing of the naming community's reporting requirements until after March/April 2016.
As I stated repeatedly on the SLE Working Group calls, there are a number of changes that are required to the RZMS software in order to support operation without NTIA in a way that can be verified to not impact the security and stability of changes to the root zone. Implementation of the various code points within the RZMS software to support the SLEs is one component of those changes. There are also changes to support the removal of NTIA from their authorization role as well as changes to the interfaces between the front end and the back end of the RZMS software to facilitate the parallel testing necessary to ensure we are not breaking anything. This is not a "delay" since we did NOT commit to a timeframe in which the development would be done (other than before the transition).
This was news to me and I assume/hope it would be unreasonable to deduce that ICANN are delaying to potentially seize control of the IANA RZM (from NTIA's oversight) without having to adhere to the operational SLE standards agreed with the community.
It would indeed be unreasonable to make such a deduction, particularly given the repeated assertions from members of the SLE Working Group "no SLEs, no transition." In fact I might even call such a deduction offensive given the efforts ICANN staff have expended on this particular topic.
A while back, I spoke with the contractor who wrote the IANA RZM system and he said most of the data is already captured, it is just that ICANN wanted the data extraction tools disabled.
Could you clarify to whom you have spoken? This is both simply incorrect and suggests malfeasance on the part of ICANN staff and I would like to understand from that individual directly why they believe this, particularly given the contractor who was part of the team that initially did the work with us (nearly a decade ago) was a she, not a he.
Having an effective transaction log of each process step is a fundamental part of Information Security Management (ISO/IEC 27001)- and is basic good practice - and it should not be a complicated task to generate the SLE reports from the transaction logs.
As mentioned previously, the granularity to which the SLE Working Group has required metrics to be collected goes beyond the detail that is logged within our transactions logs. For example, the SLEs require the RZMS code to keep track of "Time for authorization contacts to be asked to approve change request after completing previous process phase". Since there is literally nothing that occurs between the point in time at which the previous process phase has completed and sending the approval request, we're going to have to insert code and new logging statements to allow us to meet the SLE Working Group requirements even though the time measured for this SLE will likely be in the nanoseconds since it does not involve any human interaction nor branching logic. There are a number of these types of metrics that the SLE Working Group decided were necessary and as a result, there is a number of code modifications that need to be made and tested before we can put the code into production.
It is essential that at the time of transition we have a tried/tested/proven SLE standard in place so ICANN is accountable to our naming community.
Indeed, but that is not what the SLE Working Group has demanded. The SLE Working Group demanded a new set of metrics from those that NTIA had defined and which had been tried/tested/proven. In keeping with that demand, ICANN staff has now undertaken to modify the RZMS code. The inclusion of new code into the RZMS software requires us to do extensive testing. It would be quite unfortunate if due to a rush to deploy code that measures ICANN's performance, we break root management.
Can I encourage every member of the CWG (and ICG) to invite ICANN to engage the necessary resources to enable the SLE accountability reporting mechanisms from the 1st January 2016 on their test (post NTIA) RZM platform
To quote Fredrick Brooks:
"The bearing of a child takes nine months, no matter how many women are assigned." -- Fredrick Brooks, Mythical Man-Month, pg. 17
Pragmatically speaking, the issue is that in order to implement the SLEs, a fairly in-depth knowledge of the underlying RZMS code base is required. If we were to attempt to throw new additional resources at implementing the SLEs within the timeframe you assert (approximately 9 work _weeks_ from today), it would require time to find those resources and bring them up to speed on the code base. Bringing those resources up to speed would, of course, impact the productivity of those who already know the code base, resulting in delays. Further, since people new to the code base would likely make mistakes, additional testing and debugging would necessitate even more delays.
...... as they are doing for the other IANA customers (numbers and protocols)?
I'm sorry, what? As far as I understand, the SLAs for the numbers community are still being discussed with people at the RIRs and I'm unaware of any discussions relating to the routine updating of the SLAs with the IETF (which last I heard for this go around, didn't involve any code development and certainly not code that is critical to the management of the root zone of the DNS). Why would you assert we are doing something for either of those communities by 1 January 2016?
Regards, -drc
_______________________________________________ CWG-Stewardship mailing list CWG-Stewardship@icann.org https://mm.icann.org/mailman/listinfo/cwg-stewardship
I want to thank David for the very thorough response that I think provides some essential clarity on this discussion. We have been debating whether the SLEs should be implemented before or after the transition. The following statement from David seems to clarify that in my mind: ". . . we did NOT commit to a timeframe in which the development would be done (other than before the transition)." (David - please correct me if I misinterpreted this.) It seems to me that the gap in understanding relates to the January target date versus a March/April date. I also was not a party to any of the discussions on this, but it appears to me that a March/April date may be fine because I think we are now looking at a September 2016 transition date. Am I missing something here? On a minor point, I am not sure that RZM refers to Root Zone Maintainer. RZM is sometimes used to refer to Root Zone Manager or Root Zone Management. Chuck -----Original Message----- From: cwg-stewardship-bounces@icann.org [mailto:cwg-stewardship-bounces@icann.org] On Behalf Of David Conrad Sent: Tuesday, October 13, 2015 9:10 PM To: Paul M Kane - CWG; cwg-stewardship@icann.org Cc: dt1@icann.org Subject: Re: [CWG-Stewardship] [SLE Team] SLE update - ICANN seeks to delay SLE Accountability reporting...... Paul, On 10/13/15, 11:31 AM, "dt1-bounces@icann.org on behalf of Paul M Kane - CWG" <dt1-bounces@icann.org on behalf of paul.kane-cwg@icb.co.uk> wrote:
The test phase is scheduled to start 1st January 2016.
I am unaware of any commitment by my development team to a "1st January 2016" start date for SLE data collection. If you believe ICANN staff have made such a commitment, I'd very much appreciate a pointer to the recording, transcript, email, or other documentary evidence. My recollection was that ICANN staff went on record to repeatedly state that we can NOT make any commitment about when we would deliver the code until we had a chance to actually scope the development necessary. We have done a preliminary scoping of the work involved in revising the existing Root Zone Management System. Based on that scoping, we feel confident deploying the revised code by the beginning of 2Q2016. This provides for up to 6 months of data collection for the establishment of the actual contractual SLAs. If you do not believe this sufficient time, perhaps you could explain why you believe additional time will be necessary?
I attended a meeting in Brussels on Wednesday and met with Elise Gerish, IANA Manager who told me that ICANN does not have the resources to identify their current RZM processes in terms of time taken for the SLE.
Not being party to that particular meeting, I suspect confusion here. ICANN does, of course, have the resources to identify Root Zone Management System processes for the purposes of establishing the SLEs. As stated during many conversations with the SLE Working Group and/or sub-team, the granularity to which the SLE Working Group has required metrics to be collected goes beyond the detail that is logged today. As such we have to insert new code into the existing RZMS code base and make sure we don¹t break anything when we do so. We cannot, of course, comment on current Root Zone Maintainer (RZM), that is, Verisign processes -- I'll presume this to be a typo.
Consequently, they are proposing to delay the testing of the naming community's reporting requirements until after March/April 2016.
As I stated repeatedly on the SLE Working Group calls, there are a number of changes that are required to the RZMS software in order to support operation without NTIA in a way that can be verified to not impact the security and stability of changes to the root zone. Implementation of the various code points within the RZMS software to support the SLEs is one component of those changes. There are also changes to support the removal of NTIA from their authorization role as well as changes to the interfaces between the front end and the back end of the RZMS software to facilitate the parallel testing necessary to ensure we are not breaking anything. This is not a "delay" since we did NOT commit to a timeframe in which the development would be done (other than before the transition).
This was news to me and I assume/hope it would be unreasonable to deduce that ICANN are delaying to potentially seize control of the IANA RZM (from NTIA's oversight) without having to adhere to the operational SLE standards agreed with the community.
It would indeed be unreasonable to make such a deduction, particularly given the repeated assertions from members of the SLE Working Group "no SLEs, no transition." In fact I might even call such a deduction offensive given the efforts ICANN staff have expended on this particular topic.
A while back, I spoke with the contractor who wrote the IANA RZM system and he said most of the data is already captured, it is just that ICANN wanted the data extraction tools disabled.
Could you clarify to whom you have spoken? This is both simply incorrect and suggests malfeasance on the part of ICANN staff and I would like to understand from that individual directly why they believe this, particularly given the contractor who was part of the team that initially did the work with us (nearly a decade ago) was a she, not a he.
Having an effective transaction log of each process step is a fundamental part of Information Security Management (ISO/IEC 27001)- and is basic good practice - and it should not be a complicated task to generate the SLE reports from the transaction logs.
As mentioned previously, the granularity to which the SLE Working Group has required metrics to be collected goes beyond the detail that is logged within our transactions logs. For example, the SLEs require the RZMS code to keep track of "Time for authorization contacts to be asked to approve change request after completing previous process phase". Since there is literally nothing that occurs between the point in time at which the previous process phase has completed and sending the approval request, we're going to have to insert code and new logging statements to allow us to meet the SLE Working Group requirements even though the time measured for this SLE will likely be in the nanoseconds since it does not involve any human interaction nor branching logic. There are a number of these types of metrics that the SLE Working Group decided were necessary and as a result, there is a number of code modifications that need to be made and tested before we can put the code into production.
It is essential that at the time of transition we have a tried/tested/proven SLE standard in place so ICANN is accountable to our naming community.
Indeed, but that is not what the SLE Working Group has demanded. The SLE Working Group demanded a new set of metrics from those that NTIA had defined and which had been tried/tested/proven. In keeping with that demand, ICANN staff has now undertaken to modify the RZMS code. The inclusion of new code into the RZMS software requires us to do extensive testing. It would be quite unfortunate if due to a rush to deploy code that measures ICANN's performance, we break root management.
Can I encourage every member of the CWG (and ICG) to invite ICANN to engage the necessary resources to enable the SLE accountability reporting mechanisms from the 1st January 2016 on their test (post NTIA) RZM platform
To quote Fredrick Brooks: "The bearing of a child takes nine months, no matter how many women are assigned." -- Fredrick Brooks, Mythical Man-Month, pg. 17 Pragmatically speaking, the issue is that in order to implement the SLEs, a fairly in-depth knowledge of the underlying RZMS code base is required. If we were to attempt to throw new additional resources at implementing the SLEs within the timeframe you assert (approximately 9 work _weeks_ from today), it would require time to find those resources and bring them up to speed on the code base. Bringing those resources up to speed would, of course, impact the productivity of those who already know the code base, resulting in delays. Further, since people new to the code base would likely make mistakes, additional testing and debugging would necessitate even more delays.
...... as they are doing for the other IANA customers (numbers and protocols)?
I'm sorry, what? As far as I understand, the SLAs for the numbers community are still being discussed with people at the RIRs and I'm unaware of any discussions relating to the routine updating of the SLAs with the IETF (which last I heard for this go around, didn't involve any code development and certainly not code that is critical to the management of the root zone of the DNS). Why would you assert we are doing something for either of those communities by 1 January 2016? Regards, -drc
David Quoting David Conrad <david.conrad@icann.org>:
We have done a preliminary scoping of the work involved in revising the existing Root Zone Management System.
This is good news and thank you to all involved. For the record, (and I have said this before) the IANA staff were VERY helpful in developing and agreeing to the SLE - it is just essential that we have community based SLE in place and in operation at (or before) the date of any transition. My issue was (see the attached diagram) that the CRISP (numbers community) have their new SLA being tested from January and I would like the naming community to be treated equally with our numbering community colleagues ... hence January 1st.
Based on that scoping, we feel confident deploying the revised code by the beginning of 2Q2016.
It seems to be taking longer to update the RZMS than it did to write it in the first place. It would be great to see the RZMS audit trail update deployed on the test system asap, with the data capture/extraction tools being written Oct and Nov. You are responsible for this service and it is up to you, I am just trying to get this job finished efficiently so all of the paperwork can be completed well before any transition date.
This provides for up to 6 months of data collection for the establishment of the actual contractual SLAs. If you do not believe this sufficient time, perhaps you could explain why you believe additional time will be necessary?
IANA is not a particularly active service and will a 3 month window generate sufficient data? If we were able to start testing in Jan or early Feb it would provide more data or enable the SLE to be finalized sooner. By Starting in April and collecting data for say 3 months pushes us to July and IANA does not see that much activity I fear that more time may be needed for determining the thresholds. If we can start asap it will enable us to be confident of having fulfilment of the SLE in good time.... and having a community accountability mechanism for IANA service. Best Paul
Paul,
My issue was (see the attached diagram) that the CRISP (numbers community) have their new SLA being tested from January and I would like the naming community to be treated equally with our numbering community colleagues ... hence January 1st.
Simply, you have misinterpreted that diagram. As should be clear in that diagram given the orange blocks _start_ in January, the implementation of the CRISP SLA is scheduled to _start_ on January 1 with completion scheduled in September. In fact, the CWG is not being treated equally, they are getting development priority because we are required to do parallel testing of the code in a particular way that necessitates a 6 month cushion.
It seems to be taking longer to update the RZMS than it did to write it in the first place.
I do not believe hyperbole is helpful here. For the record, due to a variety of factors, most of which were non-technical, it took 2 years to develop RZMS. The changes that are being implemented are being done in a production system in an extremely sensitive environment and we're looking at around 3-4 months of development/testing.
IANA is not a particularly active service and will a 3 month window generate sufficient data?
As stated, we expect to deploy the new code at the beginning of 2Q2016. This would appear to leave (some portion of) April, May, June, July, August, and (some portion of) September for the collection of data. Currently, I'm told IANA is receiving about 100-200 root zone change requests per month. This should supply at least 400 to 800 requests. Of course, it depends on whether some requests are actually made, e.g., the SLA that requires ICANN to measure "Credential recovery time to dispatch confirmation email of forgotten username or password" (another one of those metrics that will be measured in nanoseconds since it measures something that does not require human interaction or code branches), so whether there are enough nanosecond samples depends on how many folks try to recover their passwords. Regards, -drc
participants (8)
-
Andrew Sullivan -
David Conrad -
Gomes, Chuck -
James Gannon -
Jonathan Robinson -
Mueller, Milton L -
Paul M Kane - CWG -
Seun Ojedeji