Hi Naela, I agree with Brett that this is a very good starting point :) However, I do need some clarification regarding regarding the new new metrics that are qualitative in nature: 1) The first metric "KSK ceremony Performed" is qualitative in nature with 100% SLA Threshold. How to do measure the success of the metric if lets say ceremony is delayed by few days ? a) Scenario #1: Ceremony completed successfully no later than 33 days prior to signature expiration. Success = 100% b) Scenario#2: Ceremony completed successfully with 3 days delay and 30 days prior to signature expiration. Will the success be measured at 90% (= 30 * (100/33)) ? OR just 0%. Just trying to understand how these metrics will be calculated. Regards, Gaurav On Wed, Apr 15, 2020 at 2:37 PM Brett Carr <brett.carr@nominet.uk> wrote:

Thanks Naela,

As mentioned this is a great starting point.

Just to get some additional suggestions on the record for when we do pick this up again.

1. I wonder If there is something we can track/assess in relation to the management IANA do of the ZSK Operator. 2. When the discussions have concluded on how often the KSK should be rolled we should track that activity. 3. I’d also like to suggest we track the TCR Churn, as a lot of the management and monitoring is done by the TCRs I think it’s important that new TCRs are regularly added and long term TCRs are retired.

Brett

-- Brett Carr Manager DNS and Network Engineering Nominet UK

*From: *ICANN-CSC <icann-csc-bounces@icann.org> on behalf of Naela Sarras <naela.sarras@iana.org> *Date: *Thursday, 2 April 2020 at 17:31 *To: *"icann-csc@icann.org" <icann-csc@icann.org> *Subject: *[ICANN-CSC] Action Item - Proposed KPIs for KSK performance

Dear CSC,

During the March 2020 CSC meeting, Kim Davies gave a presentation on IANA’s role in managing the Root Key Signing Key (KSK). I have attached the presentation again for your reference if needed.

Following the presentation and discussion, we were tasked with an action item to come up with possible key performance indicators to monitor performance of the KSK.

Please find attached a document regarding with proposed measurements for CSC consideration.

We look forward to discussing the attached with you.

Best regards,

Naela Sarras

Director, IANA Operations

ICANN

_______________________________________________ ICANN-CSC mailing list ICANN-CSC@icann.org https://mm.icann.org/mailman/listinfo/icann-csc

_______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.

-- Regards, Gaurav Vedi

Thanks Naela for clarifying this. I agree those SLAs should be Pass/Fail only unless there is a way to calculate success rates correctly. Stay safe and healthy 😀 Regards Gaurav Sent from my iPhone

On Apr 17, 2020, at 5:23 PM, Naela Sarras <naela.sarras@iana.org> wrote:

Hi Gaurav,

I added Brett’s points to the document we started and will incorporate them into the document before the next CSC check-in on this topic.

For calculating the SLA on the ceremony events that are attached to a number of days, the threshold is proposed at 100% which means anything other than 100% would fail the threshold regardless of how close it is to 100%. The more I think about this given your question, that more I am inclined to say that this should be a pass/fail SLA instead of a threshold up to 100%.

Let me incorporate this in the document for the next CSC check-in on this topic and see what others think.

Thank you, Naela

Naela Sarras Director, IANA Operations ICANN

From: Gaurav Vedi <gaurav.vedi@gmail.com> Date: Wednesday, April 15, 2020 at 12:12 PM To: Naela Sarras <naela.sarras@iana.org> Cc: "icann-csc@icann.org" <ICANN-CSC@icann.org> Subject: [Ext] Re: [ICANN-CSC] Action Item - Proposed KPIs for KSK performance

Hi Naela,

I agree with Brett that this is a very good starting point :)

However, I do need some clarification regarding regarding the new new metrics that are qualitative in nature:

1) The first metric "KSK ceremony Performed" is qualitative in nature with 100% SLA Threshold. How to do measure the success of the metric if lets say ceremony is delayed by few days ? a) Scenario #1: Ceremony completed successfully no later than 33 days prior to signature expiration. Success = 100% b) Scenario#2: Ceremony completed successfully with 3 days delay and 30 days prior to signature expiration. Will the success be measured at 90% (= 30 * (100/33)) ? OR just 0%.

Just trying to understand how these metrics will be calculated.

Regards, Gaurav

On Wed, Apr 15, 2020 at 2:37 PM Brett Carr <brett.carr@nominet.uk> wrote: Thanks Naela,

As mentioned this is a great starting point.

Just to get some additional suggestions on the record for when we do pick this up again.

I wonder If there is something we can track/assess in relation to the management IANA do of the ZSK Operator. When the discussions have concluded on how often the KSK should be rolled we should track that activity. I’d also like to suggest we track the TCR Churn, as a lot of the management and monitoring is done by the TCRs I think it’s important that new TCRs are regularly added and long term TCRs are retired.

Brett

-- Brett Carr Manager DNS and Network Engineering Nominet UK

From: ICANN-CSC <icann-csc-bounces@icann.org> on behalf of Naela Sarras <naela.sarras@iana.org> Date: Thursday, 2 April 2020 at 17:31 To: "icann-csc@icann.org" <icann-csc@icann.org> Subject: [ICANN-CSC] Action Item - Proposed KPIs for KSK performance

Dear CSC,

During the March 2020 CSC meeting, Kim Davies gave a presentation on IANA’s role in managing the Root Key Signing Key (KSK). I have attached the presentation again for your reference if needed.

Following the presentation and discussion, we were tasked with an action item to come up with possible key performance indicators to monitor performance of the KSK.

Please find attached a document regarding with proposed measurements for CSC consideration.

We look forward to discussing the attached with you.

Best regards,

Naela Sarras Director, IANA Operations ICANN

_______________________________________________ ICANN-CSC mailing list ICANN-CSC@icann.org https://mm.icann.org/mailman/listinfo/icann-csc

_______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy [icann.org]) and the website Terms of Service (https://www.icann.org/privacy/tos [icann.org]). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.

-- Regards, Gaurav Vedi