Thanks Donna, I agree and support your additions. Martin Boyle Sent from my iPhone

On 13 Oct 2018, at 14:55, Austin, Donna <Donna.Austin@team.neustar> wrote:

Thanks Martin

I've added some comments in blue to your document.

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Martin Boyle via CSC-EffectivenessRT Sent: Wednesday, October 10, 2018 9:02 AM To: 'Debbie Monahan' <debbie@internetnz.net.nz>; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Thanks again to Debbie for doing this work.

As agreed on the call yesterday, I have done my first shot (attached) at completing the form: as we expected, lots of N/A, but I thought it worthwhile flagging the work on the RAP and SLAs in the answers as these are important preparatory pieces.

Being confronted by them, I was not sure how the first two metrics really differed (see footnote).

For metric 7 I am willing to recognise that I have missed something. If not, we need to ask about what has been done or envisaged. And for the first bit of metric 11, I guess that there will be an input there very soon and before we report.

You will see that I have tried to differentiate results between simply achieved (because it has gone as far as it needed to in the period) and fully achieved (where it has fully met the requirements. I am not sure how reasonable such a fine line actually is, but something tells me that, for successor reviews where there may have been some drama, the nuance might be helpful. Happy to go with the majority view. (And if we have a qualified success, we will also need a qualified failure!)

Thanks again to Debbie for getting this started,

Martin

-----Original Message----- From: CSC-EffectivenessRT <csc-effectivenessrt-bounces@icann.org> On Behalf Of Debbie Monahan Sent: 09 October 2018 21:25 To: csc-effectivenessrt@icann.org Subject: [CSC-EffectivenessRT] updated document

Hi all

Please find enclosed the document with the two items I said I hadn’t included first time.

Cheers Debbie <CSC Effectiveness Review 2 MB responses_DA.docx>

Hi everyone, I also support the additions. Added a couple of elements or suggestions in the attached: - Probably minor but the nuance between "fully achieved" and "achieved" is subtle: I suppose that by "fully achieved" we mean that the result exceeded expectations ("achieved beyond expectations"?) If that is the case maybe we want to reflect that. - I suggested minor changes such as the fact that the task non only consists in reviewing SLAs but also considering creating SLAs. - On the "IANA customer complaint procedures", the procedure described at https://www.iana.org/help/escalation-procedure, although it's directed to IANA not to the CSC so I guess I _am_ missing something. If I'm being picky whilst CSC reports are fairly robust, I noted that not all CSC meetings have consistent outputs in terms of notes/transcripts etc at https://www.icann.org/csc. Thought I'd pick that up, and maybe ask when we meet up, not sure that's crucial though. Regards, Philippe -----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Debbie Monahan Sent: Monday, October 15, 2018 4:08 AM To: Martin Boyle Cc: csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document Hi all I support the comments from both Martin and Donna Thanks Debbie

On 14/10/2018, at 05:00, Martin Boyle <martin.boyle.hertford@ntlworld.com> wrote:

Thanks Donna, I agree and support your additions.

Martin Boyle Sent from my iPhone

...

On 13 Oct 2018, at 14:55, Austin, Donna <Donna.Austin@team.neustar> wrote:

Thanks Martin

I've added some comments in blue to your document.

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Martin Boyle via CSC-EffectivenessRT Sent: Wednesday, October 10, 2018 9:02 AM To: 'Debbie Monahan' <debbie@internetnz.net.nz>; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Thanks again to Debbie for doing this work.

As agreed on the call yesterday, I have done my first shot (attached) at completing the form: as we expected, lots of N/A, but I thought it worthwhile flagging the work on the RAP and SLAs in the answers as these are important preparatory pieces.

Being confronted by them, I was not sure how the first two metrics really differed (see footnote).

For metric 7 I am willing to recognise that I have missed something. If not, we need to ask about what has been done or envisaged. And for the first bit of metric 11, I guess that there will be an input there very soon and before we report.

You will see that I have tried to differentiate results between simply achieved (because it has gone as far as it needed to in the period) and fully achieved (where it has fully met the requirements. I am not sure how reasonable such a fine line actually is, but something tells me that, for successor reviews where there may have been some drama, the nuance might be helpful. Happy to go with the majority view. (And if we have a qualified success, we will also need a qualified failure!)

Thanks again to Debbie for getting this started,

Martin

-----Original Message----- From: CSC-EffectivenessRT <csc-effectivenessrt-bounces@icann.org> On Behalf Of Debbie Monahan Sent: 09 October 2018 21:25 To: csc-effectivenessrt@icann.org Subject: [CSC-EffectivenessRT] updated document

Hi all

Please find enclosed the document with the two items I said I hadn’t included first time.

Cheers Debbie <CSC Effectiveness Review 2 MB responses_DA.docx>

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://mm.icann.org/mailman/listinfo/csc-effectivenessrt _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

Thanks for the clarification Donna. So that's clearly different from the complaint reporting function that's referred to on the CSC page (IANA's), they might be related but we need to check. (at the end of that IANA complaint procedure there's some escalation through ICANN's Ombudsman, but that's the only loose tie with ICANN apparently and that seemed late in the process and not that direct reporting procedure that the charter seems to be referring to) So yes to be discussed. Thanks again. Regards, Philippe -----Original Message----- From: Austin, Donna [mailto:Donna.Austin@team.neustar] Sent: Tuesday, October 16, 2018 4:29 PM To: FOUQUART Philippe IMT/OLN Cc: Debbie Monahan; Martin Boyle; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document Thanks for your comments Philippe. I agree to addressing these as part of our work. On the complaints issue, there is a provision in the CSC Charter for the CSC to receive complaints and this is what we are referring to.

From the Charter: "The CSC may receive complaints from individual registry operators regarding the performance of the IANA Naming Function; however, the CSC will not become involved in a direct dispute between any registry operator and IANA. The CSC will review individual complaints with a view to identifying any patterns of poor performance."

Donna Sent from my iPhone

On Oct 16, 2018, at 4:02 PM, "philippe.fouquart@orange.com" <philippe.fouquart@orange.com> wrote:

Hi everyone,

I also support the additions.

Added a couple of elements or suggestions in the attached: - Probably minor but the nuance between "fully achieved" and "achieved" is subtle: I suppose that by "fully achieved" we mean that the result exceeded expectations ("achieved beyond expectations"?) If that is the case maybe we want to reflect that. - I suggested minor changes such as the fact that the task non only consists in reviewing SLAs but also considering creating SLAs. - On the "IANA customer complaint procedures", the procedure described at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iana.org_help_escal..., although it's directed to IANA not to the CSC so I guess I _am_ missing something.

If I'm being picky whilst CSC reports are fairly robust, I noted that not all CSC meetings have consistent outputs in terms of notes/transcripts etc at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_csc&d=DwI.... Thought I'd pick that up, and maybe ask when we meet up, not sure that's crucial though.

Regards, Philippe

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Debbie Monahan Sent: Monday, October 15, 2018 4:08 AM To: Martin Boyle Cc: csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Hi all

I support the comments from both Martin and Donna

Thanks Debbie

...

On 14/10/2018, at 05:00, Martin Boyle <martin.boyle.hertford@ntlworld.com> wrote:

Thanks Donna, I agree and support your additions.

Martin Boyle Sent from my iPhone

...

On 13 Oct 2018, at 14:55, Austin, Donna <Donna.Austin@team.neustar> wrote:

Thanks Martin

I've added some comments in blue to your document.

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Martin Boyle via CSC-EffectivenessRT Sent: Wednesday, October 10, 2018 9:02 AM To: 'Debbie Monahan' <debbie@internetnz.net.nz>; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Thanks again to Debbie for doing this work.

As agreed on the call yesterday, I have done my first shot (attached) at completing the form: as we expected, lots of N/A, but I thought it worthwhile flagging the work on the RAP and SLAs in the answers as these are important preparatory pieces.

Being confronted by them, I was not sure how the first two metrics really differed (see footnote).

For metric 7 I am willing to recognise that I have missed something. If not, we need to ask about what has been done or envisaged. And for the first bit of metric 11, I guess that there will be an input there very soon and before we report.

You will see that I have tried to differentiate results between simply achieved (because it has gone as far as it needed to in the period) and fully achieved (where it has fully met the requirements. I am not sure how reasonable such a fine line actually is, but something tells me that, for successor reviews where there may have been some drama, the nuance might be helpful. Happy to go with the majority view. (And if we have a qualified success, we will also need a qualified failure!)

Thanks again to Debbie for getting this started,

Martin

-----Original Message----- From: CSC-EffectivenessRT <csc-effectivenessrt-bounces@icann.org> On Behalf Of Debbie Monahan Sent: 09 October 2018 21:25 To: csc-effectivenessrt@icann.org Subject: [CSC-EffectivenessRT] updated document

Hi all

Please find enclosed the document with the two items I said I hadn't included first time.

Cheers Debbie <CSC Effectiveness Review 2 MB responses_DA.docx>

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mailman_li...

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

<CSC Effectiveness Review 2 MB responses_DA_PhF.docx> _______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mailman_li...

_________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

Sounds good Martin. I may have some time to do this tomorrow. Sent from my iPhone

On Oct 18, 2018, at 3:43 PM, Martin Boyle via CSC-EffectivenessRT <csc-effectivenessrt@icann.org> wrote:

Hi All,

Given that we all seem to be in agreement on the template, would it be useful to tidy it up (remove colours and make sure the comments are coherent and worded as from the review team) and publish it as a draft report?

I realise that it’s too late to be a formal document for Barcelona, but showing people our direction of travel could help inform our discussions (and it is short enough to be easily read by interested parties.

What do you think?

Martin Boyle Sent from my iPhone

...

On 16 Oct 2018, at 15:59, <philippe.fouquart@orange.com> <philippe.fouquart@orange.com> wrote:

Hi everyone,

I also support the additions.

Added a couple of elements or suggestions in the attached: - Probably minor but the nuance between "fully achieved" and "achieved" is subtle: I suppose that by "fully achieved" we mean that the result exceeded expectations ("achieved beyond expectations"?) If that is the case maybe we want to reflect that. - I suggested minor changes such as the fact that the task non only consists in reviewing SLAs but also considering creating SLAs. - On the "IANA customer complaint procedures", the procedure described at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iana.org_help_escal..., although it's directed to IANA not to the CSC so I guess I _am_ missing something.

If I'm being picky whilst CSC reports are fairly robust, I noted that not all CSC meetings have consistent outputs in terms of notes/transcripts etc at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_csc&d=DwI.... Thought I'd pick that up, and maybe ask when we meet up, not sure that's crucial though.

Regards, Philippe

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Debbie Monahan Sent: Monday, October 15, 2018 4:08 AM To: Martin Boyle Cc: csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Hi all

I support the comments from both Martin and Donna

Thanks Debbie

...

On 14/10/2018, at 05:00, Martin Boyle <martin.boyle.hertford@ntlworld.com> wrote:

Thanks Donna, I agree and support your additions.

Martin Boyle Sent from my iPhone

...

On 13 Oct 2018, at 14:55, Austin, Donna <Donna.Austin@team.neustar> wrote:

Thanks Martin

I've added some comments in blue to your document.

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Martin Boyle via CSC-EffectivenessRT Sent: Wednesday, October 10, 2018 9:02 AM To: 'Debbie Monahan' <debbie@internetnz.net.nz>; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Thanks again to Debbie for doing this work.

As agreed on the call yesterday, I have done my first shot (attached) at completing the form: as we expected, lots of N/A, but I thought it worthwhile flagging the work on the RAP and SLAs in the answers as these are important preparatory pieces.

Being confronted by them, I was not sure how the first two metrics really differed (see footnote).

For metric 7 I am willing to recognise that I have missed something. If not, we need to ask about what has been done or envisaged. And for the first bit of metric 11, I guess that there will be an input there very soon and before we report.

You will see that I have tried to differentiate results between simply achieved (because it has gone as far as it needed to in the period) and fully achieved (where it has fully met the requirements. I am not sure how reasonable such a fine line actually is, but something tells me that, for successor reviews where there may have been some drama, the nuance might be helpful. Happy to go with the majority view. (And if we have a qualified success, we will also need a qualified failure!)

Thanks again to Debbie for getting this started,

Martin

-----Original Message----- From: CSC-EffectivenessRT <csc-effectivenessrt-bounces@icann.org> On Behalf Of Debbie Monahan Sent: 09 October 2018 21:25 To: csc-effectivenessrt@icann.org Subject: [CSC-EffectivenessRT] updated document

Hi all

Please find enclosed the document with the two items I said I hadn’t included first time.

Cheers Debbie <CSC Effectiveness Review 2 MB responses_DA.docx>

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mailman_li...

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

<CSC Effectiveness Review 2 MB responses_DA_PhF.docx>

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mailman_li...

Sorry I cannot help with the work - I am out of action until (hopefully) Monday morning! Martin Boyle Sent from my iPhone

On 18 Oct 2018, at 16:09, <philippe.fouquart@orange.com> <philippe.fouquart@orange.com> wrote:

Donna, Martin,

Sounds good to me too. Once those leftovers of comments are taken out, it should be an easy and quick read. Regards,

Philippe

-----Original Message----- From: Austin, Donna [mailto:Donna.Austin@team.neustar] Sent: Thursday, October 18, 2018 4:04 PM To: Martin Boyle Cc: FOUQUART Philippe TGI/OLN; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Sounds good Martin.

I may have some time to do this tomorrow.

Sent from my iPhone

...

On Oct 18, 2018, at 3:43 PM, Martin Boyle via CSC-EffectivenessRT <csc-effectivenessrt@icann.org> wrote:

Hi All,

Given that we all seem to be in agreement on the template, would it be useful to tidy it up (remove colours and make sure the comments are coherent and worded as from the review team) and publish it as a draft report?

I realise that it's too late to be a formal document for Barcelona, but showing people our direction of travel could help inform our discussions (and it is short enough to be easily read by interested parties.

What do you think?

Martin Boyle Sent from my iPhone

...

On 16 Oct 2018, at 15:59, <philippe.fouquart@orange.com> <philippe.fouquart@orange.com> wrote:

Hi everyone,

I also support the additions.

Added a couple of elements or suggestions in the attached: - Probably minor but the nuance between "fully achieved" and "achieved" is subtle: I suppose that by "fully achieved" we mean that the result exceeded expectations ("achieved beyond expectations"?) If that is the case maybe we want to reflect that. - I suggested minor changes such as the fact that the task non only consists in reviewing SLAs but also considering creating SLAs. - On the "IANA customer complaint procedures", the procedure described at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iana.org_help_escal..., although it's directed to IANA not to the CSC so I guess I _am_ missing something.

If I'm being picky whilst CSC reports are fairly robust, I noted that not all CSC meetings have consistent outputs in terms of notes/transcripts etc at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_csc&d=DwI.... Thought I'd pick that up, and maybe ask when we meet up, not sure that's crucial though.

Regards, Philippe

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Debbie Monahan Sent: Monday, October 15, 2018 4:08 AM To: Martin Boyle Cc: csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Hi all

I support the comments from both Martin and Donna

Thanks Debbie

...

On 14/10/2018, at 05:00, Martin Boyle <martin.boyle.hertford@ntlworld.com> wrote:

Thanks Donna, I agree and support your additions.

Martin Boyle Sent from my iPhone

...

On 13 Oct 2018, at 14:55, Austin, Donna <Donna.Austin@team.neustar> wrote:

Thanks Martin

I've added some comments in blue to your document.

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Martin Boyle via CSC-EffectivenessRT Sent: Wednesday, October 10, 2018 9:02 AM To: 'Debbie Monahan' <debbie@internetnz.net.nz>; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Thanks again to Debbie for doing this work.

As agreed on the call yesterday, I have done my first shot (attached) at completing the form: as we expected, lots of N/A, but I thought it worthwhile flagging the work on the RAP and SLAs in the answers as these are important preparatory pieces.

Being confronted by them, I was not sure how the first two metrics really differed (see footnote).

For metric 7 I am willing to recognise that I have missed something. If not, we need to ask about what has been done or envisaged. And for the first bit of metric 11, I guess that there will be an input there very soon and before we report.

You will see that I have tried to differentiate results between simply achieved (because it has gone as far as it needed to in the period) and fully achieved (where it has fully met the requirements. I am not sure how reasonable such a fine line actually is, but something tells me that, for successor reviews where there may have been some drama, the nuance might be helpful. Happy to go with the majority view. (And if we have a qualified success, we will also need a qualified failure!)

Thanks again to Debbie for getting this started,

Martin

-----Original Message----- From: CSC-EffectivenessRT <csc-effectivenessrt-bounces@icann.org> On Behalf Of Debbie Monahan Sent: 09 October 2018 21:25 To: csc-effectivenessrt@icann.org Subject: [CSC-EffectivenessRT] updated document

Hi all

Please find enclosed the document with the two items I said I hadn't included first time.

Cheers Debbie <CSC Effectiveness Review 2 MB responses_DA.docx>

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mailman_li...

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

<CSC Effectiveness Review 2 MB responses_DA_PhF.docx>

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mailman_li...

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

Hi Everyone Attached is a cleaned-up version of the one that was in circulation earlier. Is this what we had in mind? Donna -----Original Message----- From: Debbie Monahan [mailto:debbie@internetnz.net.nz] Sent: Friday, October 19, 2018 1:11 PM To: philippe.fouquart@orange.com Cc: Austin, Donna <Donna.Austin@team.neustar>; Martin Boyle <martin.boyle.hertford@ntlworld.com>; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document Hi all I agree that that is a good approach from here Cheers Debbie

On 19/10/2018, at 03:09, <philippe.fouquart@orange.com> <philippe.fouquart@orange.com> wrote:

Donna, Martin,

Sounds good to me too. Once those leftovers of comments are taken out, it should be an easy and quick read. Regards,

Philippe

-----Original Message----- From: Austin, Donna [mailto:Donna.Austin@team.neustar] Sent: Thursday, October 18, 2018 4:04 PM To: Martin Boyle Cc: FOUQUART Philippe TGI/OLN; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Sounds good Martin.

I may have some time to do this tomorrow.

Sent from my iPhone

...

On Oct 18, 2018, at 3:43 PM, Martin Boyle via CSC-EffectivenessRT <csc-effectivenessrt@icann.org> wrote:

Hi All,

Given that we all seem to be in agreement on the template, would it be useful to tidy it up (remove colours and make sure the comments are coherent and worded as from the review team) and publish it as a draft report?

I realise that it's too late to be a formal document for Barcelona, but showing people our direction of travel could help inform our discussions (and it is short enough to be easily read by interested parties.

What do you think?

Martin Boyle Sent from my iPhone

...

On 16 Oct 2018, at 15:59, <philippe.fouquart@orange.com> <philippe.fouquart@orange.com> wrote:

Hi everyone,

I also support the additions.

Added a couple of elements or suggestions in the attached: - Probably minor but the nuance between "fully achieved" and "achieved" is subtle: I suppose that by "fully achieved" we mean that the result exceeded expectations ("achieved beyond expectations"?) If that is the case maybe we want to reflect that. - I suggested minor changes such as the fact that the task non only consists in reviewing SLAs but also considering creating SLAs. - On the "IANA customer complaint procedures", the procedure described at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iana.org_help_escal..., although it's directed to IANA not to the CSC so I guess I _am_ missing something.

If I'm being picky whilst CSC reports are fairly robust, I noted that not all CSC meetings have consistent outputs in terms of notes/transcripts etc at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_csc&d=DwI.... Thought I'd pick that up, and maybe ask when we meet up, not sure that's crucial though.

Regards, Philippe

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Debbie Monahan Sent: Monday, October 15, 2018 4:08 AM To: Martin Boyle Cc: csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Hi all

I support the comments from both Martin and Donna

Thanks Debbie

...

On 14/10/2018, at 05:00, Martin Boyle <martin.boyle.hertford@ntlworld.com> wrote:

Thanks Donna, I agree and support your additions.

Martin Boyle Sent from my iPhone

...

On 13 Oct 2018, at 14:55, Austin, Donna <Donna.Austin@team.neustar> wrote:

Thanks Martin

I've added some comments in blue to your document.

-----Original Message----- From: CSC-EffectivenessRT [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Martin Boyle via CSC-EffectivenessRT Sent: Wednesday, October 10, 2018 9:02 AM To: 'Debbie Monahan' <debbie@internetnz.net.nz>; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document

Thanks again to Debbie for doing this work.

As agreed on the call yesterday, I have done my first shot (attached) at completing the form: as we expected, lots of N/A, but I thought it worthwhile flagging the work on the RAP and SLAs in the answers as these are important preparatory pieces.

Being confronted by them, I was not sure how the first two metrics really differed (see footnote).

For metric 7 I am willing to recognise that I have missed something. If not, we need to ask about what has been done or envisaged. And for the first bit of metric 11, I guess that there will be an input there very soon and before we report.

You will see that I have tried to differentiate results between simply achieved (because it has gone as far as it needed to in the period) and fully achieved (where it has fully met the requirements. I am not sure how reasonable such a fine line actually is, but something tells me that, for successor reviews where there may have been some drama, the nuance might be helpful. Happy to go with the majority view. (And if we have a qualified success, we will also need a qualified failure!)

Thanks again to Debbie for getting this started,

Martin

-----Original Message----- From: CSC-EffectivenessRT <csc-effectivenessrt-bounces@icann.org> On Behalf Of Debbie Monahan Sent: 09 October 2018 21:25 To: csc-effectivenessrt@icann.org Subject: [CSC-EffectivenessRT] updated document

Hi all

Please find enclosed the document with the two items I said I hadn't included first time.

Cheers Debbie <CSC Effectiveness Review 2 MB responses_DA.docx>

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_ma ilman_listinfo_csc-2Deffectivenessrt&d=DwIGaQ&c=MOptNlVtIETeDALC_lUL rw&r=CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=Q7Ux9N16AmoyF5AkA YeqBB-ICZJqGlW-u7dIA6w4pqo&s=LP4S61bXbfSeRNwSw6UDGKgRSZ278r85-AIO9TQ cEd8&e=

____________________________________________________________________ _____________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

<CSC Effectiveness Review 2 MB responses_DA_PhF.docx>

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mai lman_listinfo_csc-2Deffectivenessrt&d=DwIGaQ&c=MOptNlVtIETeDALC_lULrw &r=CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=Q7Ux9N16AmoyF5AkAYeq BB-ICZJqGlW-u7dIA6w4pqo&s=LP4S61bXbfSeRNwSw6UDGKgRSZ278r85-AIO9TQcEd8 &e=

______________________________________________________________________ ___________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

_______________________________________________ CSC-EffectivenessRT mailing list CSC-EffectivenessRT@icann.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mail man_listinfo_csc-2Deffectivenessrt&d=DwIFAg&c=MOptNlVtIETeDALC_lULrw&r =CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=4FA4Hhqu4XBpD6ReBjV2UKt TH1b8plwUOeKXvOzRMWc&s=55jntfb4VJ3ioxId91s3PCpF47-JiabVfqJH0TcCXoA&e=

Thanks Trang So sorry I missed them. -----Original Message----- From: Trang Nguyen [mailto:trang.nguyen@icann.org] Sent: Friday, October 19, 2018 6:23 PM To: Austin, Donna <Donna.Austin@team.neustar>; Debbie Monahan <debbie@internetnz.net.nz>; philippe.fouquart@orange.com Cc: csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document Hi Donna, Thank you for this. For consistency with the Straw Man Presentation document that Bart put together and the suggestions that Kim and I proposed there, which got support from Phillippe, I've carried over those suggested edits to this document. The edits were proposed for clarify and for consistency with other CSC procedures. For ease of viewing, I worked off the clean version that you sent and left the suggested edits in redlines. Thank you, Trang On 10/19/18, 5:26 PM, "CSC-EffectivenessRT on behalf of Austin, Donna via CSC-EffectivenessRT" <csc-effectivenessrt-bounces@icann.org on behalf of csc-effectivenessrt@icann.org> wrote: Hi Everyone Attached is a cleaned-up version of the one that was in circulation earlier. Is this what we had in mind? Donna -----Original Message----- From: Debbie Monahan [mailto:debbie@internetnz.net.nz] Sent: Friday, October 19, 2018 1:11 PM To: philippe.fouquart@orange.com Cc: Austin, Donna <Donna.Austin@team.neustar>; Martin Boyle <martin.boyle.hertford@ntlworld.com>; csc-effectivenessrt@icann.org Subject: Re: [CSC-EffectivenessRT] updated document Hi all I agree that that is a good approach from here Cheers Debbie > On 19/10/2018, at 03:09, <philippe.fouquart@orange.com> <philippe.fouquart@orange.com> wrote: > > Donna, Martin, > > Sounds good to me too. Once those leftovers of comments are taken out, it should be an easy and quick read. > Regards, > > Philippe > > -----Original Message----- > From: Austin, Donna [mailto:Donna.Austin@team.neustar] > Sent: Thursday, October 18, 2018 4:04 PM > To: Martin Boyle > Cc: FOUQUART Philippe TGI/OLN; csc-effectivenessrt@icann.org > Subject: Re: [CSC-EffectivenessRT] updated document > > Sounds good Martin. > > I may have some time to do this tomorrow. > > Sent from my iPhone > >> On Oct 18, 2018, at 3:43 PM, Martin Boyle via CSC-EffectivenessRT <csc-effectivenessrt@icann.org> wrote: >> >> Hi All, >> >> Given that we all seem to be in agreement on the template, would it be useful to tidy it up (remove colours and make sure the comments are coherent and worded as from the review team) and publish it as a draft report? >> >> I realise that it's too late to be a formal document for Barcelona, but showing people our direction of travel could help inform our discussions (and it is short enough to be easily read by interested parties. >> >> What do you think? >> >> Martin Boyle >> Sent from my iPhone >> >>> On 16 Oct 2018, at 15:59, <philippe.fouquart@orange.com> <philippe.fouquart@orange.com> wrote: >>> >>> Hi everyone, >>> >>> I also support the additions. >>> >>> Added a couple of elements or suggestions in the attached: >>> - Probably minor but the nuance between "fully achieved" and "achieved" is subtle: I suppose that by "fully achieved" we mean that the result exceeded expectations ("achieved beyond expectations"?) If that is the case maybe we want to reflect that. >>> - I suggested minor changes such as the fact that the task non only consists in reviewing SLAs but also considering creating SLAs. >>> - On the "IANA customer complaint procedures", the procedure described at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iana.org_help_escal..., although it's directed to IANA not to the CSC so I guess I _am_ missing something. >>> >>> If I'm being picky whilst CSC reports are fairly robust, I noted that not all CSC meetings have consistent outputs in terms of notes/transcripts etc at https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_csc&d=DwI.... Thought I'd pick that up, and maybe ask when we meet up, not sure that's crucial though. >>> >>> Regards, >>> Philippe >>> >>> -----Original Message----- >>> From: CSC-EffectivenessRT >>> [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Debbie >>> Monahan >>> Sent: Monday, October 15, 2018 4:08 AM >>> To: Martin Boyle >>> Cc: csc-effectivenessrt@icann.org >>> Subject: Re: [CSC-EffectivenessRT] updated document >>> >>> Hi all >>> >>> I support the comments from both Martin and Donna >>> >>> Thanks >>> Debbie >>> >>>> On 14/10/2018, at 05:00, Martin Boyle <martin.boyle.hertford@ntlworld.com> wrote: >>>> >>>> Thanks Donna, I agree and support your additions. >>>> >>>> Martin Boyle >>>> Sent from my iPhone >>>> >>>>> On 13 Oct 2018, at 14:55, Austin, Donna <Donna.Austin@team.neustar> wrote: >>>>> >>>>> Thanks Martin >>>>> >>>>> I've added some comments in blue to your document. >>>>> >>>>> -----Original Message----- >>>>> From: CSC-EffectivenessRT >>>>> [mailto:csc-effectivenessrt-bounces@icann.org] On Behalf Of Martin >>>>> Boyle via CSC-EffectivenessRT >>>>> Sent: Wednesday, October 10, 2018 9:02 AM >>>>> To: 'Debbie Monahan' <debbie@internetnz.net.nz>; >>>>> csc-effectivenessrt@icann.org >>>>> Subject: Re: [CSC-EffectivenessRT] updated document >>>>> >>>>> Thanks again to Debbie for doing this work. >>>>> >>>>> As agreed on the call yesterday, I have done my first shot (attached) at completing the form: as we expected, lots of N/A, but I thought it worthwhile flagging the work on the RAP and SLAs in the answers as these are important preparatory pieces. >>>>> >>>>> Being confronted by them, I was not sure how the first two metrics really differed (see footnote). >>>>> >>>>> For metric 7 I am willing to recognise that I have missed something. If not, we need to ask about what has been done or envisaged. And for the first bit of metric 11, I guess that there will be an input there very soon and before we report. >>>>> >>>>> You will see that I have tried to differentiate results between >>>>> simply achieved (because it has gone as far as it needed to in the >>>>> period) and fully achieved (where it has fully met the >>>>> requirements. I am not sure how reasonable such a fine line >>>>> actually is, but something tells me that, for successor reviews >>>>> where there may have been some drama, the nuance might be helpful. >>>>> Happy to go with the majority view. (And if we have a qualified >>>>> success, we will also need a qualified failure!) >>>>> >>>>> Thanks again to Debbie for getting this started, >>>>> >>>>> Martin >>>>> >>>>> -----Original Message----- >>>>> From: CSC-EffectivenessRT <csc-effectivenessrt-bounces@icann.org> >>>>> On Behalf Of Debbie Monahan >>>>> Sent: 09 October 2018 21:25 >>>>> To: csc-effectivenessrt@icann.org >>>>> Subject: [CSC-EffectivenessRT] updated document >>>>> >>>>> >>>>> >>>>> Hi all >>>>> >>>>> Please find enclosed the document with the two items I said I hadn't included first time. >>>>> >>>>> Cheers >>>>> Debbie >>>>> <CSC Effectiveness Review 2 MB responses_DA.docx> >>>> >>> >>> _______________________________________________ >>> CSC-EffectivenessRT mailing list >>> CSC-EffectivenessRT@icann.org >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_ma >>> ilman_listinfo_csc-2Deffectivenessrt&d=DwIGaQ&c=MOptNlVtIETeDALC_lUL >>> rw&r=CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=Q7Ux9N16AmoyF5AkA >>> YeqBB-ICZJqGlW-u7dIA6w4pqo&s=LP4S61bXbfSeRNwSw6UDGKgRSZ278r85-AIO9TQ >>> cEd8&e= >>> >>> ____________________________________________________________________ >>> _____________________________________________________ >>> >>> Ce message et ses pieces jointes peuvent contenir des informations >>> confidentielles ou privilegiees et ne doivent donc pas etre >>> diffuses, exploites ou copies sans autorisation. Si vous avez recu >>> ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. >>> >>> This message and its attachments may contain confidential or >>> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. >>> If you have received this email in error, please notify the sender and delete this message and its attachments. >>> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. >>> Thank you. >>> >>> <CSC Effectiveness Review 2 MB responses_DA_PhF.docx> >> >> _______________________________________________ >> CSC-EffectivenessRT mailing list >> CSC-EffectivenessRT@icann.org >> https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mai >> lman_listinfo_csc-2Deffectivenessrt&d=DwIGaQ&c=MOptNlVtIETeDALC_lULrw >> &r=CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=Q7Ux9N16AmoyF5AkAYeq >> BB-ICZJqGlW-u7dIA6w4pqo&s=LP4S61bXbfSeRNwSw6UDGKgRSZ278r85-AIO9TQcEd8 >> &e= > > ______________________________________________________________________ > ___________________________________________________ > > Ce message et ses pieces jointes peuvent contenir des informations > confidentielles ou privilegiees et ne doivent donc pas etre diffuses, > exploites ou copies sans autorisation. Si vous avez recu ce message > par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. > > This message and its attachments may contain confidential or > privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. > Thank you. > > _______________________________________________ > CSC-EffectivenessRT mailing list > CSC-EffectivenessRT@icann.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__mm.icann.org_mail > man_listinfo_csc-2Deffectivenessrt&d=DwIFAg&c=MOptNlVtIETeDALC_lULrw&r > =CwipU91YB6EkpFXK9ynnT_QUef4yC5p7jpsDm8cU97g&m=4FA4Hhqu4XBpD6ReBjV2UKt > TH1b8plwUOeKXvOzRMWc&s=55jntfb4VJ3ioxId91s3PCpF47-JiabVfqJH0TcCXoA&e=