HI Alain Unfortunately for me, it was difficult to follow which text was yours and which was previously written because of the text format. But I did understand your concerns from previous emails. What I think will happen is when we start to compile the final report we will be able to record what areas we think we were not able to do a sufficient review and maybe recommend follow up by the next SSR 3 (if the issue is still current/relevant). We will also be able to fine tune as well the language of our conclusions. I think we could also flag your email as a reference to ensure we addressed concerns raised below as well (which I have been following in previous and this email thread). Kerry-Ann Barrett Cybersecurity Program Officer Secretariat of the Inter-American Committee against Terrorism (CICTE) Secretariat for Multidimensional Security (SMS) Organization of American States 1889 F Street NW Washington D.C. 20006   (202) 370 4675 -  (202) 458 3857  www.oas.org/cyber @KerryOAS| @OEA_Cyber Register to our distribution list here!   -----Original Message----- From: Ssr2-review [mailto:ssr2-review-bounces@icann.org] On Behalf Of ALAIN AINA Sent: Monday, November 5, 2018 6:02 AM To: ICANN SSR2 <SSR2-review@icann.org> Subject: [EXT] Re: [Ssr2-review] Please prepare for the plenary call on Thursday Hello,

On 1 Nov 2018, at 23:39, Russ Housley <housley@vigilsec.com> wrote:

...

On Nov 1, 2018, at 4:36 AM, ALAIN AINA <aalain@trstech.net> wrote:

I still think the methodology we used for the assessment of the implementation of the 28 recommendations is not very suitable. We ended the following conclusions:

1- most of the recommendations are vague and not measurable and sometimes not implementable 2- we have not seen evidence of implementation and effect, and so the recommendation has not been fully implemented, effect can’t be measured, etc...

on 1) we will have at some point, to review the board actions on the 28 recommendations and respond to certain aspects in our assessment.

https://www.icann.org/resources/board-material/resolutions-2012-10-18-en#1.e

On 2) we have not agreed on all documents or evidences required for our assessment. We shall also factor in the fact that the organisation has evolved over the years. The recommendations were issued on 2012, and implemented throughout up to 2017 and we are reviewing the impact in 2017/2018.

the SSR framework and the implementation of some of the recommendations have influenced the Strategic plan and Operation plan and the 5 years operating plan regularly updated, have KPIs for programs to y19 and y20 which are SSR related.

Most of the impact of the implementation in my opinion are to be investigated in strategic objectives and their implementation through activities portfolio throughout the years. We shall measure the impact from the KPI and deliverable in the OP, Annual report and also how SSR inputs and influences the strategic/ operation plannings and the prioritisation of the activities and projects. Engage with Staff, board and any other affected parties or constituencies by the recommendations.

On the specifics.

For example we claimed that definition of security in the registry agreement differs from the one published in the SSR framework, added to the ICANN glossary and supposed to be used in all materials.

But, section 7.3 of the registry agreement does not define “security”, or “stability” but define what “effect on them mean or refer to"

======

(a) For the purposes of this Agreement, an effect on “Security” shall mean

(1) the unauthorized disclosure, alteration, insertion or destruction of registry data, or (2) the unauthorized access to or disclosure of information or resources on the Internet by systems operating in accordance with all applicable standards.

(b) For purposes of this Agreement, an effect on “Stability” shall refer to

(1) lack of compliance with applicable relevant standards that are authoritative and published by a well-established and recognized Internet standards body, such as the

======= So using this example to justify a breach in the implementation of the recommendation is not appropriate in my opinion.

Alain:

The Bylaws say:

(iv) The SSR Review Team shall also assess the extent to which prior SSR Review recommendations have been implemented and the extent to which implementation of such recommendations has resulted in the intended effect.

I think we have done what the Bylaws require.

We are in full agreement on the task list as per the bylaws, especially on the SSR1 recommendations. I assumed it is still work in progress as you asked for comments/discussions.

Some actions were taken to implement each of the recommendations. The write-up summarizes those actions.

We concluded that some recommendations were not fully implemented, while staff concluded that they were all fully implemented. This would call for reviewing board/staff understanding of the recommendations, the implementation plan and actions taken to implement to eventually determine where the gap is.

We made an assessment regarding whether those actions "resulted in the intended effect.”

my main concern in this thread is about how this assessment was made. It needs a more structured approach than going through responses to some questions to staff and through some random documentation.

In some cases, we have follow-on recommendations.

We agreed that the wording of many SSR1 recommendations were not measurable,

SSR1 like SSR2 is an “review” and i would say a ”high level review”, from which recommendations may not be directly measurable, as they affect strategic thinking/planning and operations of the organisation and/or the ecosystem. This is my last communication of the topic and as Boban suggested, would be good to hear what the team’s thoughts are about these points. Thanks —Alain

and we agreed to two things: 1) Include some text regarding the situation. Your words above seem like a good strawman. 2) When we write our recommendations, we will make sure that they are measurable.

Russ

_______________________________________________ Ssr2-review mailing list Ssr2-review@icann.org https://mm.icann.org/mailman/listinfo/ssr2-review