Dear Steve, Please find below our responses. Apologies for the delay. Best regards, Rinalia On Wed, Jan 27, 2016 at 11:03 PM, Steve DelBianco <sdelbianco@netchoice.org> wrote:
Bruce — thank you for elaborating on the board’s general comment about bringing AoC reviews into the bylaws (Rec 9). Your email helped us understand specific areas where you want to leave more decisions to the board and to the AC/SO chairs.
I have offered response in-line with your comments (see red text below), based upon my notes from multiple CCWG discussions of this topic over the last 12 months.
From: <accountability-cross-community-bounces@icann.org> on behalf of Bruce Tonkin <Bruce.Tonkin@melbourneit.com.au> Date: Tuesday, January 26, 2016 at 3:19 PM To: CCWG Accountability <accountability-cross-community@icann.org> Subject: [CCWG-ACCT] ICANN Board comments on recommendation 9
Board Comment - Recommendation 9:
As noted in its comments on the third draft proposal, the Board supports the incorporation of the AOC reviews into the Bylaws, while noting the importance of maintaining operational standards for reviews outside of the Bylaws. While the Board agrees that operational standards should be in alignment with the provisions of the Bylaws, the Board views operational standards as a more suitable place to address multiple review-related operational items that do not belong in the Bylaws.
There are a few specific areas that the Board is flagging in relation to the operational standards -
a) The Board is concerned about potential constrains that may limit flexibility and effectiveness in the operational standards and that certain CCWG-Accountability recommendations may not be aligned with best practices or industry standards, including:
o Fixed numbers of total review team members, as well as a fixed allocation among SO/ACs, without consideration of specific issues and required expertise for a given review.
Reply: This regards para 54 in Annex 9, repeated here for convenience:
"Review Teams are established to include both a fixed number of members and an open number of participants. Each SO and AC participating in the review may suggest up to seven prospective members for the Review Team. The group of chairs of the participating SOs and ACs will select a group of up to 21 Review Team members, balanced for diversity and skills, allocating at least three members from each participating SO and AC that suggests threeor more prospective members. In addition, the ICANN Board may designate one Director as a member of the Review Team."
CCWG proposed that "chairs of participating SOs and ACs will select a group of *up to 21* Review Team members.” The phrase is “up to 21”, so we are not actually proposing a fixed number of review team members. (We do acknowledge the need to remove “fixed number of members” from the introductory sentence.)
We are proposing a maximum of 21 members, based on community experience with previous reviews and data provided by staff on the number of members proposed by the community. We proposed 21 in order to accommodate each of the 7 AC/SOs having up to 3 members. We anticipate that some ACs and SOs will not offer 3 names, and have given the AC/SO chairs flexibility to give those member slots to an AC/SO that had higher interest and offered more than 3 candidates (per a comment from GNSO CSG).
On number of Review Team Members: We recommend not specifying a fixed number for Review Teams, but suggest providing guidelines in the Operational Standards on the typical size-ranges of effective and manageable Review Teams. We also recommend giving flexibility to the chairs of the participating SO and ACs to determine the size of each Review Team depending on its purpose. It is possible to provide guidance in the Operational Standards not to exceed a certain figure (e.g. 21) for effectiveness. In terms of number of nominees per SO/AC: there may be valid reasons for an SO/AC to put forth more than 7 nominees. Example: The GNSO recently endorsed 18 nominees for the Review Team on Competition, Consumer Trust and Consumer Choice (CCT), of which six were selected. Having an arbitrary number of nominees affects available choice and may deter efforts by the selectors to maximize the diversity and expertise needed for the review team. Consider the scenario where each SO/AC puts forth at least 3 names. For the Review Teams on CCT and WHOIS, given that they focus predominantly on gTLD issues, the GNSO may need more than 3 spots on each Review Team. Under the CCWG proposal, GNSO participation would be limited and this may affect the effectiveness of the reviews.
o Unlimited number of participants, in addition to the appointed review team members, potentially affecting the team's productivity.
Reply: This regards para 54 in Annex 9, where CCWG proposed an”open number of participants”. Levels of community interest and the principle of transparency argue for that meetings, calls, and email lists for cross-community working groups and review teams be open to the community. We note that chair(s) of the Review Team could restrict posting privileges and consensus calls to appointed members of the Review Team, and thereby manage the team’s productivity. CCWG proposed this to ensure that Review Teams work in a predictable method that is consistent with cross community working groups in the ICANN context.
The Board agrees with the use of “observers” for Review Teams to manage productivity while maximizing transparency. It is important to note that the number of Review Team members and observers/other participants have cost implications. The number of Review Team members has a direct and significant impact on the cost of travel and meeting facilities. Additionally, the work of managing the meetings, remote participation facilities, coordination of activities and productive interactions with subject matter experts increases based on the number of individuals participating in the proceedings. This means increased staff costs as well as increased time required from the Review Team members to organize and conduct their work. A number of useful strategies can be applied to manage resources prudently while ensuring that the reviews proceed in an open and transparent manner with opportunities for the community to engage and provide feedback. For example observers could be invited to listen in to teleconferences with listen-only lines, and it should be possible for observers to be present during a physical meeting, but without travel funding support. Meetings should also be recorded and transcribed for ease of access to anyone interested in the work of the review team. If the Review Team is able to restrict posting privileges and consensus calls, the remaining concern is related to how to handle sensitive or confidential information. This may arise in the Review Team on Security, Stability and Resiliency (SSR) for example, where there may be concerns with broadcasting publicly areas of SSR weakness or potential mitigation tactics. While some practices from cross-community working groups may be positive and appropriate for review teams to adopt, there are fundamental differences between AoC Review Teams and cross-community working groups, and there is a need to be careful not to conflate the two.
o Exact trigger points for the commencement of reviews without taking into account the Community bandwidth, or the state of pending implementation activities.
Reply: This regards a paragraph on each of the 4 AoC reviews “shall be convened no less frequently than every five years, measured from the date the previous review was convened.” CCWG took into account community bandwidth concerns when we proposed that AoC reviews occur every 5 years — instead of the 3 year interval required in the AoC. We phrased the proposal to give flexibility for reviews to occur sooner than 5 year intervals. But we felt strongly that reviews need an interval requirement in order to ensure the reviews are not deferred indefinitely.
On the importance of flexibility: The Board agrees that reviews should not be deferred indefinitely and supports a review cycle that ensures predictability and regularity of reviews. We would like to point out that the specificity of the current AoC language led to the current situation where 3 AoC reviews were to commence simultaneously (CCT, WHOIS and SSR) in 2015. The Board reacted to Community concerns about volunteer workload by deferring two of the reviews to start in 2016 on a staggered basis. Exact trigger points that are hard coded into the Bylaws would take away this flexibility to adjust the schedule. On the challenges of ascertaining the appropriate time for triggering reviews: A key aspect of subsequent reviews is to look at whether improvements that resulted from the past review recommendations have been effective in addressing the finding. This assessment can be difficult in situations where not enough time is available to operationalize the improvement and determine its effectiveness, especially in cases where implementation projects are complex and lengthy. When the next review is triggered such factors should be taken into consideration in order to use community and ICANN resources responsibly*, *while still allowing ICANN to meet its Bylaws obligations and providing predictability on when those reviews will commence. b) To accommodate differing needs of reviews, the Board recommends
leaving the number of review team members to the selectors of a specific review team, as to prescribing a specific formula for composition. This could leave to the selectors the flexibility, for example, to include more members from a specific SO or AC that is more impacted by a specific review, without hardcoding numbers into the Bylaws that might need to be changed later.
Reply: We are proposing a maximum of 21 members, based on community experience with previous reviews and data provided by staff on the number of members proposed by the community. We proposed 21 in order to accommodate each of the 7 AC/SOs having up to 3 members. We anticipate that some ACs and SOs will not offer 3 names, and have given the AC/SO chairs flexibility to give those member slots to an AC/SO that had higher interest and offered more than 3 candidates (per a comment from GNSO CSG). .
Please refer to the responses above. c) The Board is concerned with the CCWG-Accountability's
recommendations on determinations of how consensus is applied.
Reply: This comment regards para 57: "If consensus cannot be reached among the participants, consensus will be sought among the members. In the event a consensus cannot be found among the members, a majority vote of the members may be taken. " CCWG proposed this text to ensure that Review Teams work in a predictable method that is consistent with cross community working groups in the ICANN context.
On possible limitations to adopting CCWG practices/method: While there are certain similarities between a Review Team and a cross-community working group, a Review Team is not designed to operate in the same way as a cross-community working group. While the Board supports allowing observers to review teams, asking for consensus calls among these observers requires a level of participation that would impact the Review Team's productivity. Review Teams tackle difficult issues such as WHOIS, and evaluation enhancement of competition, consumer trust and choice through the introduction of New gTLDs. They are expected to develop concise, actionable and prioritized recommendations on these complex issues through members that have been vetted and endorsed by the SOs and ACs as having the requisite knowledge, experience and skills to contribute to the review. Including observers in broader consensus calls could frustrate the targeted work that the review teams are expected to achieve.
Imposing Bylaws requirements on allowing participants and observers, or requiring consensus calls are other examples where trying to hardcode specific requirements for reviews now might actually develop reviews that are less efficient, more resource intensive, and detract from the responsibilities of the review teams.
The Board notes that the ICANN community would benefit from a review schedule that would take into consideration community bandwidth and ICANN resources in developing a staggered or phased review schedule. These factors should determine what a workable number of concurrent reviews would be and ensure that no more than that number of reviews are scheduled at the same time.
Finally, the Board would like to highlight the work that has been underway within ICANN towards improving review effectiveness so that the CCWG and the community may factor this work in the development of operational standards. Work has been underway on the development of Operational Standards since last year, originating from ATRT2 recommendation 11 to improve effectiveness of Reviews together with Board Resolution 2015.07.28.14. In July 2015, after factoring public comments, the Board endorsed the proposed process and operational improvements designed to simplify and increase the effectiveness of Reviews. The Organizational Effectiveness Board Committee is currently working to finalize Policies, Procedures and Guidelines for the Organizational Reviews mandated by the Bylaws.
Reply: We note that the drafting being done by the board is for Organizational reviews — not the AoC reviews. Still, CCWG is open to consider the board’s specific operational improvements as part of the implementation of CCWG’s proposal. If possible, could you circulate to CCWG the current draft of these policies, procedures, and guidelines?
The standards are intended to apply to both AoC Reviews and Organizational Reviews. There are similarities between the two as well as lessons learned from prior reviews that apply to both. Previous public comments and Board discussions have been focused on standardization across the two types of reviews. Work on compiling review-relevant information is ongoing by Staff with the goal of creating a single comprehensive resource that provides both a high-level overview as well as detailed operational information/guidelines. The user-friendliness of this resource is currently limited. There is a need to improve it with community input as Operational Standards provide best utility when they are continuously updated based on community needs and lessons learned, and are provided in an easy to use format. We propose the following approach: Staff will share the listing of topics to be addressed by Operational Standards and the community will be asked whether they would support putting together a W orking G roup to work with Staff on developing the detailed Standards. Other planned steps in the refinement of the operational standard include: Sharing information with the CCT Review Team as a pilot for utilizing Operational Standards and gather their feedback; Gathering feedback from former Review Team chairs and interested members; Implementing surveys at the end of each Review to gather feedback on how to improve the process; Holding webinars and other community engagement sessions for additional feedback; Creating a wiki space for interested community members to see work in process and offer feedback and suggestions; Developing an easy to use interactive means of navigating Operational Standards for improved clarity, transparency and understanding. END
_______________________________________________ Accountability-Cross-Community mailing list Accountability-Cross-Community@icann.org https://mm.icann.org/mailman/listinfo/accountability-cross-community