For me it is too early to know which direction this will go but I would like to add my thoughts, and experience, with regard to the work of evaluating proposals. There are three really difficult tasks involved. The first, in which many can participate, it getting the Terms of Reference correct so that applicants know (a) the range of acceptable proposals, and (b) the proper format in which the proposal must be submitted. [/Comment: I have seen proposals with lots of "Advisory Committee", and "Photo next to Notable" content, but little budget information..built on hope I guess]. /Such proposals are easy to discard but do flag the point that the TOR and Format are not just requirements, but are also learning materials for those new to the grant seeking process. /Some funds might be devoted to a learning process for prospective applicants./ The second difficulty is that evaluation is a lot of work, a real lot of work, with fixed timelines. The evaluation team has to be well versed in evaluation, and able to put in intense work under time and decision making pressures. At one level this is obvious, but it is worth reminding us about this as we choose the path forward here. The third difficulty is that the less experienced the evaluators are the more they are tempted to stretch the terms of reference to award points to potentially attractive projects that receive poor marks on some aspects of an evaluation points checklist. This can be kept in check but usually at the expense of time and effort at a time when evaluators are under intense work/time deadlines. Evaluation has to be based on the information in the proposal, with due diligence as required. A promising proposal that falls short may call for some mentoring and future submission. Each of these points may be obvious, but they do need to be kept in mind as we chart a path forward here. Sam L. NPOC On 8/6/2019 1:16 AM, Maureen Hilyard wrote:
Elliot
I think that in the discussion we had, Alan felt that a third party might be more appropriate for the independent panel because volunteers such as those who volunteered for the AP CCWG work for example have demonstrated a tendency to be unreliable as to their stickability to their original purpose on the CCWG and thus he does not believe that volunteers could sustain any real commitment for the load of selection work that would be required. This was refuted by others of us. But it could have been the reason for the "third party" being raised in the meeting notes.
Our discussion also touched on the fact that external experts, who know little about ICANN's mission and what the CCWG had decided might be suitable projects, would be paid, but community volunteers who would have to fit this work around their daily lives and day jobs that for many are non-ICANN-related, would not be paid. This is my recollection.
M
On Mon, Aug 5, 2019 at 2:54 PM Elliot Noss <enoss@tucows.com <mailto:enoss@tucows.com>> wrote:
Hi Emily,
Thank you for this.
Both transcripts match my recollection and back up the point I have made that the belief was that a group from the community and not ICANN Org or otherwise would be making the decisions.
I am not clear where it was decided that it would be some third party, nor do I remember any substantive discussion about what sort of third party could be considered for this. I am not inviting that discussion now, but rather am saying I do not recall it taking place.
Again, thanks.
EN
On Aug 5, 2019, at 1:40 PM, Emily Barabas <emily.barabas@icann.org <mailto:emily.barabas@icann.org>> wrote:
Hi Erika and Elliot, From the staff side, it is our recollection that the language included in the current draft of the Final Report comes out of the meeting held on 10 April 2019, in which public comments on this topic were discussed (see public comment review template for comments in relation to charter question #3:https://community.icann.org/download/attachments/102138829/Charter%20Questio...). Notes and recordings from that meeting are available at: https://community.icann.org/x/rxZIBg. The transcript is here:https://gnso.icann.org/sites/default/files/policy/2019/transcript/transcript.... The action coming out of that discussion is further elaborated in CCWG agreement #18 here:https://community.icann.org/download/attachments/102138829/new%20gTLD%20AP%2..., which resulted in the following text in the draft Final Report:“Regardless of which mechanism is chosen, an independent evaluation panel would be established and responsible for the review and evaluation of applications. Neither the Board nor staff will be taking decisions on individual applications but will instead focus its consideration of the slate on whether the rules of the process were followed by the independent panel. However, due care will need to be given during the implementation phase that safeguards are in place to ensure the independence of the members of the independent evaluation panel.” Kind regards, Emily *From:*Erika Mann <erika@erikamann.com <mailto:erika@erikamann.com>> *Date:*Monday, 5 August 2019 at 17:38 *To:*Elliot Noss <enoss@tucows.com <mailto:enoss@tucows.com>> *Cc:*Emily Barabas <emily.barabas@icann.org <mailto:emily.barabas@icann.org>>, Ccwg-auctionproceeds <ccwg-auctionproceeds@icann.org <mailto:ccwg-auctionproceeds@icann.org>> *Subject:*[Ext] Re: [Ccwg-auctionproceeds] notes and action items from the CCWG Auction Proceeds meeting on Wed, 31 July 2019 (14 UTC). I assume that this can be done Elliot. I will check with Marika, Emily and Joke to see how this this can be done. Erika On Mon, Aug 5, 2019 at 5:34 PM Elliot Noss <enoss@tucows.com <mailto:enoss@tucows.com>> wrote:
For clarity, I wasn’t questioning the meaning but rather correcting the record. Apologies if this came out otherwise. I would be very interested in transcripts of discussions where this was moved on from if it is not too much trouble. Thanks. EN
On Aug 5, 2019, at 11:23 AM, Erika Mann <erika@erikamann.com <mailto:erika@erikamann.com>> wrote: Elliot - well, you have been absent too during rounds of discussions and we moved on. In all fairness, we're re-engaging with his topic because you requested it and we're doing it therefore. Let's focus on the future and let's try to solve the remaining issues and let's not start to question the meaning behind individual sentences. Thank you, Erika On Mon, Aug 5, 2019 at 5:11 PM Elliot Noss <enoss@tucows.com <mailto:enoss@tucows.com>> wrote:
I STRONGLY reject this being presented as “recent”: "In recent CCWG discussions, some members have advocated…..”. I have been clear and unequivocal in my view on this since the CCWG first met. Most often, most participants were supportive of this when raised. Vocally and regularly. Numerous times this has been taking off the table when documents are prepared and then presented as new when I re-raise it. This has now happened too many times for me to be comfortable. To me, this has been in front of the ICANN board members of the CCWG for nearly two years with no pushback. This is the obvious model. It is the one used by CIRA and SIDN and Nominet. It is used by Nominet, iirc, after they tried third parties and moved away from them. We have heard this presented by their experts. A long time ago. With us asking questions over multiple sessions. I would like to be clearly on the record. This has always been on the table. It has always been supported by most voices in the CCWG and some people are uncomfortable with it. I respect that. A method for resolving this point need be established. We could have had straw man polls that would have determined it many times already. It should have been settled over a year ago. It seems to me that the process is looking for a means of killing it by excluding it from documents and looking for technicalities (the board won’t allow it). Let’s have a straightforward discussion on this point and settle it. Independence in this case is independence from submissions asking for money. That is simply not a high bar. (And I am a bit crusty this morning after going through another round of “you support nazis……” with the recent 8 chan nonsense so apologies if the above is harsh) EN
On Aug 5, 2019, at 10:32 AM, Emily Barabas <emily.barabas@icann.org <mailto:emily.barabas@icann.org>> wrote: Dear all, As a follow-up to action item 1 from the call on 31 July, the following question has been sent on behalf of the CCWG to the ICANN Board for consideration: In recent CCWG discussions, some members have advocated for a model in which a panel of ICANN community members evaluates funding applications and selects recipients for funding. From the perspective of these members, it is possible for a panel to be comprised of ICANN community members and also be completely independent. This continues to be a topic that the CCWG is considering with different members expressing different views. Is the ICANN Board willing to consider the possibility of a panel comprised of community members, taking into consideration legal and fiduciary responsibilities and conflict of interest requirements? Kind regards, Emily *From:*Ccwg-auctionproceeds <ccwg-auctionproceeds-bounces@icann.org <mailto:ccwg-auctionproceeds-bounces@icann.org>> on behalf of Joke Braeken <joke.braeken@icann.org <mailto:joke.braeken@icann.org>> *Date:*Wednesday, 31 July 2019 at 18:49 *To:*Ccwg-auctionproceeds <ccwg-auctionproceeds@icann.org <mailto:ccwg-auctionproceeds@icann.org>> *Subject:*[Ccwg-auctionproceeds] notes and action items from the CCWG Auction Proceeds meeting on Wed, 31 July 2019 (14 UTC). Hello All. Please find included below the notes and action items from the CCWG Auction Proceeds meeting on Wed, 31 July 2019 (14 UTC). Best regards, Joke Braeken These high-level notes are designed to help the CCWG navigate through the content of the call and are not meant as a substitute for the transcript and/or recording. The MP3, transcript, and chat are provided separately and are posted on the wiki at:https://community.icann.org/display/CWGONGAP/2019+Meetings. *ACTION ITEMS*
* *#1:*
Leadership team to formulate - as soon as possible - a question to the Board, asking whether the Board could consider a situation where the evaluation would be done by community members, and potentially outside efforts
* *#2:*
Small group to be set up, to clarify the criteria for independence. Stepped forward: Marylin Cade, Maureen Hilyard, Vanda Scartezini, Ching Chiao, Thato Mfikwe, Alan Greenberg, Jonathan Robinson, Elliot Noss
* *#3:*
Erika and Alan to look into the question whether we can strengthen the role of the community further, beyond whether what Alan and Erika originally envisaged, and based on the feedback received during the public comment period.
* *#4:*
clarifications to be provided to ICANN Board liaisons
o Alan and Erika to clean up the current text. See action item #3. o Leadership team with support from staff will review the accountability / review mechanisms, incl. audit. See how they overlap with the proposal, if at all
* *#5:*
CCWG members are encouraged to review Sam’s memo ahead of the next meeting.
* *#6:*
Sam will revise the memo about the individual appeal. Alan, possibly other community members will weigh in over email regarding Sam’s proposal
*NOTES*
1. *Welcome/Agenda Review/SOI Updates*
No roll call. Attendance will be taken from the zoom room. Please remember to mute your microphones when not speaking and state your name before speaking for transcription purposes.
2.*Next Steps Regarding Selection of Recommended Mechanism(s)*
4 mechanisms were created: looking at various scenarios that are already available, and pick the best. A: inhouse model B: inhouse + other entity C: foundation D: fund outsourced to other entity D was ruled out, C was favoured less. Public comment: quite a few comments favoured C. Different concerns raised about independence. Erika favours to recommend only 1 single mechanism, and to make a recommendation to the board. Alternatively, we could make a hierarchy and present it to the Board. Once a decision about the mechanism was taken, the next group needs to do the work on how this shall function. To help them, we have established guidelines: They formulate some principles. You can find the graphics here:https://community.icann.org/display/CWGONGAP/Graphics
And the summary of inputs from ICANN Board and Org are here:https://community.icann.org/display/CWGONGAP/FAQs
Emily runs through the e-mail sent by Joke to the list, on behalf of the co-chairs. Summary of the main topics:
* /Independence/
Topic discussed a lot in the past, and also the Board and ICANN Org provided input.
See Q&A, 2 graphics as shared in Marrakech (division responsibilities & common characteristics)
* /Role independent panel? who would sit on this panel?/
ICANN Board spoke already about this. Info included in the FAQ, and recent email exchanges from Board Liaisons to the group
* /Relevant costs linked to implementing the mechanisms/
See email sent to ICANN org
* /Mechanism D and differences with B/
Mechanism D was taken off the table. Leadership team does not want to re-open the discussion
* Role of the ICANN community
As part of the agenda for today. Proposal by Alan and Erika
Alan: Focusing too much on the names of the mechanisms, instead of on the substance, on the characteristics. If we have a foundation, what are the criteria to make sure it is indeed independent? Elliot: Critical comments on the view of the CCWG. they assumed ICANN org to take the decisions. However, Independent panel composed by the community. Seems there is strong consensus with the CCWG for mechanism A where the independent panel is ICANN community. That would require a level of independence both operationally, and in terms of COI (similar to NomCom). Suggests to focus on that view. Judith: If we outsource, and take into account what the Board has said, there is not much difference between A and B Becky: No follow-up conversations happened so far with the rest of the Board. Independence issue: the Board finds it absolutely critical that the panel that evaluates and selects winners and losers must be completely independent. Indepent of ICANN org, the Board, and the vested interests in the community itself. Board has fiduciary obligations and needs to ensure ICANN stays within its mission. Administrative tasks for ICANN Board. Community has a big role in talking about the priorities, the training for the independent panel etc. Jonathan: Conflict between Becky and Elliot his view? Elliot thinks what Becky says is correct. It does not contradict his view of community involvement. A community group is scoring and choosing proposals. There should be no COI with those proposals. Principles of independence should apply. As an example: how NomCom deals with conflicts. Members of the community group can help promote, but step out as soon as there is a COI. Becky: NomCom is not independent, in the sense that its members are drawn from the various parts of the community, including people who are very active in ICANN processes, and have vested interests in outcomes Erika: Community is stretched thin. High workload. Either the community is the evaluator, or the community has an oversight function. Latter is more appropriate in Erika’s view. Becky: The Board sees a very important role for the community. However, icann community members in the selection and evaluation panel makes independence extremely difficult to achieve. Role of the community is critical, in an advisory Board. Alan: Evaluation the applications should be done by professionals. Concerns relying on a volunteer force. Summary by Erika:
* Evaluation by community members, and potentially outside efforts: is the Board going to even consider this option? If not, it is clear what we need to do. * Bigger role of the community (Alan was working on a proposal) Can we strengthen the model of the current advisory panel and the overall role of the community. * We need to review the criteria of what we refer to as independence. To be included in the guidelines. E.g. PTI Board, is not completely independent. Whatever Board will be constructed (A and C), the role of the completely independent Board members will be higher than the role of potential ICANN Board members who sit on this Board too.
Action items:
* #1: Evaluation by community members, and potentially outside efforts: is the Board going to even consider this option? Leadership team to formulate a question to the Board, asap. * #2: Small group to be set up, to clarify the criteria for independence. * #3: Question whether we can strengthen the role of the community further, beyond whether what Alan and erika envisages, and based on the feedback received during the public comment period.
2.*Proposal – Role of the Community (see attached updated draft Final Report – pages 21-22)*
Proposal presented by Alan, that stands regardless of what the selection panel is. What mechanism do we use, once the programme is running, to make sure the programme stays on track? Mid-course corrections might be needed. Called an “Advisory Board”. Some say it is a working group, or a committee. Arbitrary name. Role of the Advisory Board (AB):
* Consist of people nominated by SO/ACs. ASO or SSAC might not have a lot of interest, so maybe a subset of the SO/ACs * Primary function is to do an annual review, starting in year 2. Looking at what projects were approved, how does it map with what we envisaged, are the projects successful? * The group could report to the ICANN Board, the mechanism itself, and to the ICANN community.
Programme Evaluation Panel (PEP):
* See organisational reviews. Done every 3 years * Taking input from Advisory Board and others. Are major shifts needed? Do we need to change the mechanism?
Project Evaluation Panel (PEP) = Group doing the actual selection Assessment = determine how well the projects are working Why are both AB and PEP needed? One is on an annual basis, looking at a low level. Minor changes, tweaks. The first group will act as a sounding board to the PEP. The second group is looking at matters at a much higher level. Erika: asking for confirmation from the group about whether this proposal is something that could be supported by the group in principle. No comments were raised. the group would keep the conversation going. Becky: Understands the different roles. But if things are going drastically wrong, the AB would notice that. And that might be the trigger for a different process. Concern why we need both the AB and the PEP. The AB would be able to interact with the independent evaluation panel and provide input in midst of evaluation: that is extremely worrisome. Inappropriate for the AB to develop the materials that go out to applicants. Training etc for those doing the evaluation. Extreme danger of affecting outcome of processes in an ad hoc way, which raises the potential for disputes. Maureen: 2 evaluation groups, that increases the potential for conflict. Alan:
* Interaction between evaluation group and advisory Board: From the discussion in Marrakech, a proposal was raised where the PEP could bring a specific project to AB and ask for opinions. This raised strong red flags.
Can the evaluators ask for clarifications somewhere? That is what we were looking into
* Instead of a 3-year review, having it triggered by the AB
Becky:
* Role of an audit. We need to understand the full range of accountability things that are going on here. We cannot have an audit doing the same thing that the programme evaluation panel would be doing
Erika: Review cycle. Audit is part of it Action item:
* #4: clarifications to be provided to ICANN Board liaisons
o Alan and Erika to clean up the current text. o Leadership team with support from staff will review the accountability / review mechanisms, incl. Audit. See how they overlap with the proposal, if at all
Erika: No operational audit at ICANN. Financial audit ICANN does have
3.*Discussion of Individual Appeal Mechanism (see attached memo from ICANN Legal) *
Introduction by Sam: If there were to be a separate appeals process, it would have to be simple. Other grant making organisations have also a lightweight process. Appeals are not about the individual judgement of the panel. It needs to be based on certain grounds. E.g. improper criteria applied, lack of compliance with published process, COI etc. Do not allow individual grant applicants, and they should not use ICANN’s accountability mechanisms. What will give them a chance to challenge decisions? Due process? Action items:
* #5: Group members are invited to review Sam’s memo ahead of the next meeting.
Alan:
* Who does it go to? * Applicant to receive rationale for rejection of an application?
Sam: Final appeals process needs to match with the process we have. Maybe not all grounds would apply. Who the evaluator is? These are appeals to the panel itself, or a person that oversees the panel. It is at the implementation level. How is the panel administered? A person responsible for coordinating the work by the panel? Into the panel decision making process. Action items:
* #6: Sam will revise the proposal about the individual appeal. Alan, possibly other community members will weigh in over email regarding Sam’s proposal
4.*AOB*
Scheduling of the next call. 14 August 2019 | 14 UTC Joke Braeken ccNSO Policy Advisor joke.braeken@icann.org <mailto:joke.braeken@icann.org> Follow @ccNSO on Twitter: https://twitter.com/ccNSO[twitter.com] <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_ccNSO&d=DwM...> Follow the ccNSO on Facebook: https://www.facebook.com/ccnso/[facebook.com] <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_ccnso_...> http://ccnso.icann.org[ccnso.icann.org] <https://urldefense.proofpoint.com/v2/url?u=http-3A__ccnso.icann.org&d=DwMGaQ...>
_______________________________________________ Ccwg-auctionproceeds mailing list Ccwg-auctionproceeds@icann.org <mailto:Ccwg-auctionproceeds@icann.org> https://mm.icann.org/mailman/listinfo/ccwg-auctionproceeds _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy [icann.org] <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_privacy_p...>) and the website Terms of Service (https://www.icann.org/privacy/tos [icann.org] <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_privacy_t...>). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
_______________________________________________ Ccwg-auctionproceeds mailing list Ccwg-auctionproceeds@icann.org <mailto:Ccwg-auctionproceeds@icann.org> https://mm.icann.org/mailman/listinfo/ccwg-auctionproceeds _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy [icann.org] <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_privacy_p...>) and the website Terms of Service (https://www.icann.org/privacy/tos [icann.org] <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.icann.org_privacy_t...>). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
_______________________________________________ Ccwg-auctionproceeds mailing list Ccwg-auctionproceeds@icann.org <mailto:Ccwg-auctionproceeds@icann.org> https://mm.icann.org/mailman/listinfo/ccwg-auctionproceeds _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
_______________________________________________ Ccwg-auctionproceeds mailing list Ccwg-auctionproceeds@icann.org https://mm.icann.org/mailman/listinfo/ccwg-auctionproceeds _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
-- ------------------------------------------------ "It is a disgrace to be rich and honored in an unjust state" -Confucius 邦有道，贫且贱焉，耻也。邦无道，富且贵焉，耻也 ------------------------------------------------ Dr Sam Lanfranco (Prof Emeritus), Econ, York U., CANADA email: sam@lanfranco.net Skype: slanfranco blog: https://samlanfranco.blogspot.com Phone: +1 613-476-0429 cell: +1 416-816-2852