Updated Subject Line: Notes and Action Items - New gTLD Subsequent Procedures PDP WG - 16 March 1500 UTC
With Correct Subject Line Dear Working Group members, Please see below the notes from the meeting on 16 March at 1500 UTC. These high-level notes are designed to help WG members navigate through the content of the call and are not a substitute for the recording, transcript, or the chat, which will be posted at: https://community.icann.org/display/NGSPP/2020-03-16+New+gTLD+Subsequent+Pro.... Kind regards, Julie Notes and Action Items: Actions: Recommendation xx (rationale 4): ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Re: Qualifications of Panelists: ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. Notes: 1. Updates to Statements of Interest: No updates provided. 2. Discussion of Final Report Topics – see the documents at: https://docs.google.com/document/d/1xXu7gPKiblS3Vh4MCuK6NWfeRmMolXf9VF5sO7OG... and attached as a PDF. a. 2.9.1 Community Applications -- Does the WG want to affirm or otherwise address this PIRR recommendation? Recommendation 4.1.a: Consider all dimensions of the feedback received to revisit the CPE scoring and framework before the next application round. Comment from Jeff: Comment from Jeff Neuman: Technically I am note sure we are affirming this whole Guideline F. For example, I am not sure yet whether we will allow mutual agreement resolution...and the last part about giving the ICANN Board discretion, etc. The only thing we are affirming is the priority of communities in a contention set. COMMENT IS ADDRESSED. -- Comment from Staff: Does the WG want to provide any detail here? Request from ICANN org: "It would be helpful if the PDP Working Group could provide more detailed guidance on the specific areas of the CPE process that “must be more transparent and predictable.” Additionally, it would be helpful if the PDP Working Group could provide more specific guidance on what should be changed or added that would enhance transparency and predictability for the CPE process." -- Started a thread on changes to the CPE guidelines. ACTION ITEM: Staff to send the updates to the CPE guidelines based on the WG discussions. DONE: See -- https://docs.google.com/document/d/1Ih_1NARViJXNNewDg-q87sQzQoC1dCtC/edit. Recommendation xx (rationale 4): ICANN must consider ways to improve evaluators’ ability to gather information about an application. Implementation Guidance xx (rationale 4): Evaluators should continue to be able to send clarifying questions to CPE applicants but further, should be able to engage in written dialogue with them as well. Implementation Guidance xx (rationale 4): Evaluators should be able to issue clarifying questions, or utilize similar methods to address potential issues, to those who submit letters of opposition to community-based applications. Discussion: -- Question: Did we get access to the actual research of the evaluators? Answer: We didn’t, but they did refer to research in their decisions. This was not consistent. -- There should be no basis for an independent evaluator to do his or her own research. Worried about how this recommendation was developed. -- Not all of the facts may be in the application, which is different from the UDRP. There should be research that may need to be done. -- Shouldn’t that be on the complaining party to make their case? -- We are saying that if there is independent research that is allowed then ICANN should improve evaluators ability to get information. -- Perhaps it would be helpful to add to the recommendation “as described in the IG below” or “as set out in the Implementation Guidance below.” -- It's okay for evaluators to find independent info - but they should disclose sources when ruling on the CPE. -- The application should be robust and supported. And the public gets a chance to comment, right? -- If you have support from major organizations, how does an evaluator know that they are real? They aren’t likely to be subject-matter experts. -- If they are going to be doing research they could research who these organizations are to make sure they meet the requirements to support the application. -- Anything that an evaluator is relying on should be disclosed to the applicant, particularly since we’ll have an appeal process. The applicant has to have the ability to challenge any independently developed information. -- Trying to choose between two models 1) that the applicant should put together a robust application, encourages a robust public comment period, and discourages the evaluator from being in the role of a biased researcher 2) evaluators being inadvertently advocating for third parties that didn’t show up. -- If we have all these criteria doesn’t it make sense that due diligence is done provided they give the applicant the ability to do clarifying questions in response. For example, the requirement for the uniqueness of the string would seem to be something that would require independent research. Also, documented support from the recognized community institutions. -- Shouldn’t rely only on the information provided by the applicant. -- The examiner should rely on a fulsome record, rather than create their own record. There is a requirement to gauge opposition from third parties. We should figure out a way to make them narrow, as with the Implementation Guidance -- maybe move those into the recommendation. -- Don’t agree that we ban a panelist from doing outside research. Seems too extreme. -- We are lacking notice to competitors. Need to let the examiners look at opposition. -- Say, “In the event panelist(s) rely on independent research, that will be disclosed to the applicant through a Clarifying Question and the applicant should be allowed to respond prior to ruling. ” -- If we require the evaluator to notify the applicant, isn’t the applicant just going to push back? The applicant should just have to provide a fulsome application. -- If the applicant provides links/sources then at a minimum the evaluator can look at those links. -- But how would an evaluator know that a term is commonly used in other industries without doing research. The Applicant is not going to put that into its own application. -- Needs to be very clear that there isn’t just a request for a string with nothing to support it. It is one thing for the evaluator to confirm the information, but another for an evaluator to seek out opposition to it. -- Need to remember that this is an evaluation -- in other evaluations in ICANN, say financial, does the evaluator only rely on what is provided by the applicant. -- Add “except as needed for verification purposes and should be disclosed”. To verify the information. -- Does the AGB currently allow evaluators to do research? Research is specifically noted. In paragraph 2 in 4.2.3. Bottom of the paragraph, pages 4-9. We are working on putting some guardrails around this. -- Need to figure out what “if deemed necessary” means. -- If we limit the ability of the evaluator to do research we should call that out in the public comment as it is a fairly significant change. -- Important to encourage dialog, but can’t require it. If you have a community application that has overwhelming support, those who are in opposition should step up and encouraged to enter into a dialog. -- The amount of opposition should be compared to the amount of support. 5 letters of opposition might be irrelevant - if you have 500 letters of support. 1 or 2 letters shouldn't automatically cost a point. Opposition should be weighed against support. -- There should be a public and transparent verification of an organization stating opposition in a letter. ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Jamie Baxter also should consider what should be added in order to encourage dialog, and public and transparent verification of statements of opposition. Re: The Working Group notes that CCT-RT Recommendation 34 states: “A thorough review of the procedures and objectives for community based applications should be carried out and improvements made to address and correct the concerns raised before a new gTLD application process is launched. Revisions or adjustments should be clearly reflected in an updated version of the 2012 AGB.” -- Staff Comment: From the CCT-RT recommendations tracking spreadsheet: As the WG refines recommendations on this topic, it may want to consider whether to make additional recommendations regarding objectives. Note that the CCT-RT recommendations consider "a higher rate of success for such applications" to be a measure of success. -- Are we trying to get a higher success rate and what can be done other than the recommendations we’ve already made? -- Need to have a requirement that the evaluators are knowledgeable about the community. Need to avoid the mistakes from the previous round. -- We do have a section on accountability and appeals -- in the section on Accountability Mechanisms. -- The qualifications of the panelists is not something we spent a lot of time on. Need to see if we can improve that. -- Need quantitative and qualitative assessment by experts in communities. ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities.
Thanks, Julie, for these notes. All, I regret not being able to join Monday's call. I've just listened to the recording and found the discussions encouraging. I wanted to propose something additional for consideration, which is that perhaps, preceding a recommendation re: Qualification of Panellists (as noted below), t*here should be a recommendation requiring ICANN Org to be more transparent in its process of selecting a community priority evaluation provider/panel*. Rationale: I don't think I would be wrong to say there have been made comments regarding the suitability of Economic Intelligence Unit as the CPE provider and perhaps to help ensure CPE panellists have suitable qualifications, an antecedent step would be to allow ICANN community consideration and intervention to take place in ICANN Org's selection of the CPE provider. Thanks, Justine ----- On Tue, 17 Mar 2020 at 01:43, Julie Hedlund <julie.hedlund@icann.org> wrote:
*With Correct Subject Line*
Dear Working Group members,
Please see below the notes from the meeting on 16 March at 1500 UTC. *These high-level notes are designed to help WG members navigate through the content of the call and are not a substitute for the recording, transcript, or the chat,* which will be posted at: https://community.icann.org/display/NGSPP/2020-03-16+New+gTLD+Subsequent+Pro... .
Kind regards, Julie
*Notes and Action Items:*
*Actions:*
*Recommendation xx (rationale 4)*:
ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9.
*Re: Qualifications of Panelists*:
ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities.
*Notes:*
1. Updates to Statements of Interest: No updates provided.
2. Discussion of Final Report Topics – see the documents at: https://docs.google.com/document/d/1xXu7gPKiblS3Vh4MCuK6NWfeRmMolXf9VF5sO7OG... and attached as a PDF.
a. 2.9.1 Community Applications
-- Does the WG want to affirm or otherwise address this PIRR recommendation? Recommendation 4.1.a: Consider all dimensions of the feedback received to revisit the CPE scoring and framework before the next application round. Comment from Jeff: Comment from Jeff Neuman: Technically I am note sure we are affirming this whole Guideline F. For example, I am not sure yet whether we will allow mutual agreement resolution...and the last part about giving the ICANN Board discretion, etc. The only thing we are affirming is the priority of communities in a contention set. COMMENT IS ADDRESSED.
-- Comment from Staff: Does the WG want to provide any detail here? Request from ICANN org: "It would be helpful if the PDP Working Group could provide more detailed guidance on the specific areas of the CPE process that “must be more transparent and predictable.” Additionally, it would be helpful if the PDP Working Group could provide more specific guidance on what should be changed or added that would enhance transparency and predictability for the CPE process."
-- Started a thread on changes to the CPE guidelines.
ACTION ITEM: Staff to send the updates to the CPE guidelines based on the WG discussions. DONE: See -- https://docs.google.com/document/d/1Ih_1NARViJXNNewDg-q87sQzQoC1dCtC/edit .
*Recommendation xx (rationale 4)*: ICANN must consider ways to improve evaluators’ ability to gather information about an application.
*Implementation Guidance xx (rationale 4)*: Evaluators should continue to be able to send clarifying questions to CPE applicants but further, should be able to engage in written dialogue with them as well.
*Implementation Guidance xx (rationale 4)*: Evaluators should be able to issue clarifying questions, or utilize similar methods to address potential issues, to those who submit letters of opposition to community-based applications.
*Discussion*:
-- Question: Did we get access to the actual research of the evaluators? Answer: We didn’t, but they did refer to research in their decisions. This was not consistent.
-- There should be no basis for an independent evaluator to do his or her own research. Worried about how this recommendation was developed.
-- Not all of the facts may be in the application, which is different from the UDRP. There should be research that may need to be done.
-- Shouldn’t that be on the complaining party to make their case?
-- We are saying that if there is independent research that is allowed then ICANN should improve evaluators ability to get information.
-- Perhaps it would be helpful to add to the recommendation “as described in the IG below” or “as set out in the Implementation Guidance below.”
-- It's okay for evaluators to find independent info - but they should disclose sources when ruling on the CPE.
-- The application should be robust and supported. And the public gets a chance to comment, right?
-- If you have support from major organizations, how does an evaluator know that they are real? They aren’t likely to be subject-matter experts.
-- If they are going to be doing research they could research who these organizations are to make sure they meet the requirements to support the application.
-- Anything that an evaluator is relying on should be disclosed to the applicant, particularly since we’ll have an appeal process. The applicant has to have the ability to challenge any independently developed information.
-- Trying to choose between two models 1) that the applicant should put together a robust application, encourages a robust public comment period, and discourages the evaluator from being in the role of a biased researcher 2) evaluators being inadvertently advocating for third parties that didn’t show up.
-- If we have all these criteria doesn’t it make sense that due diligence is done provided they give the applicant the ability to do clarifying questions in response. For example, the requirement for the uniqueness of the string would seem to be something that would require independent research. Also, documented support from the recognized community institutions.
-- Shouldn’t rely only on the information provided by the applicant.
-- The examiner should rely on a fulsome record, rather than create their own record. There is a requirement to gauge opposition from third parties. We should figure out a way to make them narrow, as with the Implementation Guidance -- maybe move those into the recommendation.
-- Don’t agree that we ban a panelist from doing outside research. Seems too extreme.
-- We are lacking notice to competitors. Need to let the examiners look at opposition.
-- Say, “In the event panelist(s) rely on independent research, that will be disclosed to the applicant through a Clarifying Question and the applicant should be allowed to respond prior to ruling. ”
-- If we require the evaluator to notify the applicant, isn’t the applicant just going to push back? The applicant should just have to provide a fulsome application.
-- If the applicant provides links/sources then at a minimum the evaluator can look at those links.
-- But how would an evaluator know that a term is commonly used in other industries without doing research. The Applicant is not going to put that into its own application.
-- Needs to be very clear that there isn’t just a request for a string with nothing to support it. It is one thing for the evaluator to confirm the information, but another for an evaluator to seek out opposition to it.
-- Need to remember that this is an evaluation -- in other evaluations in ICANN, say financial, does the evaluator only rely on what is provided by the applicant.
-- Add “except as needed for verification purposes and should be disclosed”. To verify the information.
-- Does the AGB currently allow evaluators to do research? Research is specifically noted. In paragraph 2 in 4.2.3. Bottom of the paragraph, pages 4-9. We are working on putting some guardrails around this.
-- Need to figure out what “if deemed necessary” means.
-- If we limit the ability of the evaluator to do research we should call that out in the public comment as it is a fairly significant change.
-- Important to encourage dialog, but can’t require it. If you have a community application that has overwhelming support, those who are in opposition should step up and encouraged to enter into a dialog.
-- The amount of opposition should be compared to the amount of support. 5 letters of opposition might be irrelevant - if you have 500 letters of support. 1 or 2 letters shouldn't automatically cost a point. Opposition should be weighed against support.
-- There should be a public and transparent verification of an organization stating opposition in a letter.
*ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Jamie Baxter also should consider what should be added in order to encourage dialog, and public and transparent verification of statements of opposition.*
*Re: The Working Group notes that CCT-RT Recommendation 34 states: “A thorough review of the procedures and objectives for community based applications should be carried out and improvements made to address and correct the concerns raised before a new gTLD application process is launched. Revisions or adjustments should be clearly reflected in an updated version of the 2012 AGB.” *
-- Staff Comment: From the CCT-RT recommendations tracking spreadsheet: As the WG refines recommendations on this topic, it may want to consider whether to make additional recommendations regarding objectives. Note that the CCT-RT recommendations consider "a higher rate of success for such applications" to be a measure of success.
-- Are we trying to get a higher success rate and what can be done other than the recommendations we’ve already made?
-- Need to have a requirement that the evaluators are knowledgeable about the community. Need to avoid the mistakes from the previous round.
-- We do have a section on accountability and appeals -- in the section on Accountability Mechanisms.
-- The qualifications of the panelists is not something we spent a lot of time on. Need to see if we can improve that.
-- Need quantitative and qualitative assessment by experts in communities.
*ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities.* _______________________________________________ Gnso-newgtld-wg mailing list Gnso-newgtld-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-newgtld-wg _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
+1 Justine Jamie From: Gnso-newgtld-wg <gnso-newgtld-wg-bounces@icann.org> on behalf of Justine Chew <justine.chew@gmail.com> Date: Wednesday, March 18, 2020 at 6:44 AM To: "gnso-newgtld-wg@icann.org" <gnso-newgtld-wg@icann.org> Subject: Re: [Gnso-newgtld-wg] Updated Subject Line: Notes and Action Items - New gTLD Subsequent Procedures PDP WG - 16 March 1500 UTC Thanks, Julie, for these notes. All, I regret not being able to join Monday's call. I've just listened to the recording and found the discussions encouraging. I wanted to propose something additional for consideration, which is that perhaps, preceding a recommendation re: Qualification of Panellists (as noted below), there should be a recommendation requiring ICANN Org to be more transparent in its process of selecting a community priority evaluation provider/panel. Rationale: I don't think I would be wrong to say there have been made comments regarding the suitability of Economic Intelligence Unit as the CPE provider and perhaps to help ensure CPE panellists have suitable qualifications, an antecedent step would be to allow ICANN community consideration and intervention to take place in ICANN Org's selection of the CPE provider. Thanks, Justine ----- On Tue, 17 Mar 2020 at 01:43, Julie Hedlund <julie.hedlund@icann.org<mailto:julie.hedlund@icann.org>> wrote: With Correct Subject Line Dear Working Group members, Please see below the notes from the meeting on 16 March at 1500 UTC. These high-level notes are designed to help WG members navigate through the content of the call and are not a substitute for the recording, transcript, or the chat, which will be posted at: https://community.icann.org/display/NGSPP/2020-03-16+New+gTLD+Subsequent+Pro.... Kind regards, Julie Notes and Action Items: Actions: Recommendation xx (rationale 4): ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Re: Qualifications of Panelists: ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. Notes: 1. Updates to Statements of Interest: No updates provided. 2. Discussion of Final Report Topics – see the documents at: https://docs.google.com/document/d/1xXu7gPKiblS3Vh4MCuK6NWfeRmMolXf9VF5sO7OG... and attached as a PDF. a. 2.9.1 Community Applications -- Does the WG want to affirm or otherwise address this PIRR recommendation? Recommendation 4.1.a: Consider all dimensions of the feedback received to revisit the CPE scoring and framework before the next application round. Comment from Jeff: Comment from Jeff Neuman: Technically I am note sure we are affirming this whole Guideline F. For example, I am not sure yet whether we will allow mutual agreement resolution...and the last part about giving the ICANN Board discretion, etc. The only thing we are affirming is the priority of communities in a contention set. COMMENT IS ADDRESSED. -- Comment from Staff: Does the WG want to provide any detail here? Request from ICANN org: "It would be helpful if the PDP Working Group could provide more detailed guidance on the specific areas of the CPE process that “must be more transparent and predictable.” Additionally, it would be helpful if the PDP Working Group could provide more specific guidance on what should be changed or added that would enhance transparency and predictability for the CPE process." -- Started a thread on changes to the CPE guidelines. ACTION ITEM: Staff to send the updates to the CPE guidelines based on the WG discussions. DONE: See -- https://docs.google.com/document/d/1Ih_1NARViJXNNewDg-q87sQzQoC1dCtC/edit. Recommendation xx (rationale 4): ICANN must consider ways to improve evaluators’ ability to gather information about an application. Implementation Guidance xx (rationale 4): Evaluators should continue to be able to send clarifying questions to CPE applicants but further, should be able to engage in written dialogue with them as well. Implementation Guidance xx (rationale 4): Evaluators should be able to issue clarifying questions, or utilize similar methods to address potential issues, to those who submit letters of opposition to community-based applications. Discussion: -- Question: Did we get access to the actual research of the evaluators? Answer: We didn’t, but they did refer to research in their decisions. This was not consistent. -- There should be no basis for an independent evaluator to do his or her own research. Worried about how this recommendation was developed. -- Not all of the facts may be in the application, which is different from the UDRP. There should be research that may need to be done. -- Shouldn’t that be on the complaining party to make their case? -- We are saying that if there is independent research that is allowed then ICANN should improve evaluators ability to get information. -- Perhaps it would be helpful to add to the recommendation “as described in the IG below” or “as set out in the Implementation Guidance below.” -- It's okay for evaluators to find independent info - but they should disclose sources when ruling on the CPE. -- The application should be robust and supported. And the public gets a chance to comment, right? -- If you have support from major organizations, how does an evaluator know that they are real? They aren’t likely to be subject-matter experts. -- If they are going to be doing research they could research who these organizations are to make sure they meet the requirements to support the application. -- Anything that an evaluator is relying on should be disclosed to the applicant, particularly since we’ll have an appeal process. The applicant has to have the ability to challenge any independently developed information. -- Trying to choose between two models 1) that the applicant should put together a robust application, encourages a robust public comment period, and discourages the evaluator from being in the role of a biased researcher 2) evaluators being inadvertently advocating for third parties that didn’t show up. -- If we have all these criteria doesn’t it make sense that due diligence is done provided they give the applicant the ability to do clarifying questions in response. For example, the requirement for the uniqueness of the string would seem to be something that would require independent research. Also, documented support from the recognized community institutions. -- Shouldn’t rely only on the information provided by the applicant. -- The examiner should rely on a fulsome record, rather than create their own record. There is a requirement to gauge opposition from third parties. We should figure out a way to make them narrow, as with the Implementation Guidance -- maybe move those into the recommendation. -- Don’t agree that we ban a panelist from doing outside research. Seems too extreme. -- We are lacking notice to competitors. Need to let the examiners look at opposition. -- Say, “In the event panelist(s) rely on independent research, that will be disclosed to the applicant through a Clarifying Question and the applicant should be allowed to respond prior to ruling. ” -- If we require the evaluator to notify the applicant, isn’t the applicant just going to push back? The applicant should just have to provide a fulsome application. -- If the applicant provides links/sources then at a minimum the evaluator can look at those links. -- But how would an evaluator know that a term is commonly used in other industries without doing research. The Applicant is not going to put that into its own application. -- Needs to be very clear that there isn’t just a request for a string with nothing to support it. It is one thing for the evaluator to confirm the information, but another for an evaluator to seek out opposition to it. -- Need to remember that this is an evaluation -- in other evaluations in ICANN, say financial, does the evaluator only rely on what is provided by the applicant. -- Add “except as needed for verification purposes and should be disclosed”. To verify the information. -- Does the AGB currently allow evaluators to do research? Research is specifically noted. In paragraph 2 in 4.2.3. Bottom of the paragraph, pages 4-9. We are working on putting some guardrails around this. -- Need to figure out what “if deemed necessary” means. -- If we limit the ability of the evaluator to do research we should call that out in the public comment as it is a fairly significant change. -- Important to encourage dialog, but can’t require it. If you have a community application that has overwhelming support, those who are in opposition should step up and encouraged to enter into a dialog. -- The amount of opposition should be compared to the amount of support. 5 letters of opposition might be irrelevant - if you have 500 letters of support. 1 or 2 letters shouldn't automatically cost a point. Opposition should be weighed against support. -- There should be a public and transparent verification of an organization stating opposition in a letter. ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Jamie Baxter also should consider what should be added in order to encourage dialog, and public and transparent verification of statements of opposition. Re: The Working Group notes that CCT-RT Recommendation 34 states: “A thorough review of the procedures and objectives for community based applications should be carried out and improvements made to address and correct the concerns raised before a new gTLD application process is launched. Revisions or adjustments should be clearly reflected in an updated version of the 2012 AGB.” -- Staff Comment: From the CCT-RT recommendations tracking spreadsheet: As the WG refines recommendations on this topic, it may want to consider whether to make additional recommendations regarding objectives. Note that the CCT-RT recommendations consider "a higher rate of success for such applications" to be a measure of success. -- Are we trying to get a higher success rate and what can be done other than the recommendations we’ve already made? -- Need to have a requirement that the evaluators are knowledgeable about the community. Need to avoid the mistakes from the previous round. -- We do have a section on accountability and appeals -- in the section on Accountability Mechanisms. -- The qualifications of the panelists is not something we spent a lot of time on. Need to see if we can improve that. -- Need quantitative and qualitative assessment by experts in communities. ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. _______________________________________________ Gnso-newgtld-wg mailing list Gnso-newgtld-wg@icann.org<mailto:Gnso-newgtld-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-newgtld-wg _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
Makes sense! Jorge ________________________________ Von: Jamie Baxter <jbaxter@spimarketing.com> Datum: 18. März 2020 um 15:26:14 MEZ An: Justine Chew <justine.chew@gmail.com>, gnso-newgtld-wg@icann.org <gnso-newgtld-wg@icann.org> Betreff: Re: [Gnso-newgtld-wg] Updated Subject Line: Notes and Action Items - New gTLD Subsequent Procedures PDP WG - 16 March 1500 UTC +1 Justine Jamie From: Gnso-newgtld-wg <gnso-newgtld-wg-bounces@icann.org> on behalf of Justine Chew <justine.chew@gmail.com> Date: Wednesday, March 18, 2020 at 6:44 AM To: "gnso-newgtld-wg@icann.org" <gnso-newgtld-wg@icann.org> Subject: Re: [Gnso-newgtld-wg] Updated Subject Line: Notes and Action Items - New gTLD Subsequent Procedures PDP WG - 16 March 1500 UTC Thanks, Julie, for these notes. All, I regret not being able to join Monday's call. I've just listened to the recording and found the discussions encouraging. I wanted to propose something additional for consideration, which is that perhaps, preceding a recommendation re: Qualification of Panellists (as noted below), there should be a recommendation requiring ICANN Org to be more transparent in its process of selecting a community priority evaluation provider/panel. Rationale: I don't think I would be wrong to say there have been made comments regarding the suitability of Economic Intelligence Unit as the CPE provider and perhaps to help ensure CPE panellists have suitable qualifications, an antecedent step would be to allow ICANN community consideration and intervention to take place in ICANN Org's selection of the CPE provider. Thanks, Justine ----- On Tue, 17 Mar 2020 at 01:43, Julie Hedlund <julie.hedlund@icann.org<mailto:julie.hedlund@icann.org>> wrote: With Correct Subject Line Dear Working Group members, Please see below the notes from the meeting on 16 March at 1500 UTC. These high-level notes are designed to help WG members navigate through the content of the call and are not a substitute for the recording, transcript, or the chat, which will be posted at: https://community.icann.org/display/NGSPP/2020-03-16+New+gTLD+Subsequent+Pro.... Kind regards, Julie Notes and Action Items: Actions: Recommendation xx (rationale 4): ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Re: Qualifications of Panelists: ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. Notes: 1. Updates to Statements of Interest: No updates provided. 2. Discussion of Final Report Topics – see the documents at: https://docs.google.com/document/d/1xXu7gPKiblS3Vh4MCuK6NWfeRmMolXf9VF5sO7OG... and attached as a PDF. a. 2.9.1 Community Applications -- Does the WG want to affirm or otherwise address this PIRR recommendation? Recommendation 4.1.a: Consider all dimensions of the feedback received to revisit the CPE scoring and framework before the next application round. Comment from Jeff: Comment from Jeff Neuman: Technically I am note sure we are affirming this whole Guideline F. For example, I am not sure yet whether we will allow mutual agreement resolution...and the last part about giving the ICANN Board discretion, etc. The only thing we are affirming is the priority of communities in a contention set. COMMENT IS ADDRESSED. -- Comment from Staff: Does the WG want to provide any detail here? Request from ICANN org: "It would be helpful if the PDP Working Group could provide more detailed guidance on the specific areas of the CPE process that “must be more transparent and predictable.” Additionally, it would be helpful if the PDP Working Group could provide more specific guidance on what should be changed or added that would enhance transparency and predictability for the CPE process." -- Started a thread on changes to the CPE guidelines. ACTION ITEM: Staff to send the updates to the CPE guidelines based on the WG discussions. DONE: See -- https://docs.google.com/document/d/1Ih_1NARViJXNNewDg-q87sQzQoC1dCtC/edit. Recommendation xx (rationale 4): ICANN must consider ways to improve evaluators’ ability to gather information about an application. Implementation Guidance xx (rationale 4): Evaluators should continue to be able to send clarifying questions to CPE applicants but further, should be able to engage in written dialogue with them as well. Implementation Guidance xx (rationale 4): Evaluators should be able to issue clarifying questions, or utilize similar methods to address potential issues, to those who submit letters of opposition to community-based applications. Discussion: -- Question: Did we get access to the actual research of the evaluators? Answer: We didn’t, but they did refer to research in their decisions. This was not consistent. -- There should be no basis for an independent evaluator to do his or her own research. Worried about how this recommendation was developed. -- Not all of the facts may be in the application, which is different from the UDRP. There should be research that may need to be done. -- Shouldn’t that be on the complaining party to make their case? -- We are saying that if there is independent research that is allowed then ICANN should improve evaluators ability to get information. -- Perhaps it would be helpful to add to the recommendation “as described in the IG below” or “as set out in the Implementation Guidance below.” -- It's okay for evaluators to find independent info - but they should disclose sources when ruling on the CPE. -- The application should be robust and supported. And the public gets a chance to comment, right? -- If you have support from major organizations, how does an evaluator know that they are real? They aren’t likely to be subject-matter experts. -- If they are going to be doing research they could research who these organizations are to make sure they meet the requirements to support the application. -- Anything that an evaluator is relying on should be disclosed to the applicant, particularly since we’ll have an appeal process. The applicant has to have the ability to challenge any independently developed information. -- Trying to choose between two models 1) that the applicant should put together a robust application, encourages a robust public comment period, and discourages the evaluator from being in the role of a biased researcher 2) evaluators being inadvertently advocating for third parties that didn’t show up. -- If we have all these criteria doesn’t it make sense that due diligence is done provided they give the applicant the ability to do clarifying questions in response. For example, the requirement for the uniqueness of the string would seem to be something that would require independent research. Also, documented support from the recognized community institutions. -- Shouldn’t rely only on the information provided by the applicant. -- The examiner should rely on a fulsome record, rather than create their own record. There is a requirement to gauge opposition from third parties. We should figure out a way to make them narrow, as with the Implementation Guidance -- maybe move those into the recommendation. -- Don’t agree that we ban a panelist from doing outside research. Seems too extreme. -- We are lacking notice to competitors. Need to let the examiners look at opposition. -- Say, “In the event panelist(s) rely on independent research, that will be disclosed to the applicant through a Clarifying Question and the applicant should be allowed to respond prior to ruling. ” -- If we require the evaluator to notify the applicant, isn’t the applicant just going to push back? The applicant should just have to provide a fulsome application. -- If the applicant provides links/sources then at a minimum the evaluator can look at those links. -- But how would an evaluator know that a term is commonly used in other industries without doing research. The Applicant is not going to put that into its own application. -- Needs to be very clear that there isn’t just a request for a string with nothing to support it. It is one thing for the evaluator to confirm the information, but another for an evaluator to seek out opposition to it. -- Need to remember that this is an evaluation -- in other evaluations in ICANN, say financial, does the evaluator only rely on what is provided by the applicant. -- Add “except as needed for verification purposes and should be disclosed”. To verify the information. -- Does the AGB currently allow evaluators to do research? Research is specifically noted. In paragraph 2 in 4.2.3. Bottom of the paragraph, pages 4-9. We are working on putting some guardrails around this. -- Need to figure out what “if deemed necessary” means. -- If we limit the ability of the evaluator to do research we should call that out in the public comment as it is a fairly significant change. -- Important to encourage dialog, but can’t require it. If you have a community application that has overwhelming support, those who are in opposition should step up and encouraged to enter into a dialog. -- The amount of opposition should be compared to the amount of support. 5 letters of opposition might be irrelevant - if you have 500 letters of support. 1 or 2 letters shouldn't automatically cost a point. Opposition should be weighed against support. -- There should be a public and transparent verification of an organization stating opposition in a letter. ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Jamie Baxter also should consider what should be added in order to encourage dialog, and public and transparent verification of statements of opposition. Re: The Working Group notes that CCT-RT Recommendation 34 states: “A thorough review of the procedures and objectives for community based applications should be carried out and improvements made to address and correct the concerns raised before a new gTLD application process is launched. Revisions or adjustments should be clearly reflected in an updated version of the 2012 AGB.” -- Staff Comment: From the CCT-RT recommendations tracking spreadsheet: As the WG refines recommendations on this topic, it may want to consider whether to make additional recommendations regarding objectives. Note that the CCT-RT recommendations consider "a higher rate of success for such applications" to be a measure of success. -- Are we trying to get a higher success rate and what can be done other than the recommendations we’ve already made? -- Need to have a requirement that the evaluators are knowledgeable about the community. Need to avoid the mistakes from the previous round. -- We do have a section on accountability and appeals -- in the section on Accountability Mechanisms. -- The qualifications of the panelists is not something we spent a lot of time on. Need to see if we can improve that. -- Need quantitative and qualitative assessment by experts in communities. ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. _______________________________________________ Gnso-newgtld-wg mailing list Gnso-newgtld-wg@icann.org<mailto:Gnso-newgtld-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-newgtld-wg _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
+1 Justine! From: Gnso-newgtld-wg [mailto:gnso-newgtld-wg-bounces@icann.org] On Behalf Of Justine Chew Sent: Mittwoch, 18. März 2020 06:44 To: gnso-newgtld-wg@icann.org Subject: Re: [Gnso-newgtld-wg] Updated Subject Line: Notes and Action Items - New gTLD Subsequent Procedures PDP WG - 16 March 1500 UTC Thanks, Julie, for these notes. All, I regret not being able to join Monday's call. I've just listened to the recording and found the discussions encouraging. I wanted to propose something additional for consideration, which is that perhaps, preceding a recommendation re: Qualification of Panellists (as noted below), there should be a recommendation requiring ICANN Org to be more transparent in its process of selecting a community priority evaluation provider/panel. Rationale: I don't think I would be wrong to say there have been made comments regarding the suitability of Economic Intelligence Unit as the CPE provider and perhaps to help ensure CPE panellists have suitable qualifications, an antecedent step would be to allow ICANN community consideration and intervention to take place in ICANN Org's selection of the CPE provider. Thanks, Justine ----- On Tue, 17 Mar 2020 at 01:43, Julie Hedlund <julie.hedlund@icann.org <mailto:julie.hedlund@icann.org> > wrote: With Correct Subject Line Dear Working Group members, Please see below the notes from the meeting on 16 March at 1500 UTC. These high-level notes are designed to help WG members navigate through the content of the call and are not a substitute for the recording, transcript, or the chat, which will be posted at: https://community.icann.org/display/NGSPP/2020-03-16+New+gTLD+Subsequent+Pro.... Kind regards, Julie Notes and Action Items: Actions: Recommendation xx (rationale 4): ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Re: Qualifications of Panelists: ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. Notes: 1. Updates to Statements of Interest: No updates provided. 2. Discussion of Final Report Topics – see the documents at: https://docs.google.com/document/d/1xXu7gPKiblS3Vh4MCuK6NWfeRmMolXf9VF5sO7OG... and attached as a PDF. a. 2.9.1 Community Applications -- Does the WG want to affirm or otherwise address this PIRR recommendation? Recommendation 4.1.a: Consider all dimensions of the feedback received to revisit the CPE scoring and framework before the next application round. Comment from Jeff: Comment from Jeff Neuman: Technically I am note sure we are affirming this whole Guideline F. For example, I am not sure yet whether we will allow mutual agreement resolution...and the last part about giving the ICANN Board discretion, etc. The only thing we are affirming is the priority of communities in a contention set. COMMENT IS ADDRESSED. -- Comment from Staff: Does the WG want to provide any detail here? Request from ICANN org: "It would be helpful if the PDP Working Group could provide more detailed guidance on the specific areas of the CPE process that “must be more transparent and predictable.” Additionally, it would be helpful if the PDP Working Group could provide more specific guidance on what should be changed or added that would enhance transparency and predictability for the CPE process." -- Started a thread on changes to the CPE guidelines. ACTION ITEM: Staff to send the updates to the CPE guidelines based on the WG discussions. DONE: See -- https://docs.google.com/document/d/1Ih_1NARViJXNNewDg-q87sQzQoC1dCtC/edit. Recommendation xx (rationale 4): ICANN must consider ways to improve evaluators’ ability to gather information about an application. Implementation Guidance xx (rationale 4): Evaluators should continue to be able to send clarifying questions to CPE applicants but further, should be able to engage in written dialogue with them as well. Implementation Guidance xx (rationale 4): Evaluators should be able to issue clarifying questions, or utilize similar methods to address potential issues, to those who submit letters of opposition to community-based applications. Discussion: -- Question: Did we get access to the actual research of the evaluators? Answer: We didn’t, but they did refer to research in their decisions. This was not consistent. -- There should be no basis for an independent evaluator to do his or her own research. Worried about how this recommendation was developed. -- Not all of the facts may be in the application, which is different from the UDRP. There should be research that may need to be done. -- Shouldn’t that be on the complaining party to make their case? -- We are saying that if there is independent research that is allowed then ICANN should improve evaluators ability to get information. -- Perhaps it would be helpful to add to the recommendation “as described in the IG below” or “as set out in the Implementation Guidance below.” -- It's okay for evaluators to find independent info - but they should disclose sources when ruling on the CPE. -- The application should be robust and supported. And the public gets a chance to comment, right? -- If you have support from major organizations, how does an evaluator know that they are real? They aren’t likely to be subject-matter experts. -- If they are going to be doing research they could research who these organizations are to make sure they meet the requirements to support the application. -- Anything that an evaluator is relying on should be disclosed to the applicant, particularly since we’ll have an appeal process. The applicant has to have the ability to challenge any independently developed information. -- Trying to choose between two models 1) that the applicant should put together a robust application, encourages a robust public comment period, and discourages the evaluator from being in the role of a biased researcher 2) evaluators being inadvertently advocating for third parties that didn’t show up. -- If we have all these criteria doesn’t it make sense that due diligence is done provided they give the applicant the ability to do clarifying questions in response. For example, the requirement for the uniqueness of the string would seem to be something that would require independent research. Also, documented support from the recognized community institutions. -- Shouldn’t rely only on the information provided by the applicant. -- The examiner should rely on a fulsome record, rather than create their own record. There is a requirement to gauge opposition from third parties. We should figure out a way to make them narrow, as with the Implementation Guidance -- maybe move those into the recommendation. -- Don’t agree that we ban a panelist from doing outside research. Seems too extreme. -- We are lacking notice to competitors. Need to let the examiners look at opposition. -- Say, “In the event panelist(s) rely on independent research, that will be disclosed to the applicant through a Clarifying Question and the applicant should be allowed to respond prior to ruling. ” -- If we require the evaluator to notify the applicant, isn’t the applicant just going to push back? The applicant should just have to provide a fulsome application. -- If the applicant provides links/sources then at a minimum the evaluator can look at those links. -- But how would an evaluator know that a term is commonly used in other industries without doing research. The Applicant is not going to put that into its own application. -- Needs to be very clear that there isn’t just a request for a string with nothing to support it. It is one thing for the evaluator to confirm the information, but another for an evaluator to seek out opposition to it. -- Need to remember that this is an evaluation -- in other evaluations in ICANN, say financial, does the evaluator only rely on what is provided by the applicant. -- Add “except as needed for verification purposes and should be disclosed”. To verify the information. -- Does the AGB currently allow evaluators to do research? Research is specifically noted. In paragraph 2 in 4.2.3. Bottom of the paragraph, pages 4-9. We are working on putting some guardrails around this. -- Need to figure out what “if deemed necessary” means. -- If we limit the ability of the evaluator to do research we should call that out in the public comment as it is a fairly significant change. -- Important to encourage dialog, but can’t require it. If you have a community application that has overwhelming support, those who are in opposition should step up and encouraged to enter into a dialog. -- The amount of opposition should be compared to the amount of support. 5 letters of opposition might be irrelevant - if you have 500 letters of support. 1 or 2 letters shouldn't automatically cost a point. Opposition should be weighed against support. -- There should be a public and transparent verification of an organization stating opposition in a letter. ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Jamie Baxter also should consider what should be added in order to encourage dialog, and public and transparent verification of statements of opposition. Re: The Working Group notes that CCT-RT Recommendation 34 states: “A thorough review of the procedures and objectives for community based applications should be carried out and improvements made to address and correct the concerns raised before a new gTLD application process is launched. Revisions or adjustments should be clearly reflected in an updated version of the 2012 AGB.” -- Staff Comment: From the CCT-RT recommendations tracking spreadsheet: As the WG refines recommendations on this topic, it may want to consider whether to make additional recommendations regarding objectives. Note that the CCT-RT recommendations consider "a higher rate of success for such applications" to be a measure of success. -- Are we trying to get a higher success rate and what can be done other than the recommendations we’ve already made? -- Need to have a requirement that the evaluators are knowledgeable about the community. Need to avoid the mistakes from the previous round. -- We do have a section on accountability and appeals -- in the section on Accountability Mechanisms. -- The qualifications of the panelists is not something we spent a lot of time on. Need to see if we can improve that. -- Need quantitative and qualitative assessment by experts in communities. ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. _______________________________________________ Gnso-newgtld-wg mailing list Gnso-newgtld-wg@icann.org <mailto:Gnso-newgtld-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-newgtld-wg _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
Thanks Justine. The selection process for Evaluators is documents here: https://newgtlds.icann.org/en/about/evaluation-panels-selection-process Other than “allow[ing] ICANN community consideration and intervention to take place in ICANN Org's selection of the CPE provider,” what other steps would you propose ICANN takes to become “more transparent?” I think we need to be fairly specific in our recommendations. Thanks! Jeff Neuman Senior Vice President Com Laude | Valideus D: +1.703.635.7514 E: jeff.neuman@comlaude.com<mailto:jeff.neuman@comlaude.com> From: Gnso-newgtld-wg <gnso-newgtld-wg-bounces@icann.org> On Behalf Of Justine Chew Sent: Wednesday, March 18, 2020 6:44 AM To: gnso-newgtld-wg@icann.org Subject: Re: [Gnso-newgtld-wg] Updated Subject Line: Notes and Action Items - New gTLD Subsequent Procedures PDP WG - 16 March 1500 UTC Thanks, Julie, for these notes. All, I regret not being able to join Monday's call. I've just listened to the recording and found the discussions encouraging. I wanted to propose something additional for consideration, which is that perhaps, preceding a recommendation re: Qualification of Panellists (as noted below), there should be a recommendation requiring ICANN Org to be more transparent in its process of selecting a community priority evaluation provider/panel. Rationale: I don't think I would be wrong to say there have been made comments regarding the suitability of Economic Intelligence Unit as the CPE provider and perhaps to help ensure CPE panellists have suitable qualifications, an antecedent step would be to allow ICANN community consideration and intervention to take place in ICANN Org's selection of the CPE provider. Thanks, Justine ----- On Tue, 17 Mar 2020 at 01:43, Julie Hedlund <julie.hedlund@icann.org<mailto:julie.hedlund@icann.org>> wrote: With Correct Subject Line Dear Working Group members, Please see below the notes from the meeting on 16 March at 1500 UTC. These high-level notes are designed to help WG members navigate through the content of the call and are not a substitute for the recording, transcript, or the chat, which will be posted at: https://community.icann.org/display/NGSPP/2020-03-16+New+gTLD+Subsequent+Pro.... Kind regards, Julie Notes and Action Items: Actions: Recommendation xx (rationale 4): ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Re: Qualifications of Panelists: ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. Notes: 1. Updates to Statements of Interest: No updates provided. 2. Discussion of Final Report Topics – see the documents at: https://docs.google.com/document/d/1xXu7gPKiblS3Vh4MCuK6NWfeRmMolXf9VF5sO7OG... and attached as a PDF. a. 2.9.1 Community Applications -- Does the WG want to affirm or otherwise address this PIRR recommendation? Recommendation 4.1.a: Consider all dimensions of the feedback received to revisit the CPE scoring and framework before the next application round. Comment from Jeff: Comment from Jeff Neuman: Technically I am note sure we are affirming this whole Guideline F. For example, I am not sure yet whether we will allow mutual agreement resolution...and the last part about giving the ICANN Board discretion, etc. The only thing we are affirming is the priority of communities in a contention set. COMMENT IS ADDRESSED. -- Comment from Staff: Does the WG want to provide any detail here? Request from ICANN org: "It would be helpful if the PDP Working Group could provide more detailed guidance on the specific areas of the CPE process that “must be more transparent and predictable.” Additionally, it would be helpful if the PDP Working Group could provide more specific guidance on what should be changed or added that would enhance transparency and predictability for the CPE process." -- Started a thread on changes to the CPE guidelines. ACTION ITEM: Staff to send the updates to the CPE guidelines based on the WG discussions. DONE: See -- https://docs.google.com/document/d/1Ih_1NARViJXNNewDg-q87sQzQoC1dCtC/edit. Recommendation xx (rationale 4): ICANN must consider ways to improve evaluators’ ability to gather information about an application. Implementation Guidance xx (rationale 4): Evaluators should continue to be able to send clarifying questions to CPE applicants but further, should be able to engage in written dialogue with them as well. Implementation Guidance xx (rationale 4): Evaluators should be able to issue clarifying questions, or utilize similar methods to address potential issues, to those who submit letters of opposition to community-based applications. Discussion: -- Question: Did we get access to the actual research of the evaluators? Answer: We didn’t, but they did refer to research in their decisions. This was not consistent. -- There should be no basis for an independent evaluator to do his or her own research. Worried about how this recommendation was developed. -- Not all of the facts may be in the application, which is different from the UDRP. There should be research that may need to be done. -- Shouldn’t that be on the complaining party to make their case? -- We are saying that if there is independent research that is allowed then ICANN should improve evaluators ability to get information. -- Perhaps it would be helpful to add to the recommendation “as described in the IG below” or “as set out in the Implementation Guidance below.” -- It's okay for evaluators to find independent info - but they should disclose sources when ruling on the CPE. -- The application should be robust and supported. And the public gets a chance to comment, right? -- If you have support from major organizations, how does an evaluator know that they are real? They aren’t likely to be subject-matter experts. -- If they are going to be doing research they could research who these organizations are to make sure they meet the requirements to support the application. -- Anything that an evaluator is relying on should be disclosed to the applicant, particularly since we’ll have an appeal process. The applicant has to have the ability to challenge any independently developed information. -- Trying to choose between two models 1) that the applicant should put together a robust application, encourages a robust public comment period, and discourages the evaluator from being in the role of a biased researcher 2) evaluators being inadvertently advocating for third parties that didn’t show up. -- If we have all these criteria doesn’t it make sense that due diligence is done provided they give the applicant the ability to do clarifying questions in response. For example, the requirement for the uniqueness of the string would seem to be something that would require independent research. Also, documented support from the recognized community institutions. -- Shouldn’t rely only on the information provided by the applicant. -- The examiner should rely on a fulsome record, rather than create their own record. There is a requirement to gauge opposition from third parties. We should figure out a way to make them narrow, as with the Implementation Guidance -- maybe move those into the recommendation. -- Don’t agree that we ban a panelist from doing outside research. Seems too extreme. -- We are lacking notice to competitors. Need to let the examiners look at opposition. -- Say, “In the event panelist(s) rely on independent research, that will be disclosed to the applicant through a Clarifying Question and the applicant should be allowed to respond prior to ruling. ” -- If we require the evaluator to notify the applicant, isn’t the applicant just going to push back? The applicant should just have to provide a fulsome application. -- If the applicant provides links/sources then at a minimum the evaluator can look at those links. -- But how would an evaluator know that a term is commonly used in other industries without doing research. The Applicant is not going to put that into its own application. -- Needs to be very clear that there isn’t just a request for a string with nothing to support it. It is one thing for the evaluator to confirm the information, but another for an evaluator to seek out opposition to it. -- Need to remember that this is an evaluation -- in other evaluations in ICANN, say financial, does the evaluator only rely on what is provided by the applicant. -- Add “except as needed for verification purposes and should be disclosed”. To verify the information. -- Does the AGB currently allow evaluators to do research? Research is specifically noted. In paragraph 2 in 4.2.3. Bottom of the paragraph, pages 4-9. We are working on putting some guardrails around this. -- Need to figure out what “if deemed necessary” means. -- If we limit the ability of the evaluator to do research we should call that out in the public comment as it is a fairly significant change. -- Important to encourage dialog, but can’t require it. If you have a community application that has overwhelming support, those who are in opposition should step up and encouraged to enter into a dialog. -- The amount of opposition should be compared to the amount of support. 5 letters of opposition might be irrelevant - if you have 500 letters of support. 1 or 2 letters shouldn't automatically cost a point. Opposition should be weighed against support. -- There should be a public and transparent verification of an organization stating opposition in a letter. ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Jamie Baxter also should consider what should be added in order to encourage dialog, and public and transparent verification of statements of opposition. Re: The Working Group notes that CCT-RT Recommendation 34 states: “A thorough review of the procedures and objectives for community based applications should be carried out and improvements made to address and correct the concerns raised before a new gTLD application process is launched. Revisions or adjustments should be clearly reflected in an updated version of the 2012 AGB.” -- Staff Comment: From the CCT-RT recommendations tracking spreadsheet: As the WG refines recommendations on this topic, it may want to consider whether to make additional recommendations regarding objectives. Note that the CCT-RT recommendations consider "a higher rate of success for such applications" to be a measure of success. -- Are we trying to get a higher success rate and what can be done other than the recommendations we’ve already made? -- Need to have a requirement that the evaluators are knowledgeable about the community. Need to avoid the mistakes from the previous round. -- We do have a section on accountability and appeals -- in the section on Accountability Mechanisms. -- The qualifications of the panelists is not something we spent a lot of time on. Need to see if we can improve that. -- Need quantitative and qualitative assessment by experts in communities. ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities. _______________________________________________ Gnso-newgtld-wg mailing list Gnso-newgtld-wg@icann.org<mailto:Gnso-newgtld-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-newgtld-wg _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. ________________________________ The contents of this email and any attachments are confidential to the intended recipient. They may not be disclosed, used by or copied in any way by anyone other than the intended recipient. If you have received this message in error, please return it to the sender (deleting the body of the email and attachments in your reply) and immediately and permanently delete it. Please note that the Com Laude Group does not accept any responsibility for viruses and it is your responsibility to scan or otherwise check this email and any attachments. The Com Laude Group does not accept liability for statements which are clearly the sender's own and not made on behalf of the group or one of its member entities. The Com Laude Group includes Nom-IQ Limited t/a Com Laude, a company registered in England and Wales with company number 5047655 and registered office at 28-30 Little Russell Street, London, WC1A 2HN England; Valideus Limited, a company registered in England and Wales with company number 06181291 and registered office at 28-30 Little Russell Street, London, WC1A 2HN England; Demys Limited, a company registered in Scotland with company number SC197176, having its registered office at 33 Melville Street, Edinburgh, Lothian, EH3 7JF Scotland; Consonum, Inc. dba Com Laude USA and Valideus USA, headquartered at 1751 Pinnacle Drive, Suite 600, McLean, VA 22102, USA; Com Laude (Japan) Corporation, a company registered in Japan having its registered office at Suite 319,1-3-21 Shinkawa, Chuo-ku, Tokyo, 104-0033, Japan. For further information see www.comlaude.com<https://comlaude.com>
Hi Jeff, Happy to oblige. And I will concentrate my response on the *Community Priority Evaluation Panel — formerly the Comparative Evaluation Panel* <http://www.icann.org/en/topics/new-gtlds/eoi-commun-priority-31jul09-en.pdf> for now since we're on the topic of Community Applications / CPE and given that I think suitability of the CPE provider is by far the hardest one to pin down as compared to the more easily identifiable expertise that is logically associated with the other 3 panels of (1) Applicant Evaluation Teams (Technical and Financial Evaluation), (2) Geographic Name Evaluation, and (3) String Similarity Examiners. At this juncture, recollection of the steps undertaken by ICANN Org in the Evaluation Panels Selection Process for the 2012 round eludes me. 1. Was the *Community Priority Evaluation Panel — formerly the Comparative Evaluation Panel* <http://www.icann.org/en/topics/new-gtlds/eoi-commun-priority-31jul09-en.pdf> which accompanied the relevant calls for EOI in 2009 put up for public comment prior to its use? - If not, then a public comment process should be factored in for purposes of ICANN Org adopting such a document for EOI call, especially section 4 on Criteria (noting that other sections would be subject to update to match a revised AGB) - If yes, then that public comment process should be retained. 2. Was there opportunity (and known mechanism) for the ICANN community to comment on the adoption and use of the general criteria scoring by ICANN Org, especially item 6 in the General Criteria Matrix at https://www.icann.org/news/announcement-2009-10-25-en to identify a short list of candidates? 3. Was there opportunity (and known mechanism) for the ICANN community to raise concerns over the short list of candidates generated by ICANN Org? Depending on what the answers are, a recommendation addressing one or more of the above points could be included in our working document along the lines of "If not, then propose to be included ..." or "If yes, then to retain ..." as the case may be. Hope the above is clear. Thanks, Justine ----- On Thu, 19 Mar 2020 at 00:57, Jeff Neuman <jeff.neuman@comlaude.com> wrote:
Thanks Justine. The selection process for Evaluators is documents here: https://newgtlds.icann.org/en/about/evaluation-panels-selection-process
Other than “allow[ing] ICANN community consideration and intervention to take place in ICANN Org's selection of the CPE provider,” what other steps would you propose ICANN takes to become “more transparent?” I think we need to be fairly specific in our recommendations.
Thanks!
*Jeff Neuman*
Senior Vice President
*Com Laude | Valideus*
D: +1.703.635.7514
E: *jeff.neuman@comlaude.com <jeff.neuman@comlaude.com>*
*From:* Gnso-newgtld-wg <gnso-newgtld-wg-bounces@icann.org> *On Behalf Of *Justine Chew *Sent:* Wednesday, March 18, 2020 6:44 AM *To:* gnso-newgtld-wg@icann.org *Subject:* Re: [Gnso-newgtld-wg] Updated Subject Line: Notes and Action Items - New gTLD Subsequent Procedures PDP WG - 16 March 1500 UTC
Thanks, Julie, for these notes.
All, I regret not being able to join Monday's call. I've just listened to the recording and found the discussions encouraging.
I wanted to propose something additional for consideration, which is that perhaps, preceding a recommendation re: Qualification of Panellists (as noted below), t*here should be a recommendation requiring ICANN Org to be more transparent in its process of selecting a community priority evaluation provider/panel*.
Rationale: I don't think I would be wrong to say there have been made comments regarding the suitability of Economic Intelligence Unit as the CPE provider and perhaps to help ensure CPE panellists have suitable qualifications, an antecedent step would be to allow ICANN community consideration and intervention to take place in ICANN Org's selection of the CPE provider.
Thanks,
Justine -----
On Tue, 17 Mar 2020 at 01:43, Julie Hedlund <julie.hedlund@icann.org> wrote:
*With Correct Subject Line*
Dear Working Group members,
Please see below the notes from the meeting on 16 March at 1500 UTC. *These high-level notes are designed to help WG members navigate through the content of the call and are not a substitute for the recording, transcript, or the chat,* which will be posted at: https://community.icann.org/display/NGSPP/2020-03-16+New+gTLD+Subsequent+Pro... .
Kind regards, Julie
*Notes and Action Items:*
*Actions:*
*Recommendation xx (rationale 4)*:
ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9.
*Re: Qualifications of Panelists*:
ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities.
*Notes:*
1. Updates to Statements of Interest: No updates provided.
2. Discussion of Final Report Topics – see the documents at: https://docs.google.com/document/d/1xXu7gPKiblS3Vh4MCuK6NWfeRmMolXf9VF5sO7OG... and attached as a PDF.
a. 2.9.1 Community Applications
-- Does the WG want to affirm or otherwise address this PIRR recommendation? Recommendation 4.1.a: Consider all dimensions of the feedback received to revisit the CPE scoring and framework before the next application round. Comment from Jeff: Comment from Jeff Neuman: Technically I am note sure we are affirming this whole Guideline F. For example, I am not sure yet whether we will allow mutual agreement resolution...and the last part about giving the ICANN Board discretion, etc. The only thing we are affirming is the priority of communities in a contention set. COMMENT IS ADDRESSED.
-- Comment from Staff: Does the WG want to provide any detail here? Request from ICANN org: "It would be helpful if the PDP Working Group could provide more detailed guidance on the specific areas of the CPE process that “must be more transparent and predictable.” Additionally, it would be helpful if the PDP Working Group could provide more specific guidance on what should be changed or added that would enhance transparency and predictability for the CPE process."
-- Started a thread on changes to the CPE guidelines.
ACTION ITEM: Staff to send the updates to the CPE guidelines based on the WG discussions. DONE: See -- https://docs.google.com/document/d/1Ih_1NARViJXNNewDg-q87sQzQoC1dCtC/edit .
*Recommendation xx (rationale 4)*: ICANN must consider ways to improve evaluators’ ability to gather information about an application.
*Implementation Guidance xx (rationale 4)*: Evaluators should continue to be able to send clarifying questions to CPE applicants but further, should be able to engage in written dialogue with them as well.
*Implementation Guidance xx (rationale 4)*: Evaluators should be able to issue clarifying questions, or utilize similar methods to address potential issues, to those who submit letters of opposition to community-based applications.
*Discussion*:
-- Question: Did we get access to the actual research of the evaluators? Answer: We didn’t, but they did refer to research in their decisions. This was not consistent.
-- There should be no basis for an independent evaluator to do his or her own research. Worried about how this recommendation was developed.
-- Not all of the facts may be in the application, which is different from the UDRP. There should be research that may need to be done.
-- Shouldn’t that be on the complaining party to make their case?
-- We are saying that if there is independent research that is allowed then ICANN should improve evaluators ability to get information.
-- Perhaps it would be helpful to add to the recommendation “as described in the IG below” or “as set out in the Implementation Guidance below.”
-- It's okay for evaluators to find independent info - but they should disclose sources when ruling on the CPE.
-- The application should be robust and supported. And the public gets a chance to comment, right?
-- If you have support from major organizations, how does an evaluator know that they are real? They aren’t likely to be subject-matter experts.
-- If they are going to be doing research they could research who these organizations are to make sure they meet the requirements to support the application.
-- Anything that an evaluator is relying on should be disclosed to the applicant, particularly since we’ll have an appeal process. The applicant has to have the ability to challenge any independently developed information.
-- Trying to choose between two models 1) that the applicant should put together a robust application, encourages a robust public comment period, and discourages the evaluator from being in the role of a biased researcher 2) evaluators being inadvertently advocating for third parties that didn’t show up.
-- If we have all these criteria doesn’t it make sense that due diligence is done provided they give the applicant the ability to do clarifying questions in response. For example, the requirement for the uniqueness of the string would seem to be something that would require independent research. Also, documented support from the recognized community institutions.
-- Shouldn’t rely only on the information provided by the applicant.
-- The examiner should rely on a fulsome record, rather than create their own record. There is a requirement to gauge opposition from third parties. We should figure out a way to make them narrow, as with the Implementation Guidance -- maybe move those into the recommendation.
-- Don’t agree that we ban a panelist from doing outside research. Seems too extreme.
-- We are lacking notice to competitors. Need to let the examiners look at opposition.
-- Say, “In the event panelist(s) rely on independent research, that will be disclosed to the applicant through a Clarifying Question and the applicant should be allowed to respond prior to ruling. ”
-- If we require the evaluator to notify the applicant, isn’t the applicant just going to push back? The applicant should just have to provide a fulsome application.
-- If the applicant provides links/sources then at a minimum the evaluator can look at those links.
-- But how would an evaluator know that a term is commonly used in other industries without doing research. The Applicant is not going to put that into its own application.
-- Needs to be very clear that there isn’t just a request for a string with nothing to support it. It is one thing for the evaluator to confirm the information, but another for an evaluator to seek out opposition to it.
-- Need to remember that this is an evaluation -- in other evaluations in ICANN, say financial, does the evaluator only rely on what is provided by the applicant.
-- Add “except as needed for verification purposes and should be disclosed”. To verify the information.
-- Does the AGB currently allow evaluators to do research? Research is specifically noted. In paragraph 2 in 4.2.3. Bottom of the paragraph, pages 4-9. We are working on putting some guardrails around this.
-- Need to figure out what “if deemed necessary” means.
-- If we limit the ability of the evaluator to do research we should call that out in the public comment as it is a fairly significant change.
-- Important to encourage dialog, but can’t require it. If you have a community application that has overwhelming support, those who are in opposition should step up and encouraged to enter into a dialog.
-- The amount of opposition should be compared to the amount of support. 5 letters of opposition might be irrelevant - if you have 500 letters of support. 1 or 2 letters shouldn't automatically cost a point. Opposition should be weighed against support.
-- There should be a public and transparent verification of an organization stating opposition in a letter.
*ACTION ITEM: Move the Implementation Guidance into the recommendation. Paul McGrady and Anne Aikman-Scalese will work on developing language relating to “if deemed necessary” in the AGB, paragraph 2, 4.2.3, pages 4-9. Jamie Baxter also should consider what should be added in order to encourage dialog, and public and transparent verification of statements of opposition.*
*Re: The Working Group notes that CCT-RT Recommendation 34 states: “A thorough review of the procedures and objectives for community based applications should be carried out and improvements made to address and correct the concerns raised before a new gTLD application process is launched. Revisions or adjustments should be clearly reflected in an updated version of the 2012 AGB.” *
-- Staff Comment: From the CCT-RT recommendations tracking spreadsheet: As the WG refines recommendations on this topic, it may want to consider whether to make additional recommendations regarding objectives. Note that the CCT-RT recommendations consider "a higher rate of success for such applications" to be a measure of success.
-- Are we trying to get a higher success rate and what can be done other than the recommendations we’ve already made?
-- Need to have a requirement that the evaluators are knowledgeable about the community. Need to avoid the mistakes from the previous round.
-- We do have a section on accountability and appeals -- in the section on Accountability Mechanisms.
-- The qualifications of the panelists is not something we spent a lot of time on. Need to see if we can improve that.
-- Need quantitative and qualitative assessment by experts in communities.
*ACTION ITEM: Note that we had not developed a recommendation relating to the qualifications of the panelists and need to consider the need for a quantitative and qualitative assessment by experts in communities.*
_______________________________________________ Gnso-newgtld-wg mailing list Gnso-newgtld-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-newgtld-wg _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
------------------------------ The contents of this email and any attachments are confidential to the intended recipient. They may not be disclosed, used by or copied in any way by anyone other than the intended recipient. If you have received this message in error, please return it to the sender (deleting the body of the email and attachments in your reply) and immediately and permanently delete it. Please note that the Com Laude Group does not accept any responsibility for viruses and it is your responsibility to scan or otherwise check this email and any attachments. The Com Laude Group does not accept liability for statements which are clearly the sender's own and not made on behalf of the group or one of its member entities. The Com Laude Group includes Nom-IQ Limited t/a Com Laude, a company registered in England and Wales with company number 5047655 and registered office at 28-30 Little Russell Street, London, WC1A 2HN England; Valideus Limited, a company registered in England and Wales with company number 06181291 and registered office at 28-30 Little Russell Street, London, WC1A 2HN England; Demys Limited, a company registered in Scotland with company number SC197176, having its registered office at 33 Melville Street, Edinburgh, Lothian, EH3 7JF Scotland; Consonum, Inc. dba Com Laude USA and Valideus USA, headquartered at 1751 Pinnacle Drive, Suite 600, McLean, VA 22102, USA; Com Laude (Japan) Corporation, a company registered in Japan having its registered office at Suite 319,1-3-21 Shinkawa, Chuo-ku, Tokyo, 104-0033, Japan. For further information see www.comlaude.com <https://comlaude.com>
participants (6)
-
Alexander Schubert -
Jamie Baxter -
Jeff Neuman -
Jorge.Cancio＠bakom.admin.ch -
Julie Hedlund -
Justine Chew