- Gnso-rpm-data - lists.icann.org

Re: [Gnso-rpm-data] [Ext] RE: ACTIONS & NOTES: RPM Data Sub Team 20 July 2018
by Ariel Liang July 23, 2018

July 23, 2018

Hello Michael and all, The recordings and transcripts of meetings when the original TM survey questions were being drafted can be found on the wiki pages below (before Analysis Group was selected through RFP): 08 Dec 2017: https://community.icann.org/x/lxVyB 15 Dec 2017: https://community.icann.org/x/KRxyB 05 Jan 2018: https://community.icann.org/x/YSByB 11 Jan 2018: https://community.icann.org/x/8SJyB The recordings and transcripts of meetings when Analysis Group discussed with the Sub Team the proposed TM survey questions can be found on the wiki pages below: 08 Jun 2018: https://community.icann.org/x/Mi4FBQ 11 Jun 2018: https://community.icann.org/x/NC4FBQ Please let us know if you need anything further information. Have a great weekend! Thank you, Ariel Ariel Xinyue Liang GNSO Policy Support Specialist Internet Corporation for Assigned Names and Numbers (ICANN) From: "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Friday, July 20, 2018 at 7:49 PM To: Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] RE: ACTIONS & NOTES: RPM Data Sub Team 20 July 2018 Could you provide links to the Recordings/Transcripts from our discussions of drafting the Survey questions – in particular those in which we discussed the Trademark Owners survey. Thanks. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, July 20, 2018 4:40 PM To: gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] ACTIONS & NOTES: RPM Data Sub Team 20 July 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 20 July 2018 (16:00-17:30 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/vQJdBQ [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…>. Have a great weekend everyone! Best Regards, Ariel Ariel Xinyue Liang GNSO Policy Support Specialist Internet Corporation for Assigned Names and Numbers (ICANN) == ACTION ITEMS * AG to include, in the introduction, a statement that “If you are an outside law firm or agent that receive this survey, please answer the survey with one specific client in mind” or something similar to that effect. * AG to do a global change with respect to "you/your company" – this change may affect how certain questions are worded. * AG to remove Q1. * AG to move Q2 after Q3. * AG to reword the Q5a so the respondents can provide numbers directly instead of percentages, with options like 1-5, 6-10, 10-20, etc. * AG to change "enough" to "any" in Q5b. * AG to remove Q6. * AG to change the added phrase in Q8b to: "Did you register a domain name that matches your trademark?" or “Have you registered a domain name that incorporates your trademark?” * AG to use Kristine's suggested wording in Q9. * AG to remove Q13a. * AG to change "type" to "timing" for the bullet 4 in Q17. * AG to delete Q18. * AG to include Q21 in BOTH Sunrise & Claims surveys if we split the full TM survey. * AG to create an introductory paragraph before Q22, defining NORN and setting context per Kristine's suggestion. Check Kristine's comment in the Google doc. * AG to provide estimated dates for survey finalization & launch, including the date when they will begin issuing the surveys on a rolling basis. * AG to circulate the revised surveys to the Sub Team as soon as they are ready on a rolling basis * Kristine to suggest rewording for Q13. * Kristine to provide suggestion/input to Q26. * Staff to tentatively schedule a Data Sub Team call on Wed, 25 July at 17:00-18:30 UTC (DONE) NOTES General Comments * To provide respondents the correct expectation of time needed to complete the surveys, AG will share a preview of the full surveys. AG also agrees to cut up the TM survey into shorter ones (i.e., Sunrise, Claims, Reserved Names), and invite the respondents to select which survey(s) to take. * Not sure whether we should ask respondents to choose the survey(s) to take. Trademark Owners Survey General Comments * Referencing you or your company without referencing outside counsel is a concern. * We should find out on whose behalf the respondents are answering the questions. For the person answering the questions, ask “who are you?” * Where this survey is going to? e.g., law firm, business, trademark owners, etc.? Outside counsel can answer the survey. Law firm can forward the survey to their clients. * ACTION ITEM: AG to include, in the introduction, a statement that “If you are an outside law firm or agent that receive this survey, please answer the survey with one specific client in mind” or something similar to that effect. * ACTION ITEM: AG to do a global change with respect to "you/your company" – this change may affect how certain questions are worded. * Is there a question about how large is your company? On behalf of what part of your company you are responding to the survey? * Surveys like this will usually end up at the IP/TM group of a company. * Preserving anonymity is important, but we can add an option question for people to identify themselves if people are willing to answer. * For large company, there may be more people that can answer such survey, but this is not a problem. Do not limit the number of people who can answer the survey. * Based on INTA experience, if we ask the type of company and the size of company, and ask respondents to identify themselves, the impression of the lack of anonymity would render zero response, as the respondents would be very skeptical. * INTA asked respondents to identify whether they are nonprofit, for profit, and their revenue. Company size does not strictly correspond to company spend. Spend impacts whether TM owners register their trademark in the TMCH, as well as their answers to other follow-up question. If we ask people to self-identify, they need to provide info on their company type and revenue. * Some questions are related to each other. If we split the survey, it may affect these questions. * We should do a deeper study of all the questions and see whether there are any overlapped/interconnected questions that should be addressed. It seems only two questions would be impacted so far. Q1 * ACTION ITEM: AG to remove this question. Q2 * ACTION ITEM: AG to move it after Q3 Q3 * Do not use the word 'Counsel', just use 'Agent’. An in-house person is not necessarily legal counsel; they can be in-house IT groups, paralegals, etc. * If we have external counsel answering the questions, they have to put themselves in the shoe for one specific client. * Reword the question to: "Are you responding on behalf a company, or on behalf of a client?" * Lori does not like "on behalf of" because it doesn't get straight to "inside", "outside" issue. Perhaps reword it as “Are you responding as an employee of your company or as an outside service provider?” Q4 * Can merge with Q1. How many trademarks does the company that you are answering the question for own? If it is 0, terminate the survey. Q5a * ACTION ITEM: AG to reword the question so the respondents can provide numbers directly instead of percentages, with options like 1-5, 6-10, 10-20, etc. Q5b * ACTION ITEM: AG to change "enough" to "any". Q6 * ACTION ITEM: AG to remove this question Q7 * No need to get into the specifics in the wording of the question. Q7a * Added wording intends to clarify what is being asked Q8b * ACTION ITEM: AG to change the added phrase to: "Did you register a domain name that matches your trademark?" or “Have you registered a domain name that incorporates your trademark?” Q9 * If we split the survey, and if respondents do the Claims section later, we no longer have the proactive/reactive remedies as the first part of the survey. We might end up not getting this question in relations to Sunrise. * ACTION ITEM: AG to use Kristine's suggested wording. Q10d * Leave the question open ended * Or change "likely" to "able"? * Simplify the question; don't use the Likert scale and keep the text box in. * We heard from IT people that there are budget hurdles. We want to give them the opportunity to provide the anecdotal evidence. "Do you want to tell us how it affected your ability?" They can talk about one oddball experience via the open text box. * If we have a Likert scale, the options can be: more likely, less likely, completely unable, other ____ * Be more precise for the Likert scale. "High price" or "low price" make me more likely, less likely, completely unable, other___ Q13a * ACTION ITEM: AG to remove this question. Q13c * Take out everything before the second bullet point “[If "Yes"] Do you know if any of those were due to...” * Reword the question to "Do you know any refusal is based on the string being on the reserved name list?" * Some registry operators may have given the domain names to brand owners. This is viewed as a hostile act. That's why Q13c is being asked. There may be a better way to shorten Q13 and its sub questions. * Remove the last bullet "[IF “Yes”] Do you know if any of those were due to the string being on the reserved name list? [MULTIPLE CHOICE]". This may be very rarely answered. * ACTION ITEM: Kristine to suggest rewording for Q13 Q17 * During Sunrise Period, TM owners may need more authorization to do the spend. A lot of information about strategy / portfolio management is proprietary. Many portfolio managers operate on shoestring budget. So when Respondents see the question, they may interpret it as you need budget approval or strategy approval. It is outside the range of their spend or strategy. * The added phrase "from managers or the client" seems sufficient in bullet 3. * The inside baseball question is the one about the "type of Sunrise period" question. If you don't have a struggle about which type of Sunrise period to choose, you wouldn't be able to answer the question. * ACTION ITEM: AG to change "type" to "timing" for the bullet 4. Q18 * ACTION ITEM: AG to delete the question. * We are trying to understand how effective Sunrise is. If TM owners did not participate in it, do they still need to do a lot of reactive stuff? This question belongs in Claims, so should be deleted in this section. Some Claims questions may already cover it. Q21 * If we split the survey, this question should be moved to the end, as it covers both Sunrise and Claims. * ACTION ITEM: AG to include this question in BOTH Sunrise & Claims surveys if we split the full TM survey. Q22 * We should define NORN. This is the first question in the new survey. Include an introductory paragraph. The introductory paragraph can define the NORN. Then follow that introduction with Q22. It makes sense to flag to people that we are talking about Claims in this new section of the survey, and this approach can shorten Q22. * ACTION ITEM: AG to create an introductory paragraph defining NORN and setting context per Kristine's suggestion. Check Kristine's comment in the Google doc. Q26 * We want to know whether TM owners file a UDRP as a result of NORN. If you have filed a UDRP, did you file against 'exact match', 'near match', 'something match'. We are trying to understand whether "exact match" is sufficient. Question is whether Claims Notice is effective. If a UDRP challenge has been successful and it is not an exact match, then that may necessitate the evaluation of the rule with regard to exact match. * Staff have extracted out the data of UDRP cases filed up to the end of 2017. We will get a refresh of the data in the TMCH database and notices sent, and align the notices to the domain registered. There would not be exact alignment. TM owners could have gotten a notice, but they may not file UDRP three months after, etc. * There could also be other factors that a TM owner doesn't file UDRP. NORNs do not necessarily correspond to UDRP cases filed. * ACTION ITEM: Kristine to provide suggestion/input to Q26.

2 1

ACTIONS & NOTES: RPM Data Sub Team 20 July 2018
by Ariel Liang July 20, 2018

July 20, 2018

Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 20 July 2018 (16:00-17:30 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/vQJdBQ. Have a great weekend everyone! Best Regards, Ariel Ariel Xinyue Liang GNSO Policy Support Specialist Internet Corporation for Assigned Names and Numbers (ICANN) == ACTION ITEMS * AG to include, in the introduction, a statement that “If you are an outside law firm or agent that receive this survey, please answer the survey with one specific client in mind” or something similar to that effect. * AG to do a global change with respect to "you/your company" – this change may affect how certain questions are worded. * AG to remove Q1. * AG to move Q2 after Q3. * AG to reword the Q5a so the respondents can provide numbers directly instead of percentages, with options like 1-5, 6-10, 10-20, etc. * AG to change "enough" to "any" in Q5b. * AG to remove Q6. * AG to change the added phrase in Q8b to: "Did you register a domain name that matches your trademark?" or “Have you registered a domain name that incorporates your trademark?” * AG to use Kristine's suggested wording in Q9. * AG to remove Q13a. * AG to change "type" to "timing" for the bullet 4 in Q17. * AG to delete Q18. * AG to include Q21 in BOTH Sunrise & Claims surveys if we split the full TM survey. * AG to create an introductory paragraph before Q22, defining NORN and setting context per Kristine's suggestion. Check Kristine's comment in the Google doc. * AG to provide estimated dates for survey finalization & launch, including the date when they will begin issuing the surveys on a rolling basis. * AG to circulate the revised surveys to the Sub Team as soon as they are ready on a rolling basis * Kristine to suggest rewording for Q13. * Kristine to provide suggestion/input to Q26. * Staff to tentatively schedule a Data Sub Team call on Wed, 25 July at 17:00-18:30 UTC (DONE) NOTES General Comments * To provide respondents the correct expectation of time needed to complete the surveys, AG will share a preview of the full surveys. AG also agrees to cut up the TM survey into shorter ones (i.e., Sunrise, Claims, Reserved Names), and invite the respondents to select which survey(s) to take. * Not sure whether we should ask respondents to choose the survey(s) to take. Trademark Owners Survey General Comments * Referencing you or your company without referencing outside counsel is a concern. * We should find out on whose behalf the respondents are answering the questions. For the person answering the questions, ask “who are you?” * Where this survey is going to? e.g., law firm, business, trademark owners, etc.? Outside counsel can answer the survey. Law firm can forward the survey to their clients. * ACTION ITEM: AG to include, in the introduction, a statement that “If you are an outside law firm or agent that receive this survey, please answer the survey with one specific client in mind” or something similar to that effect. * ACTION ITEM: AG to do a global change with respect to "you/your company" – this change may affect how certain questions are worded. * Is there a question about how large is your company? On behalf of what part of your company you are responding to the survey? * Surveys like this will usually end up at the IP/TM group of a company. * Preserving anonymity is important, but we can add an option question for people to identify themselves if people are willing to answer. * For large company, there may be more people that can answer such survey, but this is not a problem. Do not limit the number of people who can answer the survey. * Based on INTA experience, if we ask the type of company and the size of company, and ask respondents to identify themselves, the impression of the lack of anonymity would render zero response, as the respondents would be very skeptical. * INTA asked respondents to identify whether they are nonprofit, for profit, and their revenue. Company size does not strictly correspond to company spend. Spend impacts whether TM owners register their trademark in the TMCH, as well as their answers to other follow-up question. If we ask people to self-identify, they need to provide info on their company type and revenue. * Some questions are related to each other. If we split the survey, it may affect these questions. * We should do a deeper study of all the questions and see whether there are any overlapped/interconnected questions that should be addressed. It seems only two questions would be impacted so far. Q1 * ACTION ITEM: AG to remove this question. Q2 * ACTION ITEM: AG to move it after Q3 Q3 * Do not use the word 'Counsel', just use 'Agent’. An in-house person is not necessarily legal counsel; they can be in-house IT groups, paralegals, etc. * If we have external counsel answering the questions, they have to put themselves in the shoe for one specific client. * Reword the question to: "Are you responding on behalf a company, or on behalf of a client?" * Lori does not like "on behalf of" because it doesn't get straight to "inside", "outside" issue. Perhaps reword it as “Are you responding as an employee of your company or as an outside service provider?” Q4 * Can merge with Q1. How many trademarks does the company that you are answering the question for own? If it is 0, terminate the survey. Q5a * ACTION ITEM: AG to reword the question so the respondents can provide numbers directly instead of percentages, with options like 1-5, 6-10, 10-20, etc. Q5b * ACTION ITEM: AG to change "enough" to "any". Q6 * ACTION ITEM: AG to remove this question Q7 * No need to get into the specifics in the wording of the question. Q7a * Added wording intends to clarify what is being asked Q8b * ACTION ITEM: AG to change the added phrase to: "Did you register a domain name that matches your trademark?" or “Have you registered a domain name that incorporates your trademark?” Q9 * If we split the survey, and if respondents do the Claims section later, we no longer have the proactive/reactive remedies as the first part of the survey. We might end up not getting this question in relations to Sunrise. * ACTION ITEM: AG to use Kristine's suggested wording. Q10d * Leave the question open ended * Or change "likely" to "able"? * Simplify the question; don't use the Likert scale and keep the text box in. * We heard from IT people that there are budget hurdles. We want to give them the opportunity to provide the anecdotal evidence. "Do you want to tell us how it affected your ability?" They can talk about one oddball experience via the open text box. * If we have a Likert scale, the options can be: more likely, less likely, completely unable, other ____ * Be more precise for the Likert scale. "High price" or "low price" make me more likely, less likely, completely unable, other___ Q13a * ACTION ITEM: AG to remove this question. Q13c * Take out everything before the second bullet point “[If "Yes"] Do you know if any of those were due to...” * Reword the question to "Do you know any refusal is based on the string being on the reserved name list?" * Some registry operators may have given the domain names to brand owners. This is viewed as a hostile act. That's why Q13c is being asked. There may be a better way to shorten Q13 and its sub questions. * Remove the last bullet "[IF “Yes”] Do you know if any of those were due to the string being on the reserved name list? [MULTIPLE CHOICE]". This may be very rarely answered. * ACTION ITEM: Kristine to suggest rewording for Q13 Q17 * During Sunrise Period, TM owners may need more authorization to do the spend. A lot of information about strategy / portfolio management is proprietary. Many portfolio managers operate on shoestring budget. So when Respondents see the question, they may interpret it as you need budget approval or strategy approval. It is outside the range of their spend or strategy. * The added phrase "from managers or the client" seems sufficient in bullet 3. * The inside baseball question is the one about the "type of Sunrise period" question. If you don't have a struggle about which type of Sunrise period to choose, you wouldn't be able to answer the question. * ACTION ITEM: AG to change "type" to "timing" for the bullet 4. Q18 * ACTION ITEM: AG to delete the question. * We are trying to understand how effective Sunrise is. If TM owners did not participate in it, do they still need to do a lot of reactive stuff? This question belongs in Claims, so should be deleted in this section. Some Claims questions may already cover it. Q21 * If we split the survey, this question should be moved to the end, as it covers both Sunrise and Claims. * ACTION ITEM: AG to include this question in BOTH Sunrise & Claims surveys if we split the full TM survey. Q22 * We should define NORN. This is the first question in the new survey. Include an introductory paragraph. The introductory paragraph can define the NORN. Then follow that introduction with Q22. It makes sense to flag to people that we are talking about Claims in this new section of the survey, and this approach can shorten Q22. * ACTION ITEM: AG to create an introductory paragraph defining NORN and setting context per Kristine's suggestion. Check Kristine's comment in the Google doc. Q26 * We want to know whether TM owners file a UDRP as a result of NORN. If you have filed a UDRP, did you file against 'exact match', 'near match', 'something match'. We are trying to understand whether "exact match" is sufficient. Question is whether Claims Notice is effective. If a UDRP challenge has been successful and it is not an exact match, then that may necessitate the evaluation of the rule with regard to exact match. * Staff have extracted out the data of UDRP cases filed up to the end of 2017. We will get a refresh of the data in the TMCH database and notices sent, and align the notices to the domain registered. There would not be exact alignment. TM owners could have gotten a notice, but they may not file UDRP three months after, etc. * There could also be other factors that a TM owner doesn't file UDRP. NORNs do not necessarily correspond to UDRP cases filed. * ACTION ITEM: Kristine to provide suggestion/input to Q26.

2 1

Recordings, attendance & AC chat from the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 20 July 2018
by Andrea Glandon July 20, 2018

July 20, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 20 July 2018 at 16:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/vQJdBQ MP3: https://audio.icann.org/gnso/gnso-rpm-review-data-20jul18-en.mp3 <https://urldefense.proofpoint.com/v2/url?u=https-3A__audio.icann.org_gnso_g…> AC Recording: https://participate.icann.org/p5ziockhbsi/<https://participate.icann.org/p5ziockhbsi/?OWASP_CSRFTOKEN=b740c2fc7e873b40…> The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar [gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Andrea Glandon

1 0

Re: [Gnso-rpm-data] [Ext] RPM Data Call
by Julie Hedlund July 20, 2018

July 20, 2018

Thanks Michael! From: "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Friday, July 20, 2018 at 11:59 AM To: Julie Hedlund <julie.hedlund(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] RPM Data Call I will be a couple minutes late calling in – Michael R.

1 0

RPM Data Call
by Michael Graham (ELCA) July 20, 2018

July 20, 2018

I will be a couple minutes late calling in - Michael R.

1 0

Data Sub Team Call, Friday, 20 July - Proposed Agenda & Potential Registrant Survey Distribution
by Julie Hedlund July 19, 2018

July 19, 2018

Dear Data Sub Team members, Many thanks again to the lead authors that provided comments and suggested edits on the Google Docs. For the call tomorrow (Fri, 20 July) at 16:00-17:30 UTC. Please see below the proposed agenda: 1. Review agenda 2. Discussion of revised survey by Analysis Group and input from lead author: · Brand/Trademark Owners (Lead: Michael Graham): https://docs.google.com/document/d/1NIDTSrEIVrvX3S0n3V-DO-BMlFElNBFhxH53fQs… [docs.google.com] 3. Start discussion of the survey distribution channels and outreach plans 4. AOB Analysis Group has informed us that they will distribute the potential registrant surveys to target respondents in North America and five English-speaking countries/regions outside of North America (UK, Hong Kong, India, Singapore, South Africa). This expansion would not incur additional costs/expenses. The target response rate for the potential registrant survey would be 150 complete responses, and Analysis Group expects that this is achievable. Following up on the action item, staff are confirming with Analysis Group the survey distribution channels and target response rates for the other surveys. We will report back and consult with the Sub Team the proposed plan. Best Regards, Mary, Berry, Ariel, and Julie

1 0

Actions & Notes: RPM Data Sub Team 18 July 2018
by Ariel Liang July 18, 2018

July 18, 2018

Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 18 July 2018 (17:00-18:30 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ioxHBQ. Best Regards, Ariel Ariel Xinyue Liang GNSO Policy Support Specialist Internet Corporation for Assigned Names and Numbers (ICANN) == ACTION ITEMS * Sub Team to review the trademark owners survey and suggest which questions remove/truncate to reduce the length; Sub Team to consider whether splitting the TM survey into shorter surveys based on sections (e.g., Sunrise, Claims, Reserved Names) would be more effective in reducing lengths. * AG to consider the suggestion allowing respondents preview all the questions to set the right expectation (e.g., via a PDF). * AG to provide input on TM survey questions that respondents could not really answer. * AG to consider breaking the TM survey into three short surveys (i.e., Sunrise, Claims, Reserved Names); respondents get links to the three surveys and they can take as many as they'd like; or respondents get randomly selected to take one of the three short surveys based on certain qualification/selection criteria. * AG to consider combining/consolidating Q4g, Q5, and Q6 in the Registrar Survey. * AG to consider combining/consolidating the questions with related tables to reduce the Registry Survey length to 20 questions * AG to clarify in Registry Survey Q2 that we are seeking responses from the actual registry operators, not the backend registry operators. * AG to direct respondents from Q7 to Q10 and Q11, and then ask Q8 and Q9 as add-on questions in Registry Survey. * Staff to schedule a Sub Team meeting on Friday, 20 July at 16:00-17:30 UTC (DONE). NOTES General Comments Regarding Survey Length * Set respondents' expectation and note that they need approximately 20 minutes to complete each survey (instead of 15-20 min). * Create means (e.g., worksheet) for respondents to return to the survey and answer the questions that require research and data gathering. * If specific contact information of individuals is provided, they can return the survey with a password and complete. * Registry survey has 40 questions, registrar 36 questions, and TM survey has 56 questions. The current length of these surveys would cause a lot of drop off. * ACTION ITEM: AG to consider the suggestion to allowing respondents preview all the questions (e.g., via a PDF) * Break the long survey into smaller surveys? Prioritize and reorder the questions so the important questions go first? * If there are no follow-up questions to solicit more elaborate responses, it would be challenging to get WG's approval of the survey results. * If the WG’s interest to get certain data has negative impact on the Sub Team’s ability to obtain data in general, the Sub Team have to challenge the WG. Some data that the WG wants to obtain is not obtainable. TM owners are not keeping record the way that ICANN asks for, for example. * Expedia’s user experience surveys create two paths: 1) just do the rating (stars), 2) provide more extended responses. Consider adopting this approach to let respondents choose whether to complete a short or full survey. * Registries/Registrars have incentive to answer the surveys. * At the beginning TM survey, perhaps spell out the type of legal counsel should take the survey and the type of data/fact that respondents should have it handy to facilitate their process answering the survey. * The lack of clear indication of the time needed to complete the survey would be off putting. * The questions in the TM survey are more general and doable type of questions, and less difficult and intimidating than the CCT-RT survey. We should let survey takers understand the specific information we are seeking instead of asking them to analyze it. * Ideas for lowering the drop-off rate for the TM survey: * Present a short survey option and a full survey option (e.g., click here to answer Qs on Claims; here to answer on Sunrise?) * After presenting the short survey, ask respondents whether they want to take a subsequent full survey, * Inform respondents that they would need 40 min to complete the survey, but they will be provided with a password to save their responses and return to the survey to complete; AG could look into setting this as a default option – two-step process: Want to take the survey? Email us, and AG will provide an individual password, * Calculate the time needed to complete each section of the survey. * ACTION ITEM: AG to provide input on questions that respondents could not really answer. * The short/full survey approach sounds reasonable, but AG would leave to Data Sub Team to determine whether this approach would get the information that is ultimately useful. * Would the short survey approach skew the results? Not sure whether skewed responses matter as the goal is to get the data. It is better to get the data we can get rather than "all or nothing." * If we split the long survey into short surveys, prefer randomization instead of inviting respondents to pick and choose which survey(s) they wish to take (may need to qualify respondents differently). This would generate less bias. If there is no randomization, you could get responses from people who have strong opinions about certain issue, hence more tendency to bias. * ACTION ITEM: AG to consider breaking the TM survey into three short surveys (i.e., Sunrise, Claims, Reserved Names); respondents get links to the three surveys and they can take as many as they'd like; or respondents get randomly selected to take one of the three short surveys based on certain qualification/selection criteria. ** Registrar Survey Q1 * Make the question more direct: Which registrar are you employed by? Q2 * Ask where the headquarters of the registrar is. Q4g, Q5, Q6 * ACTION ITEM: AG to consider combining/consolidating Q4g, Q5, and Q6. Q19 * Reword the question to "At what point in the registration process do you typically display the Claims Notice..." * Options for drop down menu may include the relevant steps of a domain name registration process: * WHOIS Lookup * When the domain name is selected * When the domain name is added to the cart * When the consumer log into the registrar account or create an account * When the consumer is present with the terms & clickthrough agreements * When the consumer is present with the renewal information * When the consumer proceeds to check out * Most people may get the Claims Notice just before the check-out step. ** Registry Survey General Comments * ACTION ITEM: AG to consider combining/consolidating the questions with related tables to reduce the survey length to 20 questions Q2 * ACTION ITEM: AG to clarify that we are seeking responses from the actual registry operators, not the backend registry operators. Q7-12 * ACTION ITEM: AG to direct respondents from Q7 to Q10 and Q11, and then ask Q8 and Q9 as add-on questions Q9 * Respondents who already offer such TLDs would be able to understand the vagueness of the wording. Leave it as is. Q10a * "Publishing reserved names would violate local laws" would apply to a small number of jurisdictions. * Drop-down menu options: * Such publication would violate local laws * Trade secret * Reluctance to post competitive data * OPEN field Q16 * This is a concluding question, put at the end. Q17 * Old wording is confusing too, e.g., 'special rules', 'certain groups'. * Do we want to offer special group access based on their restricted TLDs and eligibility criteria? There may be special rules for government affiliated groups to register restricted TLDs before brand owners (e.g., police.city). Q17 should be worded broad enough but at the same to put some parameters around it. * Take out 'brand owners' in the brackets * Remove “Sunrise"; just say "…first access to their target audience..." Q21 * Did you face any challenges with regard to both of your LRP/APLP and mandatory sunrise period? ** Actual / Potential Registrant Survey Q5a * Concerned about the "more than once". Respondents may not remember. * Ask a follow up question -- For subsequent time(s) that you get a claims notice, did you do anything differently? Open Ended follow-up question could provide the opportunity for respondents to elaborate without making it onerous. * We really want to hear back from registrants that received Claims Notice more than once.

1 0

Recordings, attendance & AC chat from Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Wednesday, 18 July 2018
by Michelle DeSmyter July 18, 2018

July 18, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Wednesday, 18 July 2018 at 17:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/ioxHBQ MP3: https://audio.icann.org/gnso/gnso-rpm-review-data-18jul18-en.mp3 AC Recording: https://participate.icann.org/p377bxw073v/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar [gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Michelle

1 0

Data Sub Team Call, Wed 18 July - Proposed Agenda & Potential Registrant Survey Distribution
by Ariel Liang July 17, 2018

July 17, 2018

Dear Data Sub Team members, Many thanks again to the lead authors that provided comments and suggested edits on the Google Doc. Analysis Group are reviewing the input in preparation for tomorrow’s discussion For the call tomorrow (Wed, 18 July) at 17:00-18:30 UTC. Please see below the proposed agenda: 1. Review agenda 2. Discussion of revised surveys by Analysis Group and input from lead authors: · Registry Operators (Lead: Kristine Dorrain): https://docs.google.com/document/d/1ql3_l2W9dSq-dDAvDuSWggJF4Icww4j6G98va9-… · Registrars (Lead: Susan Payne): https://docs.google.com/document/d/1K3njd0IP8--FSxskS0JAMDrDEnS_wel5iqqkil3… · Brand/Trademark Owners (Lead: Michael Graham): https://docs.google.com/document/d/1NIDTSrEIVrvX3S0n3V-DO-BMlFElNBFhxH53fQs… · Actual/Potential Registrants (Lead: Rebecca Tushnet): https://docs.google.com/document/d/1i_1Eu8gON2J_UsL1d113cUEa7MI0E8Lt6d3bzW1… 3. Start discussion of the survey distribution channels and outreach plans 4. AOB Analysis Group also informed us that they will distribute the potential registrant surveys to target respondents in North America and five English-speaking countries/regions outside of North America (UK, Hong Kong, India, Singapore, South Africa). This expansion would not incur additional costs/expenses. The target response rate for the potential registrant survey would be 150 complete responses, and Analysis Group expects that this is achievable. Following up on the action item, staff are confirming with Analysis Group the survey distribution channels and target response rates for the other surveys. We will report back and consult with the Sub Team the proposed plan. Best Regards, Mary, Julie, Berry, Ariel

1 0

Re: [Gnso-rpm-data] [Ext] RE: REMINDER: Revised Sunrise & Claims Draft Surveys
by Ariel Liang July 13, 2018

July 13, 2018

Thank you very much Kristine! Best Regards, Mary, Julie, Ariel, and Berry From: "Dorrain, Kristine" <dorraink(a)amazon.com> Date: Friday, July 13, 2018 at 11:46 AM To: Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] RE: [Gnso-rpm-data] REMINDER: Revised Sunrise & Claims Draft Surveys Sorry, I was on vacation last week and just catching up. I’ll work on this today. Thanks. Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, July 13, 2018 9:27 AM To: gnso-rpm-data(a)icann.org Subject: Re: [Gnso-rpm-data] REMINDER: Revised Sunrise & Claims Draft Surveys Dear Data Sub Team members, Since there has been no comment/suggested edit on the Google Docs of the revised surveys yet, staff are sending this second reminder. It would be extremely helpful if Kristine, Susan, Michael, Rebecca, as well as other members could provide input on the google doc as soon as feasible, if at all possible, by COB today, so that Analysis Group could review the comments/suggested edits before the call next week. GNSO Secretariat team will schedule a call for the Sub Team on Wednesday, 18 July at 17:00-18:30 UTC. * Registry Operators (Lead: Kristine Dorrain): https://docs.google.com/document/d/1ql3_l2W9dSq-dDAvDuSWggJF4Icww4j6G98va9-… [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…> * Registrars (Lead: Susan Payne): https://docs.google.com/document/d/1K3njd0IP8--FSxskS0JAMDrDEnS_wel5iqqkil3… [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…> * Brand/Trademark Owners(Lead: Michael Graham): https://docs.google.com/document/d/1NIDTSrEIVrvX3S0n3V-DO-BMlFElNBFhxH53fQs… [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…> * Actual/Potential Registrants (Lead: Rebecca Tushnet): https://docs.google.com/document/d/1i_1Eu8gON2J_UsL1d113cUEa7MI0E8Lt6d3bzW1… [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…> For reference, redline version are in the word docs attached. Thank you again for your time and contribution! Best Regards, Mary, Julie, Ariel, and Berry From: Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>> Date: Monday, July 9, 2018 at 2:04 PM To: "gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org>" <gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org>> Subject: Revised Sunrise & Claims Draft Surveys Dear Data Sub Team members, We hope this email finds you well. Analysis Group has revised the draft surveys of the registry operators, registrars, brand/trademark holders, and actual/potential registrants. Please note that AG has combined the surveys of registrant and potential registrants into one document because the questions overlap to a high degree. As per action item from the call prior to ICANN62, the original authors of the respective surveys would take the lead to review/comment on the revised surveys. To facilitate the review process, staff have converted the clean version of the revised surveys into Google Docs; the Google Docs have been shared with AG so they can view the comments/suggested edits at the same time. You may also find the redline version (.word files) attached for reference: * Registry Operators (Lead: Kristine Dorrain): https://docs.google.com/document/d/1ql3_l2W9dSq-dDAvDuSWggJF4Icww4j6G98va9-… [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…> * Registrars (Lead: Susan Payne): https://docs.google.com/document/d/1K3njd0IP8--FSxskS0JAMDrDEnS_wel5iqqkil3… [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…> * Brand/Trademark Owners(Lead: Michael Graham): https://docs.google.com/document/d/1NIDTSrEIVrvX3S0n3V-DO-BMlFElNBFhxH53fQs… [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…> * Actual/Potential Registrants (Lead: Rebecca Tushnet): https://docs.google.com/document/d/1i_1Eu8gON2J_UsL1d113cUEa7MI0E8Lt6d3bzW1… [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…> Kristine, Susan, Michael, and Rebecca – please be so kind to review and comment/suggest edits via the Google Docs. Others in the Sub Team are welcome to review as well. Based on staff’s understanding, the lead authors could conduct their review of the revised survey online. If needed, the Sub Team may reconvene thereafter to review/discuss the lead authors’ input to the revised surveys. Hence, a Sub Team call this week is unlikely needed. Please comment on the list if you believe a call for the Sub Team this week is necessary. Thank you for your time and contributions! Best Regards, Mary, Julie, Ariel, and Berry

3 3

REMINDER: Revised Sunrise & Claims Draft Surveys
by Ariel Liang July 13, 2018

July 13, 2018

Dear Data Sub Team members & lead authors of surveys, Staff would like to remind you to review the revised TMCH Sunrise & Claims surveys by Analysis Group (email below). As a Sub Team call is likely to be held next week to review the input on the revised questions, it would be extremely helpful if the lead authors (i.e., Kristine, Susan, Michael, and Rebecca) could provide comments/suggested edits on the Google Docs by this Friday, 13 July. Analysis Group also has access to the Google Doc and can view the input prior to the call next week. This would help facilitate the timely finalization of the survey questions. Others in the Sub Team are welcome to review/comment on the Google Doc as well. Thank you! Mary, Julie, Ariel, and Berry From: Ariel Liang <ariel.liang(a)icann.org> Date: Monday, July 9, 2018 at 2:04 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Revised Sunrise & Claims Draft Surveys Dear Data Sub Team members, We hope this email finds you well. Analysis Group has revised the draft surveys of the registry operators, registrars, brand/trademark holders, and actual/potential registrants. Please note that AG has combined the surveys of registrant and potential registrants into one document because the questions overlap to a high degree. As per action item from the call prior to ICANN62, the original authors of the respective surveys would take the lead to review/comment on the revised surveys. To facilitate the review process, staff have converted the clean version of the revised surveys into Google Docs; the Google Docs have been shared with AG so they can view the comments/suggested edits at the same time. You may also find the redline version (.word files) attached for reference: * Registry Operators (Lead: Kristine Dorrain): https://docs.google.com/document/d/1ql3_l2W9dSq-dDAvDuSWggJF4Icww4j6G98va9-… * Registrars (Lead: Susan Payne): https://docs.google.com/document/d/1K3njd0IP8--FSxskS0JAMDrDEnS_wel5iqqkil3… * Brand/Trademark Owners(Lead: Michael Graham): https://docs.google.com/document/d/1NIDTSrEIVrvX3S0n3V-DO-BMlFElNBFhxH53fQs… * Actual/Potential Registrants (Lead: Rebecca Tushnet): https://docs.google.com/document/d/1i_1Eu8gON2J_UsL1d113cUEa7MI0E8Lt6d3bzW1… Kristine, Susan, Michael, and Rebecca – please be so kind to review and comment/suggest edits via the Google Docs. Others in the Sub Team are welcome to review as well. Based on staff’s understanding, the lead authors could conduct their review of the revised survey online. If needed, the Sub Team may reconvene thereafter to review/discuss the lead authors’ input to the revised surveys. Hence, a Sub Team call this week is unlikely needed. Please comment on the list if you believe a call for the Sub Team this week is necessary. Thank you for your time and contributions! Best Regards, Mary, Julie, Ariel, and Berry

3 3

Revised Sunrise & Claims Draft Surveys
by Ariel Liang July 9, 2018

July 9, 2018

Dear Data Sub Team members, We hope this email finds you well. Analysis Group has revised the draft surveys of the registry operators, registrars, brand/trademark holders, and actual/potential registrants. Please note that AG has combined the surveys of registrant and potential registrants into one document because the questions overlap to a high degree. As per action item from the call prior to ICANN62, the original authors of the respective surveys would take the lead to review/comment on the revised surveys. To facilitate the review process, staff have converted the clean version of the revised surveys into Google Docs; the Google Docs have been shared with AG so they can view the comments/suggested edits at the same time. You may also find the redline version (.word files) attached for reference: * Registry Operators (Lead: Kristine Dorrain): https://docs.google.com/document/d/1ql3_l2W9dSq-dDAvDuSWggJF4Icww4j6G98va9-… * Registrars (Lead: Susan Payne): https://docs.google.com/document/d/1K3njd0IP8--FSxskS0JAMDrDEnS_wel5iqqkil3… * Brand/Trademark Owners(Lead: Michael Graham): https://docs.google.com/document/d/1NIDTSrEIVrvX3S0n3V-DO-BMlFElNBFhxH53fQs… * Actual/Potential Registrants (Lead: Rebecca Tushnet): https://docs.google.com/document/d/1i_1Eu8gON2J_UsL1d113cUEa7MI0E8Lt6d3bzW1… Kristine, Susan, Michael, and Rebecca – please be so kind to review and comment/suggest edits via the Google Docs. Others in the Sub Team are welcome to review as well. Based on staff’s understanding, the lead authors could conduct their review of the revised survey online. If needed, the Sub Team may reconvene thereafter to review/discuss the lead authors’ input to the revised surveys. Hence, a Sub Team call this week is unlikely needed. Please comment on the list if you believe a call for the Sub Team this week is necessary. Thank you for your time and contributions! Best Regards, Mary, Julie, Ariel, and Berry

1 0

Actions & Notes: RPM Data Sub Team 20 June 2018
by Ariel Liang June 20, 2018

June 20, 2018

Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 20 June 2018 (17:30 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/aoMpBQ Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ACTION ITEM * AG to touch base with Research Now SSI and see how the cost would be impacted if the survey is expanded beyond NA region but still conducted in English. * AG to revise the questions and send to the point persons/Sub Team to review during ICANN62 and the first week of July. * Staff to discuss with AG about the distribution channels of the other surveys and start the discussion via the Sub Team mailing list. NOTES Discussions about Survey Outreach * Lori expressed concerns about the ways to identify registrants/potential registrants. * Based on Prof. Tushnet’s understanding, we are not only targeting people who attempted to register new gTLD domain names and received Claims Notice before. The target respondents are the people who are interested in registering a new gTLD domain name or tried to register new gTLDs in the past. Make sure AG is reaching the correct target group using validation checks, and not just the Sub Set Lori raised. * For registrant/potential registrant surveys, the initial screening question is that “Have you registered new gTLD domain names?”. AG can include additional screening questions to find out whether the respondents are the registrant population that the Sub Team is interested in reaching. AG can ask what type of gTLDs have they registered? * AG is planning to distribute the survey among the North American panel of the Research Now SSI, who are English speaking. * If the Sub Team need more demographic information, additional questions can be included, so the results can be cut based on different types of demographics. * More difficult way to reach the target respondents would be working with registrars and ask them to distribute the surveys to people who attempted to register new gTLD domains and received claims notice. * Sub Team would love if the registries/registrars are willing to send out the surveys to the registrants/potential registrants. Should pursue that avenue. * Statistically speaking, you will get a tiny number of people that qualify with regard to having registered new gTLD domains and receiving Claims Notice. * Perhaps people in North America aren't easily intimidated, but people from other regions are. * Survey cannot be limited to North America to be as comprehensive as we need to be. If we can't do the job correctly, then I have some fears about how this will be accepted by the community. Sub Team/Co-Chairs would face the criticisms from the community, as the survey results would not be representative. We need to provide additional rationale beyond the "cost" factor. * To expand the survey distribution to other geographic regions and use different languages would add cost significantly. Research NOW SSI do not have a lot of concerns with the lack of responses. But cost would increase due to the development of survey questions in other languages, the coding of surveys, and the review of responses in other languages. * Section 5 (Survey of Potential Respondents) Relevant Charter Q 3 addresses translations....how did this get missed? There's translating the survey, and there's the question as to whether useful data can be obtained regarding translations of the Claims Notice (the latter being the Charter question). * We know that specific Charter questions talks about translations of the Claims Notice. This of course does not imply that the survey itself is to be translated. So perhaps the question for AG is whether/how the geographical limitation can be overcome, and how responses concerning a translated Claims Notice may be obtained even via an English-language survey (if possible). The staff understanding is that the surveys will be conducted only in English. Perhaps the Sub Team and AG can discuss whether, within that limitation, the geographical coverage can be extended? Bearing the cost issue in mind. Could we do the surveys only in English, but go to other geographic regions beyond NA? * ACTION ITEM: AG to touch base with Research Now SSI and see how the cost would be impacted if the survey is expanded beyond NA region but still conducted in English. * We have not discussed in detail about the distribution channel for other surveys. * ACTION ITEM: Staff to discuss with AG about the distribution channels of the other surveys and start the discussion via the Sub Team mailing list. General Comments: Potential Registrant Survey * It is really a comprehension survey, particularly for the potential respondents. Like the short and sweet format which may likely get more responses. * Same reordering of the registrant survey can be applied to the potential registrant survey. * There is limitation in what we could learn from the data gathered through the current survey questions. * Would AG suggest that we use separate Registrant and Potential Registrant surveys or a single survey that has two tracks based on answers. This could provide another means for ensuring qualification of survey takers. Depending on their answers to the first screening question, they would be directed to the registrant/potential registrant surveys. Q1i * This question is targeting the primary demographic of the survey. Support the grid format. * The question may have missed the point "what the potential registrants think about the claims notice after they are shown the claims notice". The real focus is to understand how important they perceive the claims notice. * This question intends to transform the original open-ended question to a closed question. In the grid, AG lists these potential factors that may go into potential registrants’ decisions for continuing registration. * We do not want to limit the question only Group B and D. Make the question also available to Group C. * For Group B & D -- people who have never registered and saw the Claims Notice for the first time -- we are asking them two questions essentially: 1) Of the 5 factors, which one most influenced their decisions? 2) Why would they decide to abandon after they see the claims notice? Do we want to have follow up questions? We are not opposed to a short survey, but we just want to be sure we're getting it all. * Is this question going to the wrong people? Don't think these people are in the position to make that decision if they have hardly seen any Claims Notice. * If respondents select “would not influence me at all”, “not sure”, “I don't know”, they would not see the follow question, i.e., what are the reasons the Claims Notice would affect them. Only when they select "yes", they would see the follow-up question. * ACTION ITEM: AG to provide respondents the options to explain further even when they select “would not influence me at all”; they should also get a follow up question to explain their reason. * ACTION ITEM: AG to delete "Someone else already having claimed my domain name" in the grid. Q1j * No context for the options. Next Steps * AG to revise the questions, each author of the original survey to serve as the point person to review the revised version, and the entire Sub Team to approve all the revised questions, making sure the Sub Team is in agreement. * ACTION ITEM: AG to revise the questions and send to the point persons/Sub Team to review during ICANN62 and the first week of July. * WG to see the final surveys for information only. * As AG needs some time to code and test the survey after the survey questions are finalized, anticipate to distribute the surveys in mid/late July. * Reporting of the result would be late August. * There are many complaints about scheduling conflicts so it is hard to organize an ad-hoc meeting for the Data Sub Team in Panama.

1 0

Recordings, attendance & AC chat from the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Wednesday, 20 June 2018
by Andrea Glandon June 20, 2018

June 20, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Wednesday, 20 June 2018 at 17:30 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/aoMpBQ MP3: https://audio.icann.org/gnso/gnso-rpm-review-data-20jun18-en.mp3<https://urldefense.proofpoint.com/v2/url?u=https-3A__audio.icann.org_gnso_g…> AC Recording: https://participate.icann.org/p5ovuhjig87/<https://participate.icann.org/p5ovuhjig87/?OWASP_CSRFTOKEN=d78a3b860d49eb9d…> The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar [gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Andrea Glandon

1 0

Following My Question from Monday re: Qualification of Registrants for Survey
by Lori Schulman June 20, 2018

June 20, 2018

Dear All, I know that we are approaching today’s meeting and I wanted to reiterate something on the list rather than to absorb more time on the live call. On Monday, I had asked about how Registrants will be vetted for the survey. Stacy from AG explained that there will be “funneling” type questions that will qualify survey takers as “registrants” and they will be funneled to the rest of the questions. I am still not clear on how the questions will be drafted in order to ensure that we are getting survey takers who understand what is being asked based on their direct experience. Does AG have proposed draft questions already? If so, may we see them? Using paid test takers for random, consumer oriented surveys makes sense to me. For example, surveys on taste preferences or product comparisons. However, using that kind of pool for something as technical as our survey still troubles me. This goes back to something that Rebecca had explained regarding making sure that we are getting responses based on actual experiences rather than risking survey takers “ranking” or establishing “preferred” answers. If we want to query the understandability of claims notices then I think whether one has seen them before or not may not matter. However, I do think that if we are targeting users who have made actual decisions based on actual notices then we should only be targeting known purchasers who have received those notices. It seems to me that only the registrars would know for sure and I still have doubts about surveying the general public through a paid survey taking system. Therefore, I think it is appropriate to set some real parameters here. I want to make sure that we have the best pool possible given the technical nature of the survey. And I have further questions: * How does AG group plan to drill down to specific segments that we want? * What will the demographic profile look like? * How will we know that the respondents have actually attempted to purchase and have abandoned names based on a claims notice? * How will data be “sifted” after its collected? For example, we should identify in advance what we are looking for i.e., age, employment, number of domains registered, number of abandoned, etc. I have asked this question twice and I am still stuck on some of the details of the response. Perhaps the analysis group can respond in writing which could clarify what we are doing? Thank. Thank you. Lori S. Schulman Senior Director, Internet Policy International Trademark Association (INTA) +1-202-704-0408, Skype: lsschulman From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Julie Hedlund Sent: Tuesday, June 19, 2018 4:28 PM To: gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] REMINDER: Proposed Agenda and Documents: Meeting 20 June at 1730 UTC Dear Data Sub Team Members, In preparation for Wednesday’s meeting (20 June) at 17:30 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 3. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…>. Best Regards, Mary, Julie, Ariel, and Berry ________________________________

2 2

REMINDER: Proposed Agenda and Documents: Meeting 20 June at 1730 UTC
by Julie Hedlund June 19, 2018

June 19, 2018

Dear Data Sub Team Members, In preparation for Wednesday’s meeting (20 June) at 17:30 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 3. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

1 0

RProposed Agenda and Documents: Meeting 20 June at 1730 UTC
by Julie Hedlund June 18, 2018

June 18, 2018

Dear Data Sub Team Members, In preparation for Wednesday’s meeting (20 June) at 17:30 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 3. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

1 0

Recordings, attendance & AC chat from the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Monday, 18 June 2018
by Julie Bisland June 18, 2018

June 18, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Monday, 18 June 2018 at 16:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/aIMpBQ MP3: https://audio.icann.org/gnso/gnso-rpm-review-18jun18-en.mp3 AC Recording: https://participate.icann.org/p7tsywnxxt7/<https://participate.icann.org/p7tsywnxxt7/?OWASP_CSRFTOKEN=a3029c3a5e9ca4be…> The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Julie

1 0

Actions & Notes: RPM Data Sub Team 18 June 2018
by Ariel Liang June 18, 2018

June 18, 2018

Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 18 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/aIMpBQ Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ACTION ITEM * Staff to email the full WG the schedule of upcoming meetings, including Sub Team calls (DONE) NOTES General Comments * AG will collaborate with ICANN staff to distribute the registrant survey among ICANN community groups that may include potential respondents. * There would be a filter question at the beginning, asking the respondents whether they have registered domains in the new gTLD. If not, they will be filtered to complete the potential registrant survey. Hypothetical questions are posted to both actual registrants/potential registrants. * The way that AG broke out the surveys is correct. * Is there a way to protect the responses to earlier questions being “contaminated”? You have a forward track and you cannot go back. Consider setting up the survey in a way that does not allow the respondents to go back and revise their responses to earlier questions? * There is concern about the single use case, as we may not receive representative responses. * Registrant survey targets 50 complete responses, and potential registrant survey targets 200 complete responses. If we get 200 potential registrants, sub group of 50 is acceptable. * The three domain name examples at the very beginning of the registrant survey are not new gTLDs. Any examples we use must be live. Discussions of the Hypothetical Questions (Doc sent by Kristine) * The respondents do not necessarily need to have already seen a Claims Notice. * Rebecca likes the scenarios/examples proposed by Kristine, but is concerned with asking the respondent multiple times for the various scenarios. This may not match the registrant’s actual experience registering a domain name. * The key is to show a Claims Notice and ask respondents to provide responses to specific questions. virtue.food example would be helpful to elicit thoughts from registrants. We cannot have a survey without some imagined domain name. Other than that, supports examples from Kristine. * Is there a threshold question? The current filter of the hypothetical question is that the respondent attempted to register for a new gTLD domain name but did not complete the registration. * Survey takers would be sophisticated enough to understand that people register different domain names for different uses. * We would be showing a Claims Notice specifically for VIRTUE. * From the way the survey is being set up, isn't the idea to describe the sequence/process that a registrant/potential registrant goes through in order to see how they interact with the domain registration process? That seems to staff to be what is typically described as a "use case". * If we give respondents a particular example, there would be a huge effect on the survey responses. We should at least give a couple of examples. * Regarding Kristine’s proposal, the issue is with regard to survey administration. It can be troublesome to present survey takers a whole list of examples/scenarios. Respondents may shape their responses by comparing these different examples/scenarios. The response we get would not be representative. Happy to add a xerox example. We need to do some subgrouping and random rotation, making sure one respondent will get one xerox example and one virtue example. Simulate the survey to what the registrant would be experiencing. * Don't understand why all the examples could not be presented to the respondents at once. * Strongly oppose to using an actual brand, especially a famous mark. * A survey does best when it replicates the actual reality. What is the best way to get people’s thought process via this survey by simulating the scenario that they are registering a domain name in the new gTLDs? * AG agrees with Rebecca's points. AG proposes to include a screening question: if you are going to register a domain name, what would you most likely register the domain name for? Based on respondents’ answers to the screening question, port/funnel them into a grid question. They would see some, but not all of the options in the grid. * Would the virtue.food example be an arbitrary example? * Funneling could work, but concerned whether it would be too limiting. Can we open the registrant survey up to more than 50 respondents? Can we ask this hypothetical question also to the potential registrants? * AG would ask this hypothetical question to both registrant and potential registrants (targeting a total of 250 responses). Can certainly target more than 50 registrants too. * In the scenario of virtue.food, when we present the Claims Notice of virtue, we need to know what the trademark is registered for. The reason for the registrants to proceed or not to proceed depends on what the TM is registered for. * An actual Claims Notice for https://domainnamewire.com/2014/01/30/trademark-claims-notice/. Virtue is registered for "medical equipment". * There is no chance that an English word is not already registered as a TM, but we can use a TM example of limited rights. We will need to fictionalize the Claims Notice for “virtue”, especially the goods/classes/jurisdiction elements. Do not set people up for an actual cybersquatting experience. Example, "virtue" for doors? * Sub Team is coalescing on an agreement to use virtue.food as an example, show a fictionalized Claims Notice for virtue, use a funnel to direct respondents to questions, ask them what they would do with the domain name based on their own experience. Procedural Matters * On Wed, 20 June, first 30 min is a brief meeting for the full WG to get on the same page in preparation for ICANN62; the remaining 60 min (17:30-18:30 UTC) would be used for the Data Sub Team. * Goal is to finish off the surveys by the Wed meeting for AG to revise/finalize the questions.

1 0

Re: [Gnso-rpm-data] [Ext] RE: REMINDER: Proposed Agenda and Documents: Meeting 18 June at 1600 UTC
by Julie Hedlund June 18, 2018

June 18, 2018

Thanks so much Kristine. We’ll get these loaded into the Adobe Connect room. Best, Julie From: "Dorrain, Kristine" <dorraink(a)amazon.com> Date: Monday, June 18, 2018 at 10:18 AM To: Julie Hedlund <julie.hedlund(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] RE: [Gnso-rpm-data] REMINDER: Proposed Agenda and Documents: Meeting 18 June at 1600 UTC Thanks Julie! All, I’m planning to be on the call since we moved it up (my 5 yo may join me so I’ll adjust my apology to being “mostly” there). In advance of our call, though I admit, not very far in advance, I have put together two straw documents. They’re rough but ready for discussion. Last week I committed to thinking about two things: How we could streamline the sunrise duration questions, which is currently two grids. (not for discussion today, but we’ve been intending to circle back to these eventually) How to reorganize the Claims Notice questions to be situation/use case specific without diving into hypotheticals (I think we’re picking up here today.) [Here, you will notice I removed the reference to virtue.food as I still don’t think it adds anything, but as I said before, I’m not going to die on that hill.] I’m not wedded to these docs but they do reflect my current best suggestion for how to get this information. We will definitely need to review the specific line items and tweak and wordsmith these and of course, we’ll need AG to evaluate the likelihood these will elicit the information we want. Talk to you all soon, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Julie Hedlund Sent: Monday, June 18, 2018 6:45 AM To: gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] REMINDER: Proposed Agenda and Documents: Meeting 18 June at 1600 UTC Dear Data Sub Team Members, In preparation for today’s (Monday) meeting (18 June) at 16:00 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/18K-zhP541IGCR0bXJpBtdmWdpfQ6BbKhFZaaZsf… [docs.google.com] 3. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… [docs.google.com] 4. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

1 0

REMINDER: Proposed Agenda and Documents: Meeting 18 June at 1600 UTC
by Julie Hedlund June 18, 2018

June 18, 2018

Dear Data Sub Team Members, In preparation for today’s (Monday) meeting (18 June) at 16:00 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/18K-zhP541IGCR0bXJpBtdmWdpfQ6BbKhFZaaZsf… 3. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 4. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

2 1

Re: [Gnso-rpm-data] [Ext] RE: RPM Data Sub Team Meeting Options: Monday, 18 June
by Julie Hedlund June 17, 2018

June 17, 2018

Thanks Kristine! Best, Julie From: "Dorrain, Kristine" <dorraink(a)amazon.com> Date: Friday, June 15, 2018 at 5:31 PM To: Julie Hedlund <julie.hedlund(a)icann.org>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] RE: RPM Data Sub Team Meeting Options: Monday, 18 June I can do either option 2 or option 3. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Julie Hedlund Sent: Friday, June 15, 2018 12:03 PM To: Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] RPM Data Sub Team Meeting Options: Monday, 18 June Importance: High Hello everyone, As we noted on the call and per the notes below, several Sub Team members are not available at the suggested time of 17:00-18:30 UTC on Monday, 18 June, and for others the time is not ideal. Please let us know at your earliest convenience if you would be able to attend the call if it is moved to the any of the following options: OPTION 1: 1 hour earlier, to 16:00-17:30 UTC at 90 minutes OPTION 2: 1 hour earlier to 16:00-17:00 UTC at 60 minutes OPTION 3: 2 hours earlier to 15:00-16:30 UTC at 90 minutes Thank you very much for your assistance and please be on the lookout for a meeting notice on Sunday, or very early Monday. We hope you have a nice weekend! Best, Mary, Ariel, Berry, and Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of Ariel Liang <ariel.liang(a)icann.org> Date: Friday, June 15, 2018 at 2:56 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ [community.icann.org] Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b Change 3 years to 5 years. Q2c Change 3 years to 5 years Q2e "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. Suggestions for options to be included in a multiple-choice question I would not be sued if I continued I would not be subject to an action to take the domain if I continued No one else had a legal right to the name I had legitimate or legal right to the domain It does not seem like too much trouble to continue I consulted with an attorney and it was fine I was committed to the domain name for person/business reasons It matched investments names I already own in a portfolio (going with "theme" registrants) I didn't understand the notice Something else [explain] There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. Registrants with big portfolio might actually prefer a narrative option. Q2g There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) The structure is a bit confusing Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) Re virtue.food - oppose to specify as people may give a false hypothetical May want to specify "I plan to start a business or already have a business under that name" If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. We did give examples on other surveys to target non-expert. How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: Would you be nervous to register the domain name? -- provide options If yes, what actions would you take? -- provide options If no, what is the reason? -- provide options Does trademark law matter? Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. Don't need to limit this question for the people who have not completed the registration. Proposed new order: Show trademark claims notice. Q2g(iv) – confidence? Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] Q2h(i) – Why you abandoned? Q2g(iii) – trademark law knowledge We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. Those general Qs should appear after the more specific responses to virtue.food Q2h(i) We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] AG suggestion: Keep the hypo question for the potential registrant survey In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? Is it possible that most people taking the survey are legacy domain name registrants? We account for those potential respondents and try to filter them in the survey. It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

1 1

Surveying potential registrants (Re: Actions & Notes: RPM Data Sub Team 15 June 2018)
by Mary Wong June 16, 2018

June 16, 2018

Dear all, Thank you to Rebecca, Kristine and Michael for continuing the discussion. At the end of the last sub team call, the following appeared to be the decision points under consideration for the actual and potential registrant surveys: * The “actual registrant” and “potential registrant” surveys are two separate surveys; however, for the “actual registrant” target group, it may be helpful to also have a section that mirrors part of the “potential registrant” survey, so as to capture the views of those who succeeded in registering one/some domain(s) but decided not to proceed with other attempted registrations. * In both the “actual registrant” and “potential registrant” surveys, it will be important (possibly even necessary) to make clear that the respondent knows what a Claims Notice looks like – this may entail actually showing the respondent (during the survey?) a copy of the Claims Notice. * While “use cases” and illustrative examples using a hypothetical domain name can be considered but may not ultimately be necessary, what remains critical is eliciting responses where the respondent understands what a Claims Notice is and knows what it looks like. * Some reordering/reformatting of the questions may be necessary in order to present respondents with a logical process flow and to facilitate responses. If it will be of assistance, do you wish staff to request that Greg and/or Stacey provide a more specific description, on one of your next calls with them, as to what methods they and their subcontractor will be using to identify, target and reach the category of respondents that we are describing as “actual registrants” (i.e. those that succeeded in registering domain names especially in the new gTLDs) and “potential registrants” (i.e. those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future)? Thanks and cheers Mary, Julie, Ariel & Berry From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Saturday, June 16, 2018 at 06:38 To: "Dorrain, Kristine" <dorraink(a)amazon.com>, "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks for taking the time to go through all this, Kristine – especially since my attendance today was somewhat sporadic. I do understand the scope of what we are trying to get, but I agree that we should ask specific questions about actual responses to the Claims Notice, and not ask hypotheticals. So, I can go along with asking registrants, applicants who did not obtain a registration, and potential domain name applicants what they understand the Claims Notice to say, what their reaction to it would be, etc. However, I think we need to ask the same question of registrants in two classes: Those who have received Trademark Claims Notices, and those who have not. More importantly, in asking, parsing, analyzing and attempting to draw any conclusions from the results, we must be sure that AG draws clear distinctions between the different groups being asked these questions, their answers, and the significance of their answers in light of their different levels and types of experience and concern. I am therefore agreeable, but extremely cautious in light of past surveys and analyses. Michael R. From: Dorrain, Kristine [mailto:dorraink@amazon.com] Sent: Friday, June 15, 2018 2:49 PM To: Michael Graham (ELCA) <migraham(a)expedia.com>; Tushnet, Rebecca <rtushnet(a)law.harvard.edu>; Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Hi Michael, I do understand your concern but I think we ARE providing the Analysis Group with clear instructions about who to survey and how to ask the questions – and they’ve told us how they will get the participants. That’s the point of the calls and that’s indeed what we’re discussing here, now. Correct me if I’m misunderstanding, but I take your suggestion to mean you oppose surveying potential registrants at all. I don’t think we are at liberty to do that and I’ll explain: 1. We have charter questions that ask how understandable the Claims Notice is and we have some (at least anecdotal, abandonment) information that people may have been scared away by the Claims Notice. 2. As a result, we are expressly asked by the full WG to figure out if the Claims Notice is scary or needs to be re-worded. 3. We are also asked to figure out how people from different places of the world perceive the Claims Notice, as presented and worded (ideally, in a variety of languages- I have not seen how AG plans to address this) ^^These, as I understand it, are not negotiable. These are the pieces of information the WG asked to get survey data on. Therefore, we MUST present the Claims Notice to some people (ideally, people who would actually register a domain name) to learn how people react to it. Now, we can for sure get some of this information from people who succeeded in registering domain names – they completed the process. But we are also looking to get this information from people who opted not to proceed for whatever reason, when presented with a claims notice. These are “potential registrants.” And yes, for people who actually didn’t try yet or didn’t proceed, these questions are somewhat hypothetical, but what we’re attempting to get is a REACTION and an idea of how it might affect future behavior, not past data. Think of this part as market research, not historical data. The “hypothetical” disagreement is just in the level detail, as I understand it. I’ve been an outspoken critic of hypothetical questions. My solution is to give some general use cases (you want a domain name for a blog or for your business or as an investment) and then ask if the Claims Notice would be scary or a deterrent and why. Rebecca likes the idea of being more specific (providing an example name at least, maybe more of a detailed scenario). If I’ve misunderstood, I hope you’ll set me straight. Best, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Michael Graham (ELCA) Sent: Friday, June 15, 2018 2:42 PM To: Tushnet, Rebecca <rtushnet(a)law.harvard.edu<mailto:rtushnet@law.harvard.edu>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Understood. However, this sort of research requires a clear identification of the universe to be studied, the wording that will be used, and the environment in which it will be asked. I think we need some clear parameters and questions, and I don’t think we have those yet. Hopefully after our call today, AG can provide some guidance. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:27 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 As I said on the call, this is standard comprehension research, relied on by courts and marketing departments. When you want to know what members of a group take away from a message, you show it to them and ask them about it, even if they haven't seen it before. It is precisely the kind of data driven research this working group is supposed to conduct. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 5:12:17 PM To: Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Rebecca. From your explanation, then, we do not want to ask hypothetical questions, but want to ask what subjects’ understanding of the Claims Notice is? That makes sense if asked of actual registrants or applicants who received the notices. I don’t think it makes sense, however, to ask this question of those who have not gone through the process. Then it becomes a hypothetical: If you received this notice, what does it tell you? Without the context of involvement in the process I don’t think “potential registrants” would provide relevant or useful information. I’ll go back to the questions after the weekend to see how and who they address the issue of effectiveness of the actual Claims Notice to. That will better answer my concerns I hope. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:01 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 Surveying potential registrants (defined in the survey as those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future) for their understanding of the meaning of the claims notice is not hypothetical--it is a standard reception survey of the target audience. The question is not generally what issues kept some potential registrants away--that is a good deal broader than our task, I believe. The question is what people considering new gTLD registrations take away from the Claims Notice when they read it. That is one of the key questions this working group has about the functioning of the Claims Notice. Past and future new gTLD registrants are our universe. Depending on their past experiences, including receipt of claims notices (if we get any such respondents) we may have different questions for different subgroups. But they are the target group as a whole, and should get the same questions, except for those that don't apply to them. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org<mailto:gnso-rpm-data-bounces@icann.org>> on behalf of Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 4:50:32 PM To: Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Ariel. I will review the Tape (as they say), but believe I disagree with the last point unless the meaning is that we would send a single survey to both registrants and “potential registrants” (although I would like to understand better how this last universe will be identified/selected) but according to which group they fall into, they will follow separate branches – registrants being asked whether they received Claims Notices and what they did/how they reacted; and potential registrants being asked why they did not apply for registration – i.e. what hypothetical issues could arise that kept them away. I do not think we should mix them or that we should ask purely hypothetical questions of anyone since these will only obtain opinions, beliefs, and perceptions – not what the effects of the RPMs actually have been. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, June 15, 2018 11:56 AM To: gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM * Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 * 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years * If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b * Change 3 years to 5 years. Q2c * Change 3 years to 5 years Q2e * "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. * We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. * Suggestions for options to be included in a multiple-choice question - I would not be sued if I continued - I would not be subject to an action to take the domain if I continued - No one else had a legal right to the name - I had legitimate or legal right to the domain - It does not seem like too much trouble to continue - I consulted with an attorney and it was fine - I was committed to the domain name for person/business reasons - It matched investments names I already own in a portfolio (going with "theme" registrants) - I didn't understand the notice - Something else [explain] * There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. * Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. * Registrants with big portfolio might actually prefer a narrative option. Q2g * There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) * The structure is a bit confusing * Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not * Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) * Re virtue.food - oppose to specify as people may give a false hypothetical * May want to specify "I plan to start a business or already have a business under that name" * If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. * We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. * We did give examples on other surveys to target non-expert. * How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? * We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: - Would you be nervous to register the domain name? -- provide options - If yes, what actions would you take? -- provide options - If no, what is the reason? -- provide options - Does trademark law matter? * Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. * Don't need to limit this question for the people who have not completed the registration. * Proposed new order: - Show trademark claims notice. - Q2g(iv) – confidence? - Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] - Q2h(i) – Why you abandoned? - Q2g(iii) – trademark law knowledge * We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... * One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. * Those general Qs should appear after the more specific responses to virtue.food Q2h(i) * We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? * This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions * Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. * Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. * Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. * There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions * Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions * Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. * If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? * I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. * I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 * The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. * Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. * We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. * The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] * AG suggestion: - Keep the hypo question for the potential registrant survey - In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? * Is it possible that most people taking the survey are legacy domain name registrants? * We account for those potential respondents and try to filter them in the survey. * It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

3 2

Re: [Gnso-rpm-data] [Ext] Re: Surveying potential registrants (Re: Actions & Notes: RPM Data Sub Team 15 June 2018)
by Mary Wong June 16, 2018

June 16, 2018

Hello Rebecca and everyone – please accept apologies for any lack of clarity; my note was not intended to supersede or suggest a direction for any of the ongoing discussions among and between this Sub Team and the Analysis Group. Rather, and based on a review of the call recording and the ongoing email discussions, it was an attempt to summarize where the discussion seemed to be. Please see some additional clarifying notes in blue below. I am sorry if, in an attempt to provide a quick summary that staff had hoped might facilitate your next discussion, the message may instead have been mistaken as to its intent and meaning. Please feel free to disregard the summary as a result. Best regards, Mary From: "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu> Date: Saturday, June 16, 2018 at 11:14 To: Mary Wong <mary.wong(a)icann.org>, "Michael Graham (ELCA)" <migraham(a)expedia.com>, "Dorrain, Kristine" <dorraink(a)amazon.com>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] Re: Surveying potential registrants (Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018) Mary, can you explain what you mean by decision points? MW: All that phrase was intended to convey is that these are the topics that the Sub Team discussed on the call for which the Analysis Group will likely need to either clarify or revise certain questions and their sequencing, or for which Sub Team members may agree that more specific clarity is needed. The thought was that a summary may be helpful as there seemed to be some confusion and lack of clarity as to what was presented by the Analysis Group and why certain questions were either included or phrased in a certain way or sequence. What you have presented below does not completely track with what the AG presented or my understanding of what's under discussion: For point one, "it may be helpful to have a section...." is inconsistent with the questions with which we were presented, where the two groups (which will actually be constituted by their responses to the first few questions of the survey) got a number of the same questions. Michael apparently doesn't want to survey potential registrants, but that's a very different suggestion from completely rewriting the survey tracks. MW: This was, again, an attempt to summarize the main point under discussion on this topic. It was not a staff attempt to predict or suggest what the final decision as between the Sub Team and the Analysis Group will be. As you noted, Michael had a specific question about doing two surveys which Kristine answered as part of the subsequent email discussion. There was no attempt by staff to rewrite anything or to suggest a different type or number of surveys, we are merely trying to provide a quick summary/refresher to facilitate the next call and discussion. For point two "possibly even necessary"--our mandate is, among other things, to figure out if the Claims Notice is understood by its recipients. I can't see any scenario where "possibly" is in the picture: it will absolutely be necessary to let people read the thing we're asking if they understand, or understood if they saw it in the past. (Among other things, a person who says they got a Claims Notice and then realizes that they don't recognize what they're seeing should not be counted as someone who got a Claims Notice.) I didn't understand anyone on the call to suggest that we shouldn't show the Claims Notice and I'm surprised that's entered the conversation. MW: Again, this is an attempt to summarize what the discussion was about, and to ensure that there is a clear understanding among the Sub Team and the Analysis Group that the Claims Notice must be shown as part of the survey process. I strongly disagree with your characterization of point three; we are apparently debating what it means to "understand what a Claims Notice is." Knowing what it looks like is one thing; understanding what they think it means is one of our goals in the WG. I agree with Kristine: we are charged with figuring out how recipients understand it, which can only be answered by asking what they think it means. Perhaps this should be reframed as "answering the relevant charter questions about the intelligibility of the Claims Notice." MW: I apologize if the choice of summary language was imprecise or gave the wrong impression. Again, this was a quick summary of the points that were discussed. It was not a suggestion about framing or phrasing the specific survey questions – that is for the Analysis Group to do as the professional contractor retained for this purpose. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Mary Wong <mary.wong(a)icann.org> Sent: Friday, June 15, 2018 10:33 PM To: Michael Graham (ELCA); Dorrain, Kristine; Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org Subject: Surveying potential registrants (Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018) Dear all, Thank you to Rebecca, Kristine and Michael for continuing the discussion. At the end of the last sub team call, the following appeared to be the decision points under consideration for the actual and potential registrant surveys: * The “actual registrant” and “potential registrant” surveys are two separate surveys; however, for the “actual registrant” target group, it may be helpful to also have a section that mirrors part of the “potential registrant” survey, so as to capture the views of those who succeeded in registering one/some domain(s) but decided not to proceed with other attempted registrations. * In both the “actual registrant” and “potential registrant” surveys, it will be important (possibly even necessary) to make clear that the respondent knows what a Claims Notice looks like – this may entail actually showing the respondent (during the survey?) a copy of the Claims Notice. * While “use cases” and illustrative examples using a hypothetical domain name can be considered but may not ultimately be necessary, what remains critical is eliciting responses where the respondent understands what a Claims Notice is and knows what it looks like. * Some reordering/reformatting of the questions may be necessary in order to present respondents with a logical process flow and to facilitate responses. If it will be of assistance, do you wish staff to request that Greg and/or Stacey provide a more specific description, on one of your next calls with them, as to what methods they and their subcontractor will be using to identify, target and reach the category of respondents that we are describing as “actual registrants” (i.e. those that succeeded in registering domain names especially in the new gTLDs) and “potential registrants” (i.e. those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future)? Thanks and cheers Mary, Julie, Ariel & Berry From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Saturday, June 16, 2018 at 06:38 To: "Dorrain, Kristine" <dorraink(a)amazon.com>, "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks for taking the time to go through all this, Kristine – especially since my attendance today was somewhat sporadic. I do understand the scope of what we are trying to get, but I agree that we should ask specific questions about actual responses to the Claims Notice, and not ask hypotheticals. So, I can go along with asking registrants, applicants who did not obtain a registration, and potential domain name applicants what they understand the Claims Notice to say, what their reaction to it would be, etc. However, I think we need to ask the same question of registrants in two classes: Those who have received Trademark Claims Notices, and those who have not. More importantly, in asking, parsing, analyzing and attempting to draw any conclusions from the results, we must be sure that AG draws clear distinctions between the different groups being asked these questions, their answers, and the significance of their answers in light of their different levels and types of experience and concern. I am therefore agreeable, but extremely cautious in light of past surveys and analyses. Michael R. From: Dorrain, Kristine [mailto:dorraink@amazon.com] Sent: Friday, June 15, 2018 2:49 PM To: Michael Graham (ELCA) <migraham(a)expedia.com>; Tushnet, Rebecca <rtushnet(a)law.harvard.edu>; Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Hi Michael, I do understand your concern but I think we ARE providing the Analysis Group with clear instructions about who to survey and how to ask the questions – and they’ve told us how they will get the participants. That’s the point of the calls and that’s indeed what we’re discussing here, now. Correct me if I’m misunderstanding, but I take your suggestion to mean you oppose surveying potential registrants at all. I don’t think we are at liberty to do that and I’ll explain: 1. We have charter questions that ask how understandable the Claims Notice is and we have some (at least anecdotal, abandonment) information that people may have been scared away by the Claims Notice. 2. As a result, we are expressly asked by the full WG to figure out if the Claims Notice is scary or needs to be re-worded. 3. We are also asked to figure out how people from different places of the world perceive the Claims Notice, as presented and worded (ideally, in a variety of languages- I have not seen how AG plans to address this) ^^These, as I understand it, are not negotiable. These are the pieces of information the WG asked to get survey data on. Therefore, we MUST present the Claims Notice to some people (ideally, people who would actually register a domain name) to learn how people react to it. Now, we can for sure get some of this information from people who succeeded in registering domain names – they completed the process. But we are also looking to get this information from people who opted not to proceed for whatever reason, when presented with a claims notice. These are “potential registrants.” And yes, for people who actually didn’t try yet or didn’t proceed, these questions are somewhat hypothetical, but what we’re attempting to get is a REACTION and an idea of how it might affect future behavior, not past data. Think of this part as market research, not historical data. The “hypothetical” disagreement is just in the level detail, as I understand it. I’ve been an outspoken critic of hypothetical questions. My solution is to give some general use cases (you want a domain name for a blog or for your business or as an investment) and then ask if the Claims Notice would be scary or a deterrent and why. Rebecca likes the idea of being more specific (providing an example name at least, maybe more of a detailed scenario). If I’ve misunderstood, I hope you’ll set me straight. Best, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Michael Graham (ELCA) Sent: Friday, June 15, 2018 2:42 PM To: Tushnet, Rebecca <rtushnet(a)law.harvard.edu<mailto:rtushnet@law.harvard.edu>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Understood. However, this sort of research requires a clear identification of the universe to be studied, the wording that will be used, and the environment in which it will be asked. I think we need some clear parameters and questions, and I don’t think we have those yet. Hopefully after our call today, AG can provide some guidance. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:27 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 As I said on the call, this is standard comprehension research, relied on by courts and marketing departments. When you want to know what members of a group take away from a message, you show it to them and ask them about it, even if they haven't seen it before. It is precisely the kind of data driven research this working group is supposed to conduct. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 5:12:17 PM To: Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Rebecca. From your explanation, then, we do not want to ask hypothetical questions, but want to ask what subjects’ understanding of the Claims Notice is? That makes sense if asked of actual registrants or applicants who received the notices. I don’t think it makes sense, however, to ask this question of those who have not gone through the process. Then it becomes a hypothetical: If you received this notice, what does it tell you? Without the context of involvement in the process I don’t think “potential registrants” would provide relevant or useful information. I’ll go back to the questions after the weekend to see how and who they address the issue of effectiveness of the actual Claims Notice to. That will better answer my concerns I hope. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:01 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 Surveying potential registrants (defined in the survey as those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future) for their understanding of the meaning of the claims notice is not hypothetical--it is a standard reception survey of the target audience. The question is not generally what issues kept some potential registrants away--that is a good deal broader than our task, I believe. The question is what people considering new gTLD registrations take away from the Claims Notice when they read it. That is one of the key questions this working group has about the functioning of the Claims Notice. Past and future new gTLD registrants are our universe. Depending on their past experiences, including receipt of claims notices (if we get any such respondents) we may have different questions for different subgroups. But they are the target group as a whole, and should get the same questions, except for those that don't apply to them. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org<mailto:gnso-rpm-data-bounces@icann.org>> on behalf of Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 4:50:32 PM To: Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Ariel. I will review the Tape (as they say), but believe I disagree with the last point unless the meaning is that we would send a single survey to both registrants and “potential registrants” (although I would like to understand better how this last universe will be identified/selected) but according to which group they fall into, they will follow separate branches – registrants being asked whether they received Claims Notices and what they did/how they reacted; and potential registrants being asked why they did not apply for registration – i.e. what hypothetical issues could arise that kept them away. I do not think we should mix them or that we should ask purely hypothetical questions of anyone since these will only obtain opinions, beliefs, and perceptions – not what the effects of the RPMs actually have been. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, June 15, 2018 11:56 AM To: gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM * Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 * 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years * If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b * Change 3 years to 5 years. Q2c * Change 3 years to 5 years Q2e * "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. * We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. * Suggestions for options to be included in a multiple-choice question - I would not be sued if I continued - I would not be subject to an action to take the domain if I continued - No one else had a legal right to the name - I had legitimate or legal right to the domain - It does not seem like too much trouble to continue - I consulted with an attorney and it was fine - I was committed to the domain name for person/business reasons - It matched investments names I already own in a portfolio (going with "theme" registrants) - I didn't understand the notice - Something else [explain] * There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. * Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. * Registrants with big portfolio might actually prefer a narrative option. Q2g * There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) * The structure is a bit confusing * Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not * Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) * Re virtue.food - oppose to specify as people may give a false hypothetical * May want to specify "I plan to start a business or already have a business under that name" * If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. * We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. * We did give examples on other surveys to target non-expert. * How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? * We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: - Would you be nervous to register the domain name? -- provide options - If yes, what actions would you take? -- provide options - If no, what is the reason? -- provide options - Does trademark law matter? * Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. * Don't need to limit this question for the people who have not completed the registration. * Proposed new order: - Show trademark claims notice. - Q2g(iv) – confidence? - Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] - Q2h(i) – Why you abandoned? - Q2g(iii) – trademark law knowledge * We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... * One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. * Those general Qs should appear after the more specific responses to virtue.food Q2h(i) * We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? * This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions * Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. * Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. * Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. * There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions * Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions * Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. * If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? * I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. * I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 * The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. * Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. * We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. * The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] * AG suggestion: - Keep the hypo question for the potential registrant survey - In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? * Is it possible that most people taking the survey are legacy domain name registrants? * We account for those potential respondents and try to filter them in the survey. * It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

1 0

Recordings, attendance & AC chat from the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 15 June 2018
by Michelle DeSmyter June 16, 2018

June 16, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 15 June 2018 at 16:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/ZYMpBQ MP3: https://audio.icann.org/gnso/gnso-review-rpms-pdp-sub-team-data-15jun18-en.… AC Recording: https://participate.icann.org/p7igmayvcel/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar [gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Michelle

1 0