February 2018 - Gnso-rpm-data

Re: [Gnso-rpm-data] Rationales for/against use of hypotheticals
by Ariel Liang Feb. 28, 2018

Feb. 28, 2018

Hello Kristine, Thank you and Rebecca for submitting the text! If no objection from the Sub Team, I will append this text on the Data Table (on a separate page, right after the suggested hypothetical question), and circulate the updated Data Table with the full WG. The interactive phase in the RFP was closed yesterday (ideally we should seize all communications with the vendor until 9 March), but I will ask Procurement whether they could forward this comment individually to the vendors. All – if you have any comment/objection to this text, please be so kind to share with the group, so that we could forward the agreed text to the vendors asap. Thank you, Ariel From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Dorrain, Kristine via Gnso-rpm-data" <gnso-rpm-data(a)icann.org> Reply-To: "Dorrain, Kristine" <dorraink(a)amazon.com> Date: Tuesday, February 27, 2018 at 5:41 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Rationales for/against use of hypotheticals Hi team, Rebecca (timely) submitted her text and I have (belatedly) attached mine (thanks for your valuable contributions, Susan). Please see the language below. If the group does not object, I will ask staff to append this to the email with the data table that was circulated to the broader group. Julie/Ariel, I’m not sure what the best method is for making sure everyone has had a chance to see this. Best, Kristine ******************** To the RPMs WG: Our subteam largely achieved consensus on the data questions as submitted to you. Our primary disagreement was in determining if we should include hypotheticals in the survey in the “questions for potential registrants” section. The full WG will need to determine whether to include hypothetical questions in the data survey. We have prepared two rationales: one in support of hypotheticals, and one opposed; both are provided below. We hope you will consider our viewpoints when deciding. Thanks, RPMs Data Subteam ************************* Rationale SUPPORTING use of hypotheticals: Although asking respondents to interpret the notice in the abstract will provide some information about how they understand it, they will still be answering an abstract question. When we ask what they would do in the abstract if they received such a notice when they were trying to register a domain name, we don’t know what kind of domain name they imagine themselves trying to register. To deal with that variance, concrete scenarios can provide additional information. In particular, a key question we want answers to is whether the current notice produces overdeterrence. Would a person who should win a URS/UDRP if one were actually brought understand that they have the right to proceed with a registration? Secondarily: would a person who should lose understand that a rights holder is going to be notified? We don’t need to ask about tough cases to get better information about that; we can ask about clear cases, including providing information about the intended use as well as the domain name, e.g., assuming there’s a famous brand of Horse Computers, horse.farm for a farm offering horse rides and horse.computers for offering a competing computer brand. (Such questions should rotate among respondents and otherwise follow survey best practices to avoid leading questions.) Rationale OPPOSING use of hypotheticals: Hypotheticals require the WG to make too many assumptions. Let’s take the example of apple.farm and apple.computer (for illustrative purposes only). We cannot assume that the registrant of apple.farm is not intending to register and use the domain name in bad faith, just like we cannot assume the registrant of apple.computer is intending to register and use the domain name in bad faith. If we say “when presented by this notice [display notice], would you feel confident registering a) apple.farm , b) apple.computer, c) both, d) neither?” we learn nothing. Because our interpretation of what this means relies our own qualitative opinion that the survey taker is not privy to. The notice should deter someone with bad faith intent; but we cannot know or determine any survey taker’s actual intent from their response to this type of question. On the other hand, we can get information about what they’re thinking by asking the following user stories that go to state of mind: “If you were presented with this notice [display notice] when registering a domain name, would you a) proceed to register, b) stop trying to register the name, c) stop for now and maybe come back after you’ve done some research, or d) other [explain]?” What if you were: 1. Getting a domain name for a business you’d already started? [Proceed/stop/research/other] 1. Where that business is in the same field of activity as the brand that was the subject of the notice [because even if you plan to run a business it doesn’t assume that you’ve chosen an appropriate name] 2. Where that business is in an unrelated field 1. Getting a domain name for a business you were thinking about starting? [Proceed/stop/research/other] * Where that business will be in the same field of activity as the brand that was the subject of the notice * Where that business will be in an unrelated field 1. Getting a domain name for fun (family use, email address, vanity) [Proceed/stop/research/other] 2. Getting a domain name as an investment (to maybe sell or use later)? [Proceed/stop/research/other] What concerns do you have about your next steps? [free form space]

1 1

Rationales for/against use of hypotheticals
by Dorrain, Kristine Feb. 27, 2018

Feb. 27, 2018

Hi team, Rebecca (timely) submitted her text and I have (belatedly) attached mine (thanks for your valuable contributions, Susan). Please see the language below. If the group does not object, I will ask staff to append this to the email with the data table that was circulated to the broader group. Julie/Ariel, I'm not sure what the best method is for making sure everyone has had a chance to see this. Best, Kristine ******************** To the RPMs WG: Our subteam largely achieved consensus on the data questions as submitted to you. Our primary disagreement was in determining if we should include hypotheticals in the survey in the "questions for potential registrants" section. The full WG will need to determine whether to include hypothetical questions in the data survey. We have prepared two rationales: one in support of hypotheticals, and one opposed; both are provided below. We hope you will consider our viewpoints when deciding. Thanks, RPMs Data Subteam ************************* Rationale SUPPORTING use of hypotheticals: Although asking respondents to interpret the notice in the abstract will provide some information about how they understand it, they will still be answering an abstract question. When we ask what they would do in the abstract if they received such a notice when they were trying to register a domain name, we don't know what kind of domain name they imagine themselves trying to register. To deal with that variance, concrete scenarios can provide additional information. In particular, a key question we want answers to is whether the current notice produces overdeterrence. Would a person who should win a URS/UDRP if one were actually brought understand that they have the right to proceed with a registration? Secondarily: would a person who should lose understand that a rights holder is going to be notified? We don't need to ask about tough cases to get better information about that; we can ask about clear cases, including providing information about the intended use as well as the domain name, e.g., assuming there's a famous brand of Horse Computers, horse.farm for a farm offering horse rides and horse.computers for offering a competing computer brand. (Such questions should rotate among respondents and otherwise follow survey best practices to avoid leading questions.) Rationale OPPOSING use of hypotheticals: Hypotheticals require the WG to make too many assumptions. Let's take the example of apple.farm and apple.computer (for illustrative purposes only). We cannot assume that the registrant of apple.farm is not intending to register and use the domain name in bad faith, just like we cannot assume the registrant of apple.computer is intending to register and use the domain name in bad faith. If we say "when presented by this notice [display notice], would you feel confident registering a) apple.farm , b) apple.computer, c) both, d) neither?" we learn nothing. Because our interpretation of what this means relies our own qualitative opinion that the survey taker is not privy to. The notice should deter someone with bad faith intent; but we cannot know or determine any survey taker's actual intent from their response to this type of question. On the other hand, we can get information about what they're thinking by asking the following user stories that go to state of mind: "If you were presented with this notice [display notice] when registering a domain name, would you a) proceed to register, b) stop trying to register the name, c) stop for now and maybe come back after you've done some research, or d) other [explain]?" What if you were: a) Getting a domain name for a business you'd already started? [Proceed/stop/research/other] i. Where that business is in the same field of activity as the brand that was the subject of the notice [because even if you plan to run a business it doesn't assume that you've chosen an appropriate name] ii. Where that business is in an unrelated field b) Getting a domain name for a business you were thinking about starting? [Proceed/stop/research/other] i. Where that business will be in the same field of activity as the brand that was the subject of the notice ii. Where that business will be in an unrelated field c) Getting a domain name for fun (family use, email address, vanity) [Proceed/stop/research/other] d) Getting a domain name as an investment (to maybe sell or use later)? [Proceed/stop/research/other] What concerns do you have about your next steps? [free form space]

1 0

Re: [Gnso-rpm-data] [Ext] Re: Inquiry re: Next Meeting per Action Items from 23 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 26, 2018

Feb. 26, 2018

Dear All, Many thanks for all who have responded to the inquiry. Several Sub Team members have indicated support for a meeting this Friday, 02 March, at 1700 UTC so we’ll go ahead and get that on the schedule. With respect to a meeting at ICANN61, we do still have the possibility of using one of the four sessions as a working session. However, please note that we will not be able to change the room setup, which is for a U-shape plus additional seating. We can change the description of the meeting to emphasize that it is a working session. With respect to holding a meeting outside of one of the public session slots, there are a few things we’ll need to keep in mind. First, we would need to book a “pop-up” room and these will not be available to schedule until Friday, 09 March. Second, pop-up rooms do not have remote access, and any recordings would have to be done via the Adobe Connect room. So, even if only a small number of Sub Team members are meeting, if the plan is to make it a working session it should still be a formal meeting whereby others in the full Working Group would be able to know what was discussed, and notes and recordings should be posted on the wiki. Perhaps the Sub Team members may wish to decide whether a meeting at ICANN61 is needed based on the results of a call this Friday. Thanks again, Julie From: Susan Payne <susan.payne(a)valideus.com> Date: Monday, February 26, 2018 at 6:29 AM To: Kurt Pritz <kurt(a)kjpritz.com> Cc: Julie Hedlund <julie.hedlund(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] Re: [Gnso-rpm-data] Inquiry re: Next Meeting per Action Items from 23 February 2018 RPM Data Sub Team Call I can meet Friday, and happy to try a pirate meeting if one of the allocated RPM slots isn't suitable. Susan Payne Head of Legal Policy | Valideus Ltd 28-30 Little Russell Street London, WC1A 2HN, United Kingdom E: susan.payne(a)valideus.com D: +44 20 7421 8255 T: +44 20 7421 8299 M: +44 7971 661175 Sent from my iPhone On 26 Feb 2018, at 00:12, Kurt Pritz <kurt(a)kjpritz.com> wrote: Please see below. I am for Friday, but we should still meet at ICANN. At the end of the last session we discussed this and I (not jokingly) suggested “pirate meeting." If you take out the participation of the working group chairs there are only 5 or 6 of us. Why cannot we meet over lunch or in the bar at sometime. I suggest we meet in the restaurant or bar for lunch coffee or drinks and go through a pre-developed agenda. We don”t have the need for recording etc, just an informal venue where we might be effective. What do you think? Kurt On Feb 23, 2018, at 1:06 PM, Julie Hedlund <julie.hedlund(a)icann.org> wrote: All, Per this action item: Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61, please indicate below whether you wish to hold a meeting next Friday, 02 March, at the usual time of 1700 UTC and/or a working session at ICANN61. Note that separately you will receive a poll asking about your attendance for the meetings during ICANN61. The topic of the meeting would be the continued discussion of the additional RPMs. __X__ Meet next Friday, 02 March at 1700 UTC ____ Meet at ICANN61 using one of the 4 public sessions (the session description would specifically note that this is a working session) Thank you for your assistance. Kind regards, Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of Julie Hedlund <julie.hedlund(a)icann.org> Date: Friday, February 23, 2018 at 1:38 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Action Items from 23 February 2018 RPM Data Sub Team Call All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (23 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Data Table: Michael to email the group his suggestions for deleting duplication on page 26, and make the changes to the Google Doc (DONE). Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. Additional RPMs: 1) Add questions 4 and 5 to the Data Table if they are not already covered. 2) Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. Staff will update the Data Table based on the above input and the actions from the discussion during the meeting, send the Data Table to the full Working Group, and send it to ICANN Procurement staff. Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61 Notes: 1. Discussion of the remaining part of the Data Table -- Review of the Section 4 (Registrant Questions) See: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN… If you attempted to register a domain name, did you receive an objection based on trademark to your choice of domain name?... Action: Fine to add, but delete from "based on..." and add "If you received an objection, what was the basis?" To identify people who may not know what the specific name of the process they triggered was..." Actions: Ask “Did you receive a warning or notice of possible trademark conflict?” These are guidance for the survey provider to provide “cues” for the respondents. Discussion: -- Put this in the guidance for the survey provider. -- Should we keep this narrow, focusing on trademark claims and Sunrise? I think objection, to a registrant -- just say "did you receive this notice" and keep the exact words of the notice in. Unless we put this more vague question at the end. -- Assuming we are looking for who received a claims notice and what did they do. "objection" could mean all sorts of things. We want to find out did they receive a claims notice and what they did in response. -- Disagree. Want to know about objections. Asking about the notice in column three, but there could be a wide variety of objections. -- Think we have a disagreement on methodology. Don't think we should put the text of the notice to them, but get them thinking about the context. The surveyor can help finalize this. Did you receive this before you completed the registration and we can ask what happened. If you asked if you received the notice then they might not be able to tell if the wording is the same. Let them think about it and their answers may reveal what they received. -- Agree with not using the word objection. Some people may not understand what an objection or trademark notice is. Ask them questions about the notice and find out if they misperceived the notice, or even knew if it was an objection. -- Page 26: Rather than "did you receive an objection" did you receive a warning notice. It is outside scope to start asking questions about post registration whether they got a cease and desist letter. I think we are trying to determine if they triggered a notice, did they understand it, what did they do. -- Be careful about using “objection”, “protest”, or “challenge” since that hasn't happened yet. The rights holder hasn't made the objection. -- Go to page 29 -- captures where we could come out. Three buckets of people. Ask all of the questions about the language of the claims notice. Realize that people don't just sit in one of these buckets. We aren't the survey operators, we are trying to explain what we are seeking. -- Didn't want to lose potential registrants who tried to register but couldn't get through. Did they receive a warning or notice of possible trademark conflict. -- Left with whether or not we want to ask about other RPMs. Sounds like keeping it to trademark and Sunrise. -- At some point we do need to ask if what they received was a Trademark Claims Notice or something else. -- Notes for survey provider: preparing the survey people for the different kinds of things they might want to elicit. Prepare the respondents. The option that this is the type of objection/notice I received, or other. Include "cues". Section 6: Actions: -- Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. -- Add questions 4 and 5 if they are not already covered. Discussion: -- Have done some thinking about it. -- Don't understand why we are doing a separate set of questions for public interest groups and trade associations. -- The thinking was that we wanted to capture as many groups as possible. One of the challenges in looking at the table, except for very large non-profits most would not be able to answer the questions. It would depend on who we reached out to. So, unless we reach out to very large non-profits (such as Red Cross, March of Dimes, YMCA, etc.) you aren't going to get answers to the questions. -- Problematic that we would be asking different questions of that group since we wouldn't be able to compare to the rest of the groups. -- Include a note to the survey provider that NGOs may need a bit more coaching and background. So they can help us identify who could be reached for the survey. Teese out a classification of users among the public interest groups. -- Another General Note to keep the inconsistency in the Claims Notices, Trademark Notification, NORNs. -- Redundancy in Section 3, page 21, third bullet point in 3rd column; 4th column, primary method. Michael to email the group about his suggestions and make the change to the Google Doc 2. Check the Additional Marketplace RPMs and see whether Qs fall within scope: Action: Add questions 4 and 5 to the Data Table -- Question 1 and 3 do not belong to the survey (out of scope) -- Question 4 * Could possibly ask the ROs, but not in the survey as it is not related Sunrise and Trademark Claims * These questions have impact on Sunrise and Trademark Claims. Two for ROs, one for registrar, and one for TM owners. Make sense to tag these onto the survey * GNSO Council did not give sign off to add these questions from additional marketplace RPMs. * Questions can only be added if it is specifically in scope for the RFP, fitting into the Sunrise and Trademark Claims question table. * ACTION: Question 4 to add to the TM owner section, related to Sunrise -- Question 5 * It is more germane * ACTION: Within scope, and can be added to the table, related to Trademark Claims, aim at ROs and Registrars -- No other questions in the additional RPMs need to be included in the survey? -- No more time for this survey to go through the additional RPMs thoroughly. In early stages of the PDP, questions were sent to ROs that ran these additional RPMs. Donuts and one/two others provided feedback. -- ACTION: Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. -- Timing of future meeting. ICANN61 Planning / repurposing a session for the Data Sub Team? -- Participation in ICANN61 * Susan, Kristin * Kurt (leaving Wed AM) * Michael (remote participation) _______________________________________________ Gnso-rpm-data mailing list Gnso-rpm-data(a)icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-data _______________________________________________ Gnso-rpm-data mailing list Gnso-rpm-data(a)icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-data

2 1

Inquiry re: Next Meeting per Action Items from 23 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 26, 2018

Feb. 26, 2018

All, Per this action item: Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61, please indicate below whether you wish to hold a meeting next Friday, 02 March, at the usual time of 1700 UTC and/or a working session at ICANN61. Note that separately you will receive a poll asking about your attendance for the meetings during ICANN61. The topic of the meeting would be the continued discussion of the additional RPMs. ____ Meet next Friday, 02 March at 1700 UTC ____ Meet at ICANN61 using one of the 4 public sessions (the session description would specifically note that this is a working session) Thank you for your assistance. Kind regards, Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of Julie Hedlund <julie.hedlund(a)icann.org> Date: Friday, February 23, 2018 at 1:38 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Action Items from 23 February 2018 RPM Data Sub Team Call All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (23 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Data Table: Michael to email the group his suggestions for deleting duplication on page 26, and make the changes to the Google Doc (DONE). Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. Additional RPMs: 1) Add questions 4 and 5 to the Data Table if they are not already covered. 2) Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. Staff will update the Data Table based on the above input and the actions from the discussion during the meeting, send the Data Table to the full Working Group, and send it to ICANN Procurement staff. Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61 Notes: 1. Discussion of the remaining part of the Data Table -- Review of the Section 4 (Registrant Questions) See: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN… If you attempted to register a domain name, did you receive an objection based on trademark to your choice of domain name?... Action: Fine to add, but delete from "based on..." and add "If you received an objection, what was the basis?" To identify people who may not know what the specific name of the process they triggered was..." Actions: Ask “Did you receive a warning or notice of possible trademark conflict?” These are guidance for the survey provider to provide “cues” for the respondents. Discussion: -- Put this in the guidance for the survey provider. -- Should we keep this narrow, focusing on trademark claims and Sunrise? I think objection, to a registrant -- just say "did you receive this notice" and keep the exact words of the notice in. Unless we put this more vague question at the end. -- Assuming we are looking for who received a claims notice and what did they do. "objection" could mean all sorts of things. We want to find out did they receive a claims notice and what they did in response. -- Disagree. Want to know about objections. Asking about the notice in column three, but there could be a wide variety of objections. -- Think we have a disagreement on methodology. Don't think we should put the text of the notice to them, but get them thinking about the context. The surveyor can help finalize this. Did you receive this before you completed the registration and we can ask what happened. If you asked if you received the notice then they might not be able to tell if the wording is the same. Let them think about it and their answers may reveal what they received. -- Agree with not using the word objection. Some people may not understand what an objection or trademark notice is. Ask them questions about the notice and find out if they misperceived the notice, or even knew if it was an objection. -- Page 26: Rather than "did you receive an objection" did you receive a warning notice. It is outside scope to start asking questions about post registration whether they got a cease and desist letter. I think we are trying to determine if they triggered a notice, did they understand it, what did they do. -- Be careful about using “objection”, “protest”, or “challenge” since that hasn't happened yet. The rights holder hasn't made the objection. -- Go to page 29 -- captures where we could come out. Three buckets of people. Ask all of the questions about the language of the claims notice. Realize that people don't just sit in one of these buckets. We aren't the survey operators, we are trying to explain what we are seeking. -- Didn't want to lose potential registrants who tried to register but couldn't get through. Did they receive a warning or notice of possible trademark conflict. -- Left with whether or not we want to ask about other RPMs. Sounds like keeping it to trademark and Sunrise. -- At some point we do need to ask if what they received was a Trademark Claims Notice or something else. -- Notes for survey provider: preparing the survey people for the different kinds of things they might want to elicit. Prepare the respondents. The option that this is the type of objection/notice I received, or other. Include "cues". Section 6: Actions: -- Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. -- Add questions 4 and 5 if they are not already covered. Discussion: -- Have done some thinking about it. -- Don't understand why we are doing a separate set of questions for public interest groups and trade associations. -- The thinking was that we wanted to capture as many groups as possible. One of the challenges in looking at the table, except for very large non-profits most would not be able to answer the questions. It would depend on who we reached out to. So, unless we reach out to very large non-profits (such as Red Cross, March of Dimes, YMCA, etc.) you aren't going to get answers to the questions. -- Problematic that we would be asking different questions of that group since we wouldn't be able to compare to the rest of the groups. -- Include a note to the survey provider that NGOs may need a bit more coaching and background. So they can help us identify who could be reached for the survey. Teese out a classification of users among the public interest groups. -- Another General Note to keep the inconsistency in the Claims Notices, Trademark Notification, NORNs. -- Redundancy in Section 3, page 21, third bullet point in 3rd column; 4th column, primary method. Michael to email the group about his suggestions and make the change to the Google Doc 2. Check the Additional Marketplace RPMs and see whether Qs fall within scope: Action: Add questions 4 and 5 to the Data Table -- Question 1 and 3 do not belong to the survey (out of scope) -- Question 4 * Could possibly ask the ROs, but not in the survey as it is not related Sunrise and Trademark Claims * These questions have impact on Sunrise and Trademark Claims. Two for ROs, one for registrar, and one for TM owners. Make sense to tag these onto the survey * GNSO Council did not give sign off to add these questions from additional marketplace RPMs. * Questions can only be added if it is specifically in scope for the RFP, fitting into the Sunrise and Trademark Claims question table. * ACTION: Question 4 to add to the TM owner section, related to Sunrise -- Question 5 * It is more germane * ACTION: Within scope, and can be added to the table, related to Trademark Claims, aim at ROs and Registrars -- No other questions in the additional RPMs need to be included in the survey? -- No more time for this survey to go through the additional RPMs thoroughly. In early stages of the PDP, questions were sent to ROs that ran these additional RPMs. Donuts and one/two others provided feedback. -- ACTION: Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. -- Timing of future meeting. ICANN61 Planning / repurposing a session for the Data Sub Team? -- Participation in ICANN61 * Susan, Kristin * Kurt (leaving Wed AM) * Michael (remote participation)

7 7

Re: [Gnso-rpm-data] Google Doc Closes in 12 Hours & Suggested Agenda
by Julie Hedlund Feb. 23, 2018

Feb. 23, 2018

Many thanks Michael. This is quite helpful. Kind regards, Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Friday, February 23, 2018 at 1:04 PM To: Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Google Doc Closes in 12 Hours & Suggested Agenda Further to my suggestions online, I attach page 21 from our chart of questions showing the two deletions I would like to make. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Thursday, February 22, 2018 9:02 PM To: gnso-rpm-data(a)icann.org Subject: Re: [Gnso-rpm-data] Google Doc Closes in 12 Hours & Suggested Agenda Hello everyone, Please note that the Google Doc has been changed to “view only”. Kindly find attached the PDF document that records the redlines/comments made by 05:00 UTC today on the Google Doc. At the time of the meeting, I will revert the Google Doc back to “comment” to facilitate the discussions. Thank you! Ariel From: Ariel Liang <ariel.liang(a)icann.org> Date: Thursday, February 22, 2018 at 11:45 AM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Google Doc Closes in 12 Hours & Suggested Agenda Hello everyone, Kindly note that staff have incorporated Sub Team’s suggestions and comments for Section 5 (brought back the deleted questions on page 34-35 and relocated the introductory text above these questions as a comment): https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN…. We will close “commenting/suggesting” on the Data Request Google doc at around 05:00 UTC on Friday, 23 February. We will also distribute a PDF document that records the redlines and comments made on the Google Doc for reference. As a friendly reminder (esp. for Kristine, Susan, and Rebecca), with respect to the hypothetical question on page 35-36, kindly provide your summary of rationale on why such question would be useful or not useful. Tomorrow’s meeting would be the last one before we forward the updated Data Table to the responding vendors. By EOB tomorrow, the updated Data Table will need to be shared with the full WG and sent to Procurement. The remaining part of the Data Table is Section 4 (page 28 et seq) and Section 6 (currently blank). We would like to get the Sub Team’s input on this – Should we: Try to tackle the remaining part and develop some agreed suggested questions? Incorporate existing input and comment in Section 4 as is, leave Section 6 blank, and still include the instruction text previously drafted by Kurt/Rebecca/Kathy for Section 4? Leave Section 4 and Section 6 blank, and include the instruction text previously drafted by Kurt/Rebecca/Kathy for Section 4? Furthermore, if the Data Sub Team intend to incorporate any survey questions related to the Additional Marketplace RPMs, tomorrow’s meeting is the last opportunity to provide any high-level topics to be incorporated in the Data Table. If we do so, they must be within scope of the survey project. Last, it would be very useful if the Sub Team could do a quick review of the Data Table from the beginning, making sure everything is in order before it is distributed. Staff would like to suggest the following agenda for tomorrow’s discussion: Discussion of the remaining part of the Data Table (Section 4, Section 6); whether to include any high-level topics related to Additional Marketplace RPMs Last review/sanity check of the Data Table Timing of future meetings / ICANN61 (whether to repurpose one session for the Data Sub Team) AOB Thank you for your time and contributions! Best Regards, Mary, Julie, Berry, and Ariel

4 3

Google Doc Closes in 12 Hours & Suggested Agenda
by Ariel Liang Feb. 23, 2018

Feb. 23, 2018

Hello everyone, Kindly note that staff have incorporated Sub Team’s suggestions and comments for Section 5 (brought back the deleted questions on page 34-35 and relocated the introductory text above these questions as a comment): https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN…. We will close “commenting/suggesting” on the Data Request Google doc at around 05:00 UTC on Friday, 23 February. We will also distribute a PDF document that records the redlines and comments made on the Google Doc for reference. As a friendly reminder (esp. for Kristine, Susan, and Rebecca), with respect to the hypothetical question on page 35-36, kindly provide your summary of rationale on why such question would be useful or not useful. Tomorrow’s meeting would be the last one before we forward the updated Data Table to the responding vendors. By EOB tomorrow, the updated Data Table will need to be shared with the full WG and sent to Procurement. The remaining part of the Data Table is Section 4 (page 28 et seq) and Section 6 (currently blank). We would like to get the Sub Team’s input on this – Should we: 1. Try to tackle the remaining part and develop some agreed suggested questions? 2. Incorporate existing input and comment in Section 4 as is, leave Section 6 blank, and still include the instruction text previously drafted by Kurt/Rebecca/Kathy for Section 4? 3. Leave Section 4 and Section 6 blank, and include the instruction text previously drafted by Kurt/Rebecca/Kathy for Section 4? Furthermore, if the Data Sub Team intend to incorporate any survey questions related to the Additional Marketplace RPMs, tomorrow’s meeting is the last opportunity to provide any high-level topics to be incorporated in the Data Table. If we do so, they must be within scope of the survey project. Last, it would be very useful if the Sub Team could do a quick review of the Data Table from the beginning, making sure everything is in order before it is distributed. Staff would like to suggest the following agenda for tomorrow’s discussion: 1. Discussion of the remaining part of the Data Table (Section 4, Section 6); whether to include any high-level topics related to Additional Marketplace RPMs 2. Last review/sanity check of the Data Table 3. Timing of future meetings / ICANN61 (whether to repurpose one session for the Data Sub Team) 4. AOB Thank you for your time and contributions! Best Regards, Mary, Julie, Berry, and Ariel

6 6

Recordings, attendance & AC chat for Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 23 February 2018
by Terri Agnew Feb. 23, 2018

Feb. 23, 2018

Dear All, Please find the attendance and AC Chat of the call attached to this email. The MP3 and Adobe Connect recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 23 February 2018 at 17:00 UTC. Attendance of the call is posted on the agenda wiki page: https://community.icann.org/x/mRa8B MP3: https://audio.icann.org/gnso/gnso-rpm-review-23feb18-en.mp3 Adobe Connect recording: <https://participate.icann.org/p2ycaw9k7mi/?OWASP_CSRFTOKEN=1606aaffffef62ce 66f816ddb69d4c60c61173e4ac6e610e3fb9c1fa3d197b3a> https://participate.icann.org/p2ycaw9k7mi/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar ** Please let me know if your name has been left off the list ** Mailing list archives: <http://mm.icann.org/pipermail/gnso-rpm-data/> http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the sub team: https://community.icann.org/x/eI1EB Thank you. With kind regards, Terri

1 0

Action Items from 23 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 23, 2018

Feb. 23, 2018

All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (23 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Data Table: Michael to email the group his suggestions for deleting duplication on page 26, and make the changes to the Google Doc (DONE). Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. Additional RPMs: 1) Add questions 4 and 5 to the Data Table if they are not already covered. 2) Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. Staff will update the Data Table based on the above input and the actions from the discussion during the meeting, send the Data Table to the full Working Group, and send it to ICANN Procurement staff. Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61 Notes: 1. Discussion of the remaining part of the Data Table -- Review of the Section 4 (Registrant Questions) See: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN… If you attempted to register a domain name, did you receive an objection based on trademark to your choice of domain name?... Action: Fine to add, but delete from "based on..." and add "If you received an objection, what was the basis?" To identify people who may not know what the specific name of the process they triggered was..." Actions: Ask “Did you receive a warning or notice of possible trademark conflict?” These are guidance for the survey provider to provide “cues” for the respondents. Discussion: -- Put this in the guidance for the survey provider. -- Should we keep this narrow, focusing on trademark claims and Sunrise? I think objection, to a registrant -- just say "did you receive this notice" and keep the exact words of the notice in. Unless we put this more vague question at the end. -- Assuming we are looking for who received a claims notice and what did they do. "objection" could mean all sorts of things. We want to find out did they receive a claims notice and what they did in response. -- Disagree. Want to know about objections. Asking about the notice in column three, but there could be a wide variety of objections. -- Think we have a disagreement on methodology. Don't think we should put the text of the notice to them, but get them thinking about the context. The surveyor can help finalize this. Did you receive this before you completed the registration and we can ask what happened. If you asked if you received the notice then they might not be able to tell if the wording is the same. Let them think about it and their answers may reveal what they received. -- Agree with not using the word objection. Some people may not understand what an objection or trademark notice is. Ask them questions about the notice and find out if they misperceived the notice, or even knew if it was an objection. -- Page 26: Rather than "did you receive an objection" did you receive a warning notice. It is outside scope to start asking questions about post registration whether they got a cease and desist letter. I think we are trying to determine if they triggered a notice, did they understand it, what did they do. -- Be careful about using “objection”, “protest”, or “challenge” since that hasn't happened yet. The rights holder hasn't made the objection. -- Go to page 29 -- captures where we could come out. Three buckets of people. Ask all of the questions about the language of the claims notice. Realize that people don't just sit in one of these buckets. We aren't the survey operators, we are trying to explain what we are seeking. -- Didn't want to lose potential registrants who tried to register but couldn't get through. Did they receive a warning or notice of possible trademark conflict. -- Left with whether or not we want to ask about other RPMs. Sounds like keeping it to trademark and Sunrise. -- At some point we do need to ask if what they received was a Trademark Claims Notice or something else. -- Notes for survey provider: preparing the survey people for the different kinds of things they might want to elicit. Prepare the respondents. The option that this is the type of objection/notice I received, or other. Include "cues". Section 6: Actions: -- Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. -- Add questions 4 and 5 if they are not already covered. Discussion: -- Have done some thinking about it. -- Don't understand why we are doing a separate set of questions for public interest groups and trade associations. -- The thinking was that we wanted to capture as many groups as possible. One of the challenges in looking at the table, except for very large non-profits most would not be able to answer the questions. It would depend on who we reached out to. So, unless we reach out to very large non-profits (such as Red Cross, March of Dimes, YMCA, etc.) you aren't going to get answers to the questions. -- Problematic that we would be asking different questions of that group since we wouldn't be able to compare to the rest of the groups. -- Include a note to the survey provider that NGOs may need a bit more coaching and background. So they can help us identify who could be reached for the survey. Teese out a classification of users among the public interest groups. -- Another General Note to keep the inconsistency in the Claims Notices, Trademark Notification, NORNs. -- Redundancy in Section 3, page 21, third bullet point in 3rd column; 4th column, primary method. Michael to email the group about his suggestions and make the change to the Google Doc 2. Check the Additional Marketplace RPMs and see whether Qs fall within scope: Action: Add questions 4 and 5 to the Data Table -- Question 1 and 3 do not belong to the survey (out of scope) -- Question 4 * Could possibly ask the ROs, but not in the survey as it is not related Sunrise and Trademark Claims * These questions have impact on Sunrise and Trademark Claims. Two for ROs, one for registrar, and one for TM owners. Make sense to tag these onto the survey * GNSO Council did not give sign off to add these questions from additional marketplace RPMs. * Questions can only be added if it is specifically in scope for the RFP, fitting into the Sunrise and Trademark Claims question table. * ACTION: Question 4 to add to the TM owner section, related to Sunrise -- Question 5 * It is more germane * ACTION: Within scope, and can be added to the table, related to Trademark Claims, aim at ROs and Registrars -- No other questions in the additional RPMs need to be included in the survey? -- No more time for this survey to go through the additional RPMs thoroughly. In early stages of the PDP, questions were sent to ROs that ran these additional RPMs. Donuts and one/two others provided feedback. -- ACTION: Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. -- Timing of future meeting. ICANN61 Planning / repurposing a session for the Data Sub Team? -- Participation in ICANN61 * Susan, Kristin * Kurt (leaving Wed AM) * Michael (remote participation)

1 0

Additional Marketplace RPMs - Survey Incorporation
by Ariel Liang Feb. 16, 2018

Feb. 16, 2018

Hello everyone, Following up on today’s call, staff would like to provide some additional clarification regarding the potential incorporation of survey questions related to the Additional Marketplace RPMs offered by some registry operators on a voluntary basis. Since the scope of work stated in the RFP<https://www.icann.org/en/system/files/files/rfp-rpm-survey-29jan18-en.pdf#p…> is to assess the efficacy of Sunrise and Trademark Claims (specifically, we stated that the Data Table should be considered as the framework for scoping the project), any potential survey question about the Additional Marketplace RPMs should not expand the scope of the project. Change of the scope would have timeline and budget impact to the RFP, as well as the PDP in general. If the Sub Team could identify questions that won’t expand the scope of the project, it would be doable for vendors to submit proposals taking into account the data collection needs for Additional Marketplace RPMs. Requirements related to the Additional Marketplace RPMs need to be incorporated ASAP in the RFP documents, no later than 23 February when we pass the updated Data Table to the Procurement Team. Due to the time limitation, the Sub Team may not be able to develop detailed guidance or propose suggested survey questions for the Additional Marketplace RPMs. Staff would recommend the Sub Team to quickly run through these Additional Marketplace RPMs, identify high-level topics for data collection (must be within the scope), and include this guidance in the Data Table. If necessary, the Sub Team can organize an extended call on Friday, 23 February (90 minutes or so) to allow more time going through the additional RPMs, as well as reviewing the Data Table one last time. Please let us know if a longer call is needed and we will update the calendar invitation. Thank you all, and have a great weekend! Best Regards, Mary, Julie, Berry, and Ariel

1 0

Action Items from 16 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 16, 2018

Feb. 16, 2018

All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (16 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Note also that the next call will be on Friday, 23 February at 1700 UTC. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Staff will update Section 5. Survey of Potential Registrants beginning on page 33 in the Google Doc based on the discussion during today’s call. With respect to hypotheticals (question on 35-36 in the Google Doc) Sub Team members should volunteer to produce two documents and circulate them on the list prior to the 23 February call: Short summary of what they hope to get from the use of hypotheticals; Short summary of why not to use a hypothetical. Kristine has volunteered for the second item. RPM Data Sub Team meeting at ICANN61: The RPM Co-Chairs will discuss whether one of the slots could be repurposed for the RPM Data Sub Team. Section 3, Survey of TM and Brand Owners: Sub Team members should review and comment on the list concerning the final anecdotal and data questions for the question “What is the evidence of harm under the existing [exact match] system?”, page 22-23. Notes: 5. Survey of Potential Registrants. Page 33 -- Inserted note to the survey providers: Language is helpful and clear. Page 34 -- Have you attempted to register a domain name in one of these new domains? [If no to both, terminate the survey] -- Thought we weren't going to suggest terminating the survey. But if they aren't considering registering and haven't tried to register then they aren't a potential registrant. Or do we still want them to look at the claims notice? If they continue with the survey, it might be good to find if the process seems confusing. -- Leave the language as is, and also give the survey provider latitude to change it. Page 34-35 -- The wording of this set of questions should be discussed with the surveyor... [etc.] -- Gather information multiple different ways. -- Question: In other cases we've been more specific; have we given enough direction here? -- Thought we were leaving in all of the variations of the questions? Examples -- deleted in the Google doc: "If, when registering your domain, you received a notice with the following wording, would you believe you were: legally allowed to continue with the registration, not legally allowed to continue, not be sure if you were legally allowed to continue?" etc. -- Bring back the deleted questions and include them in the note: The wording of this set of questions should be discussed with the surveyor. We would like to present the language of the notice to people in all relevant groups (1-3) and ask comprehension questions. The questions should be as neutral and non-leading as possible. This may involve beginning with “explain in your own words” and then asking more directed questions about perceptions of legal rights and risks, as well as “what would you do next if you received the notice when you attempted to register?” (e.g., stop, continue, consult someone else, etc.) and “why or why not?” – Anecdotal question. Other sample questions: If, when registering your domain, you received a notice with the following wording, would you believe you were: a. legally allowed to continue with the registration b. not legally allowed to continue b. not be sure if you were legally allowed to continue? If, when registering your domain, you received a notice with the following wording, would you believe you would: a. definitely get sued if you continued b. might or might not get sued c. definitely would not get sued if you continued? If, when registering your domain, you received a notice with the following wording, would you believe that: a. you had a legal right to continue with the registration? b. you might or might not have a legal right to continue with the registration? c. you had no legal right to continue with the registration? What would you do if you received a notice with the following wording: a. continue with the registration b. not continue with the registration c. consult someone else [who] something else [explain] Page 35-36: Consider offering Hypothetical: Famous maker of computers, Horse, and scenarios at extremes such as horse.computers and horse.farm -- if consumer attempting to register these received notice, what would they do? -- Did not have consensus on this. Guidance on what to present to the full WG. -- Trying to get to a general understanding of what is trademark law, but not sure offering hypotheticals is the best way to do this. Perhaps in a more direct fashion? -- Is that the purpose of the Hypothetical? Not automatically opposed to hypotheticals, but need to understand what info we're trying to get at & whether there's a less subjective way to get that info. -- The idea is to try to figure out if people would be scared away from non-infringing uses of TM terms. -- Could ask the survey provider if this helps. -- It actually sounds like we don't all have the same understanding of what info the hypotheticals are trying to elicit. -- But the problem is we can never know what intent is and even if someone says I think it's ok to get "apple.farm" if their intent is bad faith (which we would not know). -- There is no right answer. -- How useful is the data? Could also get adverse inferences. Not in agreement on what we would get by using a hypothetical. Okay if we are trying to elicite specific information that is actionable. -- Anything that helps us to make the claims notice is helpful. ACTION: Leave this suggestion (the hypothetical) and ask for a volunteer to make a short summary of what they hope to get from the hypothetical (Rebecca?). Kristine would be willing to write why not to use a hypothetical. Susan has offered to help. 2. Timing of future meetings & ICANN61 -- Try to knock the Additional Marketplace RPMs on the head as quickly as possible. -- Have a big session in PR to get it done. Make the room set up as a working session. Make it clear that people shouldn't be there if they aren't going to be working. Not a presentation of findings. Make sure we have remote participation. -- Could be difficult to schedule an additional meeting given conflicts, and “pop-up” rooms do not have remote access. -- The RPM WG has 4 session slots for a total of 6 hours; the Co-Chairs will discuss whether one of the slots could be repurposed for the RPM Data Sub Team.

3 3

Recordings, attendance & AC chat for Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 16 February 2018
by Michelle DeSmyter Feb. 16, 2018

Feb. 16, 2018

Dear All, Please find the attendance and AC Chat of the call attached to this email, and the MP3 and Adobe Connect recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 16 February 2018 at 17:00 UTC. Attendance of the call is also posted on the agenda wiki page:https://community.icann.org/x/ChO8B MP3: https://audio.icann.org/gnso/gnso-rpm-review-16feb18-en.mp3 Adobe Connect recording: https://participate.icann.org/p3grvcpu24b/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar[gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB[community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind regards, Michelle

1 0

Google Doc Closes in 8 Hours & Suggested Agenda
by Ariel Liang Feb. 16, 2018

Feb. 16, 2018

Hello everyone, Kindly note that Rebecca has revised the suggested questions for Section 5 (Potential Registrant, page 33-36) with redlines, as well as some suggested questions for Section 4 (page 27), incorporating comments/input from the Sub Team: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN… (thank you, Rebecca!). We will close “commenting/suggesting” on the Data Request Google doc at around 05:00 UTC on Friday, 16 February. We will also distribute a PDF document that records the redlines and comments made on the Google Doc for reference. Regarding tomorrow’s meeting, Staff would like to suggest that we continue the discussion of Section 4 (Registrants) led by Kurt. Kurt, kindly let us know whether this would work for you (hope your visit in Australia is going well!): 1. Continue discussion of Section 4 (Registrants) (page 28 et seq) 2. Timing of future meetings & ICANN61 3. AOB As communicated previously, Staff will provide responding vendors (eight so far) with the updated Data Table no later than Monday, 26 February. This means that the Data Team will need to complete the deliberation on the Data Table by Friday, 23 February, at the very latest (two more meetings, including tomorrow’s). By 23:59 UTC on Friday, 23 February, we will close the Google doc and produce a clean version, incorporating input and comment as is, then circulate it with the full WG and forward it to the Procurement Team. In the meantime, Sub Team members are encouraged to review and comment on the existing suggested survey questions, as well as section 6 that does not yet contain suggested survey questions. Please don’t hesitate to jump in if you have any input to fill the gap. Furthermore, one more task for the Sub Team is to examine potential data needs in relation to preliminary agreed questions for examining Additional Marketplace RPMs currently being offered on a voluntary basis by some registry operators. The Sub Team would need to determine on the timing to commence the discussion. Some the options are: a. start on Friday, 2 March (last Sub Team meeting before ICANN61), b. during ICANN61 and take a break beforehand, c. after ICANN61. If the Sub Team would like to meet in ICANN61, then the format and logistics also need to be discussed (e.g., use one of the four RPMs sessions? meet as a small group in a pop-up room?). This can be discussed in agenda no.2 tomorrow. Thank you for your time and contributions! Best Regards, Mary, Julie, Berry, and Ariel

1 1

UPDATED Recordings, attendance & AC chat Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 09 February 2018
by Michelle DeSmyter Feb. 10, 2018

Feb. 10, 2018

Dear All, Please find the attendance of the call attached to this email. The MP3, Adobe Connect recording and Adobe Connect chat below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 09 February 2018 at 17:00 UTC. MP3: https://audio.icann.org/gnso/gnso-rpm-review-09feb18-en.mp3[audio.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__audio.icann.org_gnso_g…> Adobe Connect recording: https://participate.icann.org/p9c1dyjp5c5/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page:https://gnso.icann.org/en/group-activities/calendar[gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Wiki agenda page: https://community.icann.org/x/Nw68B[community.icann.org] Thank you. Kind regards, Michelle

1 0

Recordings, attendance & AC chat Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 09 February 2018
by Michelle DeSmyter Feb. 9, 2018

Feb. 9, 2018

Dear All, Please find the attendance of the call attached to this email. The MP3, Adobe Connect recording and Adobe Connect chat below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 09 February 2018 at 17:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/Nw68B MP3: https://audio.icann.org/gnso/gnso-rpm-review-09feb18-en.mp3 Adobe Connect recording: https://participate.icann.org/p9c1dyjp5c5/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page:https://gnso.icann.org/en/group-activities/calendar[gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB[community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind regards, Michelle

1 0

Action Items from 09 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 9, 2018

Feb. 9, 2018

All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (09 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Note also that the next call will be on Friday, 16 February at 1700 UTC. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Section 5, Survey of Potential Registrants: Rebecca will revise the Google document based on the discussion on the call on 09 February and the notes (below). Section 3, Survey of TM and Brand Owners: Sub Team members should review and comment on the list concerning the final anecdotal and data questions for the question “What is the evidence of harm under the existing [exact match] system?”, page 22-23. Notes: 1. Section 5, Survey of Potential Registrants – see: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN…. -- We should make sure that everyone who is in a category should be asked the same questions. -- Make sure we are sending people down the right path. How the question should be ask should be asked should be for the survey provider to weigh in on. If someone received an objection then we ask them what they did -- this is the key question. Important to get as much information as we can, with tick-box options. Did they think it wasn't worth the time, were they unsure of their legal rights? -- Overarching question: How are we going to find registrants generally who registered, or those who got a notice? How to identify that subset? -- Two ways: Try and see if someone in the registry/registrar space would be willing to send out a survey -- category 2 (tried but weren't able with respect to at least one TLD); for category 3 -- there are significant survey panels for people doing internet things -- ask Survey Monkey to implement the survey or other large internet survey panel. -- Does Survey Monkey rely on self-qualification of respondents? Important to recognize in category 3 we are talking about potential consumers -- ask them, is this the type of thing you would buy? -- In the RFP we also specified that the vendor should propose outreach strategy for reaching target audience, especially registrants / potential registrants who are not necessarily involved in ICANN community. Show notice: [Explain in your own words what you understand the notice to mean] [Ask more comprehension questions: you may or may not have rights…] -- Ask them to explain in their own words, but the survey provider might think that is too much. We want the least guided response. -- Showing the notice and getting feedback -- only suggestion is to switch the bracketed suggestions for the provider. We want to ask comprehension questions. Ideally we want them to say it in their own words, but worried we might not get the responses we need, prefer have tick boxes. Given the guidance first and then the suggestion for how after. What would you do if you received a notice with the following wording: continue with the registration not continue with the registration consult someone else [who] something else [explain] [Consider some cells using examples: e.g., xerox.careers, apple.farms, chipotles.sucks—what would they do if they received notices?] -- Engaging in hypotheticals can have value, but could be a slippery slope. But, we are kind of asking people about the notice in a vacuum -- we don't know what they are trying to register. -- chipotles.sucks -- could register as a criticism site; could work up some scenarios -- the untility of asking specifics is to have respondents apply the notice to a specific example. -- xerox.careers for example -- hypothetically someone might be in the recruitment business or as a way to scam people -- not sure how we address that. -- Nervous about the potential to misread the hypothetical data. -- Danger that the respondent may not know the brand. -- Whether a brand cares about being used in the survey it doesn't matter for our data gathering; but should use global well-known brands as examples. -- Or, perhaps we could set out a "fake" company - saying it could be a competitor of major brand X? -- Need to try and decide WHAT we are trying to solve for here. -- Are there other ways to get at the same information without using hypotheticals? -- We don't know whether the notice can differentiate between false positive and true positives. We have zero empirical evidence. Could also make up a famous brand. -- Good faith/bad faith is not as much a dichotomy as a spectrum. Suggest that ask if people being part of a focus group -- if you are interested please offramp here. But then funding becomes an issue. -- The notice is one thing. Do you understand the notice? What is this notice say to you? etc...All of these questions about comprehension can be answered without any hypo. The hypos require a lot of nuance that I think the average citizen is unaware of. -- Concerned with hypotheticals is because they could be quite nuanced. You could ask objective questions about the notice without the hypothetical. -- Don't think our mandate is limited to whether the notice is terrifying people or not, but also can we improve it. Need to know more about what it is doing. -- One way to get at without the hypo: After reading this notice, would you think about what you intended to do with the domain name before you decided if you wanted to proceed? That goes to: what is the potential registrant thinking about rather than us guessing about what they're thinking about by looking at the the names they selected as "ok to proceed" in hypothetical. -- Multiple questions are to avoid non-leading questions, that there are several different ways to ask the question, but we should ask the survey provider. If you have registered a domain name, have you received any other kind of objection to your choice of a domain name? If yes, what was it? URS, UDRP, letter from a lawyer, lawsuit, don’t know/not sure, something else [fill in w/ survey expert consultation] Did you keep the domain name after the objection? [Yes/no/explain] Have you received such a notice on more than one occasion? -- Move that up front, but for those people who have registered -- categories 2 and 3. Precedes I am going to show you the claims notice, then go to the claims notice. -- There are overlapping budgets so we should consider the questions as potentially overlapping. 2. RFP Status: --- Responses from 6 vendors. -- 12 February is the deadline for vendors to express interest. -- That will be followed by a period during which the vendors can ask questions. -- The RFP will close on 09 March. See the timeline: https://www.icann.org/resources/pages/governance/rfps-en

1 0

Google Doc Closes in 12 Hours & Suggested Agenda
by Ariel Liang Feb. 9, 2018

Feb. 9, 2018

Hello everyone, Staff have incorporated comments and input discussed in the last Sub Team meeting on the Google Doc. Please reference the redlines and comments made on page 26-28: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN…<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_docume…>. We will close “commenting/suggesting” on the Data Request Google doc at around 05:00 UTC on Friday, 09 February. We will also distribute a PDF document that records the redlines and comments made on the Google Doc for reference. Since Kurt won’t make it to the Friday Sub Team call, as he suggested, perhaps we could commence discussion on Section 5 (Potential Registrant), which is led by Rebecca? Rebecca, kindly let us know whether this would work for you. 1. Commence discussion on Section 5 (Potential Registrants) 2. Timing of future meetings 3. AOB As mentioned in the previous meeting, to meet the RFP timeline of sharing the updated Data Table with participating vendors, it would be ideal if the Sub Team could finalize or be close to finalize this table by 19 February (two more Sub Team meetings, including tomorrow’s meeting). Thank you! Mary, Julie, Berry, and Ariel

2 2

Reminder: Google Doc Closes in 12 Hours
by Ariel Liang Feb. 8, 2018

Feb. 8, 2018

Hello everyone, As per action item for staff, we have incorporated comments and input discussed in the last Sub Team meeting on the Google Doc. Please reference the redlines and comments made on page 14-15: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN…. As before, staff only incorporate edits (with redlines/track changes) and comments provided by the Sub Team in the sections that the group has already discussed. As a friendly reminder, staff will close “commenting/suggesting” on the Data Request Google doc at around 05:00 UTC on Friday, 02 February. We will also distribute a PDF document that records the redlines and comments made on the Google Doc for reference. Last, please review the proposed draft agenda for tomorrow’s call: 1. Commence discussion on Section 4 (Registrants) 2. Timing of future meetings 3. AOB Thank you! Mary, Julie, Berry, and Ariel

7 7

Action Items from 02 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 2, 2018

Feb. 2, 2018

All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (02 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Note also that the next call will be on Friday, 09 February at 1700 UTC. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Section 4, Survey of Registrants: Staff will update the Google document based on the discussion on the call on 02 February. Section 3, Survey of TM and Brand Owners: Sub Team members should review and comment on the list concerning the final anecdotal and data questions for the question “What is the evidence of harm under the existing [exact match] system?”, page 22-23 at https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN…. Notes: 1. Update on the RFP -- Goal is to complete the table by 19 February (staff recommendation). 2. Commence discussion on Section 4 (Registrants): Google Doc: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN… General Points (Add to “General Recommendations” section of the table: -- We are surveying registrants: experience with conducting surveys when surveying this type of audience we should keep the survey short and keep away from open-ended questions and try to use multiple choice questions. Decide whether to come up with the questions ourselves, or let the provider do it? Less sophisticated audience. Discussion might also inform the selection of the survey provider. -- Consider how we get our registrants -- one is engage the survey company; an alternative would be to survey Registrars' customers, though that might be tricky. Might to talk about the direction we want to give to the survey provider on how to reach registrants. -- Trying to keep costs down and a cost driver is open-ended questions since someone has to categorize them. Questions: Have you ever registered a domain name, if the answer is "no" then terminate the survey -- Deleted because a number of people have tried to register but couldn't (not a potential registrant). -- Could include: "Have you registered? If not, why?" If to the population in general they might ask "what is a domain name?" We want to get into to it for why they haven't registered, but we aren't at the point of a domain name survey. -- Or, does this fit into the potential registrant section, but don't want to count people twice. Maybe if you answer "yes" you keep going, if "no" then you go to the survey of potential registrants. -- Could put it in a note to the survey provider: those who have registered, those who have tried but failed for whatever reason, and those who are interested but haven't tried. Have you received any kind of objection to your choice of a domain name? If so what type? -- Will garner information we won't need and is likely to turn off the respondent. -- Leave a lot of leeway to the survey provider. -- Somehow signal that the best way to ascertain whether they received an official trademark claims notice is to have it direct to a standard notice and ask if that is what they received. Point to "Official Trademark Claims Notice". -- Scan down in the questions and note that we ask "if you received a notice that said..." Then add, "Yes, no, or not sure". or: "Which of the following did you receive: a) Trademark Claims Notice from Registrar, b) cease and desist letter from another party, c) other type of objection (if so, Explain ____)"? Cover the possibilities in the multiple choice questions. Ask the survey provider how we best funnel people. -- If we ask something really open-ended that could capture all kinds of steps in the process, such as a cease-and-desist letter. In this question that is what we are trying to elicit -- did you get a claims notice even if you didn't know what it was. Don't want to get people who got cease-and-desist letters. -- Undecided if we need to ask this question, and ask the survey provider which they think is the best approach. When you registered names in any of the new top-level domains, did you receive a notice that stated... -- Then if "no" terminate the survey? If yes, then go to the comprehensive questions. Or if no, continue to get hypothetical information -- if they got a notice, would they understand it. If you receive a notice that looked like this, what would you do about it. If you abandoned you registration why did you -- skip ahead to other questions. These people that have registered, they have or haven't abandoned, two paths: haven't received a claims notice but we want to ask how they would react if the receive on, 2) abandonment that don't apply to trademark claims but apply to other parts of the survey. Once we have someone in the survey we want data about claims notice but behaviors that might apply to other transactions. Capture all the experience -- word it broadly enough to get a good collective experience. -- Gateway Question #1: Have you applied to register a New gTLD Domain Name? If No: Go to Potential Registrants no Application Qs -- If Yes: Go to Gateway Question #2: Have you registered a New gTLD Domain Name? If No: Go to Potential Registrants but No Registration Qs -- If Yes: Go to Registrants Qs. -- Charter questions ask about Trademark Claims, but also general abandonment. When we have a registrant and we ask them if they registered, we also ask if they have registrations that you weren't granted take them through the potential registrant questions. -- Flow from Section 4 into Section 5. -- This part of the survey is where you succeeded, and this part is about where you didn't. -- Suggestion: Do continue with these questions as they are still valid, make sure the questions are clear. Let the survey provider decide where we off ramp or on ramp -- or not "off ramp" but being directed to Qus that apply. These next questions should be asked of people who said they did receive an objection... -- Very open ended -- first try to capture information in multiple-choice questions and then proceed to open-ended questions. -- Or, start open-ended and avoid leading people to answer in a certain way. Given them a chance to say what they would in their own words and then given them multiple choice. Include -- "did you proceed or not" as an off ramp. -- Or, have multiple choice with "other" included. Also, with open-ended it might be hard to categorize the responses. -- Or, provide "other" with explanation, but also specific multiple choice so that metrics can be obtained. -- Give them a chance to say what actually happened.

1 0

Recordings, attendance & AC chat from Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call Friday, 02 February 2018
by Julie Bisland Feb. 2, 2018

Feb. 2, 2018

Dear All, Please find the attendance and AC Chat of the call attached to this email, and the MP3 and Adobe Connect recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 02 February 2018 at 17:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/iQm8B MP3: https://audio.icann.org/gnso/gnso-rpm-review-02feb18-en.mp3 Adobe Connect recording: https://participate.icann.org/p7e7he5nz1r/<https://participate.icann.org/p7e7he5nz1r/?OWASP_CSRFTOKEN=d1dce525fc2ff5e0…> The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB Thank you. Kind regards, Julie

1 0