February 2018 - Gnso-rpm-data

Re: [Gnso-rpm-data] Rationales for/against use of hypotheticals
by Ariel Liang Feb. 28, 2018

Feb. 28, 2018

Hello Kristine, Thank you and Rebecca for submitting the text! If no objection from the Sub Team, I will append this text on the Data Table (on a separate page, right after the suggested hypothetical question), and circulate the updated Data Table with the full WG. The interactive phase in the RFP was closed yesterday (ideally we should seize all communications with the vendor until 9 March), but I will ask Procurement whether they could forward this comment individually to the vendors. All – if you have any comment/objection to this text, please be so kind to share with the group, so that we could forward the agreed text to the vendors asap. Thank you, Ariel From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Dorrain, Kristine via Gnso-rpm-data" <gnso-rpm-data(a)icann.org> Reply-To: "Dorrain, Kristine" <dorraink(a)amazon.com> Date: Tuesday, February 27, 2018 at 5:41 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Rationales for/against use of hypotheticals Hi team, Rebecca (timely) submitted her text and I have (belatedly) attached mine (thanks for your valuable contributions, Susan). Please see the language below. If the group does not object, I will ask staff to append this to the email with the data table that was circulated to the broader group. Julie/Ariel, I’m not sure what the best method is for making sure everyone has had a chance to see this. Best, Kristine ******************** To the RPMs WG: Our subteam largely achieved consensus on the data questions as submitted to you. Our primary disagreement was in determining if we should include hypotheticals in the survey in the “questions for potential registrants” section. The full WG will need to determine whether to include hypothetical questions in the data survey. We have prepared two rationales: one in support of hypotheticals, and one opposed; both are provided below. We hope you will consider our viewpoints when deciding. Thanks, RPMs Data Subteam ************************* Rationale SUPPORTING use of hypotheticals: Although asking respondents to interpret the notice in the abstract will provide some information about how they understand it, they will still be answering an abstract question. When we ask what they would do in the abstract if they received such a notice when they were trying to register a domain name, we don’t know what kind of domain name they imagine themselves trying to register. To deal with that variance, concrete scenarios can provide additional information. In particular, a key question we want answers to is whether the current notice produces overdeterrence. Would a person who should win a URS/UDRP if one were actually brought understand that they have the right to proceed with a registration? Secondarily: would a person who should lose understand that a rights holder is going to be notified? We don’t need to ask about tough cases to get better information about that; we can ask about clear cases, including providing information about the intended use as well as the domain name, e.g., assuming there’s a famous brand of Horse Computers, horse.farm for a farm offering horse rides and horse.computers for offering a competing computer brand. (Such questions should rotate among respondents and otherwise follow survey best practices to avoid leading questions.) Rationale OPPOSING use of hypotheticals: Hypotheticals require the WG to make too many assumptions. Let’s take the example of apple.farm and apple.computer (for illustrative purposes only). We cannot assume that the registrant of apple.farm is not intending to register and use the domain name in bad faith, just like we cannot assume the registrant of apple.computer is intending to register and use the domain name in bad faith. If we say “when presented by this notice [display notice], would you feel confident registering a) apple.farm , b) apple.computer, c) both, d) neither?” we learn nothing. Because our interpretation of what this means relies our own qualitative opinion that the survey taker is not privy to. The notice should deter someone with bad faith intent; but we cannot know or determine any survey taker’s actual intent from their response to this type of question. On the other hand, we can get information about what they’re thinking by asking the following user stories that go to state of mind: “If you were presented with this notice [display notice] when registering a domain name, would you a) proceed to register, b) stop trying to register the name, c) stop for now and maybe come back after you’ve done some research, or d) other [explain]?” What if you were: 1. Getting a domain name for a business you’d already started? [Proceed/stop/research/other] 1. Where that business is in the same field of activity as the brand that was the subject of the notice [because even if you plan to run a business it doesn’t assume that you’ve chosen an appropriate name] 2. Where that business is in an unrelated field 1. Getting a domain name for a business you were thinking about starting? [Proceed/stop/research/other] * Where that business will be in the same field of activity as the brand that was the subject of the notice * Where that business will be in an unrelated field 1. Getting a domain name for fun (family use, email address, vanity) [Proceed/stop/research/other] 2. Getting a domain name as an investment (to maybe sell or use later)? [Proceed/stop/research/other] What concerns do you have about your next steps? [free form space]

1 1

Rationales for/against use of hypotheticals
by Dorrain, Kristine Feb. 27, 2018

Feb. 27, 2018

Hi team, Rebecca (timely) submitted her text and I have (belatedly) attached mine (thanks for your valuable contributions, Susan). Please see the language below. If the group does not object, I will ask staff to append this to the email with the data table that was circulated to the broader group. Julie/Ariel, I'm not sure what the best method is for making sure everyone has had a chance to see this. Best, Kristine ******************** To the RPMs WG: Our subteam largely achieved consensus on the data questions as submitted to you. Our primary disagreement was in determining if we should include hypotheticals in the survey in the "questions for potential registrants" section. The full WG will need to determine whether to include hypothetical questions in the data survey. We have prepared two rationales: one in support of hypotheticals, and one opposed; both are provided below. We hope you will consider our viewpoints when deciding. Thanks, RPMs Data Subteam ************************* Rationale SUPPORTING use of hypotheticals: Although asking respondents to interpret the notice in the abstract will provide some information about how they understand it, they will still be answering an abstract question. When we ask what they would do in the abstract if they received such a notice when they were trying to register a domain name, we don't know what kind of domain name they imagine themselves trying to register. To deal with that variance, concrete scenarios can provide additional information. In particular, a key question we want answers to is whether the current notice produces overdeterrence. Would a person who should win a URS/UDRP if one were actually brought understand that they have the right to proceed with a registration? Secondarily: would a person who should lose understand that a rights holder is going to be notified? We don't need to ask about tough cases to get better information about that; we can ask about clear cases, including providing information about the intended use as well as the domain name, e.g., assuming there's a famous brand of Horse Computers, horse.farm for a farm offering horse rides and horse.computers for offering a competing computer brand. (Such questions should rotate among respondents and otherwise follow survey best practices to avoid leading questions.) Rationale OPPOSING use of hypotheticals: Hypotheticals require the WG to make too many assumptions. Let's take the example of apple.farm and apple.computer (for illustrative purposes only). We cannot assume that the registrant of apple.farm is not intending to register and use the domain name in bad faith, just like we cannot assume the registrant of apple.computer is intending to register and use the domain name in bad faith. If we say "when presented by this notice [display notice], would you feel confident registering a) apple.farm , b) apple.computer, c) both, d) neither?" we learn nothing. Because our interpretation of what this means relies our own qualitative opinion that the survey taker is not privy to. The notice should deter someone with bad faith intent; but we cannot know or determine any survey taker's actual intent from their response to this type of question. On the other hand, we can get information about what they're thinking by asking the following user stories that go to state of mind: "If you were presented with this notice [display notice] when registering a domain name, would you a) proceed to register, b) stop trying to register the name, c) stop for now and maybe come back after you've done some research, or d) other [explain]?" What if you were: a) Getting a domain name for a business you'd already started? [Proceed/stop/research/other] i. Where that business is in the same field of activity as the brand that was the subject of the notice [because even if you plan to run a business it doesn't assume that you've chosen an appropriate name] ii. Where that business is in an unrelated field b) Getting a domain name for a business you were thinking about starting? [Proceed/stop/research/other] i. Where that business will be in the same field of activity as the brand that was the subject of the notice ii. Where that business will be in an unrelated field c) Getting a domain name for fun (family use, email address, vanity) [Proceed/stop/research/other] d) Getting a domain name as an investment (to maybe sell or use later)? [Proceed/stop/research/other] What concerns do you have about your next steps? [free form space]

1 0

Re: [Gnso-rpm-data] [Ext] Re: Inquiry re: Next Meeting per Action Items from 23 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 26, 2018

Feb. 26, 2018

Dear All, Many thanks for all who have responded to the inquiry. Several Sub Team members have indicated support for a meeting this Friday, 02 March, at 1700 UTC so we’ll go ahead and get that on the schedule. With respect to a meeting at ICANN61, we do still have the possibility of using one of the four sessions as a working session. However, please note that we will not be able to change the room setup, which is for a U-shape plus additional seating. We can change the description of the meeting to emphasize that it is a working session. With respect to holding a meeting outside of one of the public session slots, there are a few things we’ll need to keep in mind. First, we would need to book a “pop-up” room and these will not be available to schedule until Friday, 09 March. Second, pop-up rooms do not have remote access, and any recordings would have to be done via the Adobe Connect room. So, even if only a small number of Sub Team members are meeting, if the plan is to make it a working session it should still be a formal meeting whereby others in the full Working Group would be able to know what was discussed, and notes and recordings should be posted on the wiki. Perhaps the Sub Team members may wish to decide whether a meeting at ICANN61 is needed based on the results of a call this Friday. Thanks again, Julie From: Susan Payne <susan.payne(a)valideus.com> Date: Monday, February 26, 2018 at 6:29 AM To: Kurt Pritz <kurt(a)kjpritz.com> Cc: Julie Hedlund <julie.hedlund(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] Re: [Gnso-rpm-data] Inquiry re: Next Meeting per Action Items from 23 February 2018 RPM Data Sub Team Call I can meet Friday, and happy to try a pirate meeting if one of the allocated RPM slots isn't suitable. Susan Payne Head of Legal Policy | Valideus Ltd 28-30 Little Russell Street London, WC1A 2HN, United Kingdom E: susan.payne(a)valideus.com D: +44 20 7421 8255 T: +44 20 7421 8299 M: +44 7971 661175 Sent from my iPhone On 26 Feb 2018, at 00:12, Kurt Pritz <kurt(a)kjpritz.com> wrote: Please see below. I am for Friday, but we should still meet at ICANN. At the end of the last session we discussed this and I (not jokingly) suggested “pirate meeting." If you take out the participation of the working group chairs there are only 5 or 6 of us. Why cannot we meet over lunch or in the bar at sometime. I suggest we meet in the restaurant or bar for lunch coffee or drinks and go through a pre-developed agenda. We don”t have the need for recording etc, just an informal venue where we might be effective. What do you think? Kurt On Feb 23, 2018, at 1:06 PM, Julie Hedlund <julie.hedlund(a)icann.org> wrote: All, Per this action item: Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61, please indicate below whether you wish to hold a meeting next Friday, 02 March, at the usual time of 1700 UTC and/or a working session at ICANN61. Note that separately you will receive a poll asking about your attendance for the meetings during ICANN61. The topic of the meeting would be the continued discussion of the additional RPMs. __X__ Meet next Friday, 02 March at 1700 UTC ____ Meet at ICANN61 using one of the 4 public sessions (the session description would specifically note that this is a working session) Thank you for your assistance. Kind regards, Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of Julie Hedlund <julie.hedlund(a)icann.org> Date: Friday, February 23, 2018 at 1:38 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Action Items from 23 February 2018 RPM Data Sub Team Call All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (23 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Data Table: Michael to email the group his suggestions for deleting duplication on page 26, and make the changes to the Google Doc (DONE). Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. Additional RPMs: 1) Add questions 4 and 5 to the Data Table if they are not already covered. 2) Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. Staff will update the Data Table based on the above input and the actions from the discussion during the meeting, send the Data Table to the full Working Group, and send it to ICANN Procurement staff. Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61 Notes: 1. Discussion of the remaining part of the Data Table -- Review of the Section 4 (Registrant Questions) See: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN… If you attempted to register a domain name, did you receive an objection based on trademark to your choice of domain name?... Action: Fine to add, but delete from "based on..." and add "If you received an objection, what was the basis?" To identify people who may not know what the specific name of the process they triggered was..." Actions: Ask “Did you receive a warning or notice of possible trademark conflict?” These are guidance for the survey provider to provide “cues” for the respondents. Discussion: -- Put this in the guidance for the survey provider. -- Should we keep this narrow, focusing on trademark claims and Sunrise? I think objection, to a registrant -- just say "did you receive this notice" and keep the exact words of the notice in. Unless we put this more vague question at the end. -- Assuming we are looking for who received a claims notice and what did they do. "objection" could mean all sorts of things. We want to find out did they receive a claims notice and what they did in response. -- Disagree. Want to know about objections. Asking about the notice in column three, but there could be a wide variety of objections. -- Think we have a disagreement on methodology. Don't think we should put the text of the notice to them, but get them thinking about the context. The surveyor can help finalize this. Did you receive this before you completed the registration and we can ask what happened. If you asked if you received the notice then they might not be able to tell if the wording is the same. Let them think about it and their answers may reveal what they received. -- Agree with not using the word objection. Some people may not understand what an objection or trademark notice is. Ask them questions about the notice and find out if they misperceived the notice, or even knew if it was an objection. -- Page 26: Rather than "did you receive an objection" did you receive a warning notice. It is outside scope to start asking questions about post registration whether they got a cease and desist letter. I think we are trying to determine if they triggered a notice, did they understand it, what did they do. -- Be careful about using “objection”, “protest”, or “challenge” since that hasn't happened yet. The rights holder hasn't made the objection. -- Go to page 29 -- captures where we could come out. Three buckets of people. Ask all of the questions about the language of the claims notice. Realize that people don't just sit in one of these buckets. We aren't the survey operators, we are trying to explain what we are seeking. -- Didn't want to lose potential registrants who tried to register but couldn't get through. Did they receive a warning or notice of possible trademark conflict. -- Left with whether or not we want to ask about other RPMs. Sounds like keeping it to trademark and Sunrise. -- At some point we do need to ask if what they received was a Trademark Claims Notice or something else. -- Notes for survey provider: preparing the survey people for the different kinds of things they might want to elicit. Prepare the respondents. The option that this is the type of objection/notice I received, or other. Include "cues". Section 6: Actions: -- Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. -- Add questions 4 and 5 if they are not already covered. Discussion: -- Have done some thinking about it. -- Don't understand why we are doing a separate set of questions for public interest groups and trade associations. -- The thinking was that we wanted to capture as many groups as possible. One of the challenges in looking at the table, except for very large non-profits most would not be able to answer the questions. It would depend on who we reached out to. So, unless we reach out to very large non-profits (such as Red Cross, March of Dimes, YMCA, etc.) you aren't going to get answers to the questions. -- Problematic that we would be asking different questions of that group since we wouldn't be able to compare to the rest of the groups. -- Include a note to the survey provider that NGOs may need a bit more coaching and background. So they can help us identify who could be reached for the survey. Teese out a classification of users among the public interest groups. -- Another General Note to keep the inconsistency in the Claims Notices, Trademark Notification, NORNs. -- Redundancy in Section 3, page 21, third bullet point in 3rd column; 4th column, primary method. Michael to email the group about his suggestions and make the change to the Google Doc 2. Check the Additional Marketplace RPMs and see whether Qs fall within scope: Action: Add questions 4 and 5 to the Data Table -- Question 1 and 3 do not belong to the survey (out of scope) -- Question 4 * Could possibly ask the ROs, but not in the survey as it is not related Sunrise and Trademark Claims * These questions have impact on Sunrise and Trademark Claims. Two for ROs, one for registrar, and one for TM owners. Make sense to tag these onto the survey * GNSO Council did not give sign off to add these questions from additional marketplace RPMs. * Questions can only be added if it is specifically in scope for the RFP, fitting into the Sunrise and Trademark Claims question table. * ACTION: Question 4 to add to the TM owner section, related to Sunrise -- Question 5 * It is more germane * ACTION: Within scope, and can be added to the table, related to Trademark Claims, aim at ROs and Registrars -- No other questions in the additional RPMs need to be included in the survey? -- No more time for this survey to go through the additional RPMs thoroughly. In early stages of the PDP, questions were sent to ROs that ran these additional RPMs. Donuts and one/two others provided feedback. -- ACTION: Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. -- Timing of future meeting. ICANN61 Planning / repurposing a session for the Data Sub Team? -- Participation in ICANN61 * Susan, Kristin * Kurt (leaving Wed AM) * Michael (remote participation) _______________________________________________ Gnso-rpm-data mailing list Gnso-rpm-data(a)icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-data _______________________________________________ Gnso-rpm-data mailing list Gnso-rpm-data(a)icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-data

2 1

Inquiry re: Next Meeting per Action Items from 23 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 26, 2018

Feb. 26, 2018

All, Per this action item: Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61, please indicate below whether you wish to hold a meeting next Friday, 02 March, at the usual time of 1700 UTC and/or a working session at ICANN61. Note that separately you will receive a poll asking about your attendance for the meetings during ICANN61. The topic of the meeting would be the continued discussion of the additional RPMs. ____ Meet next Friday, 02 March at 1700 UTC ____ Meet at ICANN61 using one of the 4 public sessions (the session description would specifically note that this is a working session) Thank you for your assistance. Kind regards, Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of Julie Hedlund <julie.hedlund(a)icann.org> Date: Friday, February 23, 2018 at 1:38 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Action Items from 23 February 2018 RPM Data Sub Team Call All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (23 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Data Table: Michael to email the group his suggestions for deleting duplication on page 26, and make the changes to the Google Doc (DONE). Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. Additional RPMs: 1) Add questions 4 and 5 to the Data Table if they are not already covered. 2) Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. Staff will update the Data Table based on the above input and the actions from the discussion during the meeting, send the Data Table to the full Working Group, and send it to ICANN Procurement staff. Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61 Notes: 1. Discussion of the remaining part of the Data Table -- Review of the Section 4 (Registrant Questions) See: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN… If you attempted to register a domain name, did you receive an objection based on trademark to your choice of domain name?... Action: Fine to add, but delete from "based on..." and add "If you received an objection, what was the basis?" To identify people who may not know what the specific name of the process they triggered was..." Actions: Ask “Did you receive a warning or notice of possible trademark conflict?” These are guidance for the survey provider to provide “cues” for the respondents. Discussion: -- Put this in the guidance for the survey provider. -- Should we keep this narrow, focusing on trademark claims and Sunrise? I think objection, to a registrant -- just say "did you receive this notice" and keep the exact words of the notice in. Unless we put this more vague question at the end. -- Assuming we are looking for who received a claims notice and what did they do. "objection" could mean all sorts of things. We want to find out did they receive a claims notice and what they did in response. -- Disagree. Want to know about objections. Asking about the notice in column three, but there could be a wide variety of objections. -- Think we have a disagreement on methodology. Don't think we should put the text of the notice to them, but get them thinking about the context. The surveyor can help finalize this. Did you receive this before you completed the registration and we can ask what happened. If you asked if you received the notice then they might not be able to tell if the wording is the same. Let them think about it and their answers may reveal what they received. -- Agree with not using the word objection. Some people may not understand what an objection or trademark notice is. Ask them questions about the notice and find out if they misperceived the notice, or even knew if it was an objection. -- Page 26: Rather than "did you receive an objection" did you receive a warning notice. It is outside scope to start asking questions about post registration whether they got a cease and desist letter. I think we are trying to determine if they triggered a notice, did they understand it, what did they do. -- Be careful about using “objection”, “protest”, or “challenge” since that hasn't happened yet. The rights holder hasn't made the objection. -- Go to page 29 -- captures where we could come out. Three buckets of people. Ask all of the questions about the language of the claims notice. Realize that people don't just sit in one of these buckets. We aren't the survey operators, we are trying to explain what we are seeking. -- Didn't want to lose potential registrants who tried to register but couldn't get through. Did they receive a warning or notice of possible trademark conflict. -- Left with whether or not we want to ask about other RPMs. Sounds like keeping it to trademark and Sunrise. -- At some point we do need to ask if what they received was a Trademark Claims Notice or something else. -- Notes for survey provider: preparing the survey people for the different kinds of things they might want to elicit. Prepare the respondents. The option that this is the type of objection/notice I received, or other. Include "cues". Section 6: Actions: -- Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. -- Add questions 4 and 5 if they are not already covered. Discussion: -- Have done some thinking about it. -- Don't understand why we are doing a separate set of questions for public interest groups and trade associations. -- The thinking was that we wanted to capture as many groups as possible. One of the challenges in looking at the table, except for very large non-profits most would not be able to answer the questions. It would depend on who we reached out to. So, unless we reach out to very large non-profits (such as Red Cross, March of Dimes, YMCA, etc.) you aren't going to get answers to the questions. -- Problematic that we would be asking different questions of that group since we wouldn't be able to compare to the rest of the groups. -- Include a note to the survey provider that NGOs may need a bit more coaching and background. So they can help us identify who could be reached for the survey. Teese out a classification of users among the public interest groups. -- Another General Note to keep the inconsistency in the Claims Notices, Trademark Notification, NORNs. -- Redundancy in Section 3, page 21, third bullet point in 3rd column; 4th column, primary method. Michael to email the group about his suggestions and make the change to the Google Doc 2. Check the Additional Marketplace RPMs and see whether Qs fall within scope: Action: Add questions 4 and 5 to the Data Table -- Question 1 and 3 do not belong to the survey (out of scope) -- Question 4 * Could possibly ask the ROs, but not in the survey as it is not related Sunrise and Trademark Claims * These questions have impact on Sunrise and Trademark Claims. Two for ROs, one for registrar, and one for TM owners. Make sense to tag these onto the survey * GNSO Council did not give sign off to add these questions from additional marketplace RPMs. * Questions can only be added if it is specifically in scope for the RFP, fitting into the Sunrise and Trademark Claims question table. * ACTION: Question 4 to add to the TM owner section, related to Sunrise -- Question 5 * It is more germane * ACTION: Within scope, and can be added to the table, related to Trademark Claims, aim at ROs and Registrars -- No other questions in the additional RPMs need to be included in the survey? -- No more time for this survey to go through the additional RPMs thoroughly. In early stages of the PDP, questions were sent to ROs that ran these additional RPMs. Donuts and one/two others provided feedback. -- ACTION: Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. -- Timing of future meeting. ICANN61 Planning / repurposing a session for the Data Sub Team? -- Participation in ICANN61 * Susan, Kristin * Kurt (leaving Wed AM) * Michael (remote participation)

7 7

Re: [Gnso-rpm-data] Google Doc Closes in 12 Hours & Suggested Agenda
by Julie Hedlund Feb. 23, 2018

Feb. 23, 2018

Many thanks Michael. This is quite helpful. Kind regards, Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Friday, February 23, 2018 at 1:04 PM To: Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Google Doc Closes in 12 Hours & Suggested Agenda Further to my suggestions online, I attach page 21 from our chart of questions showing the two deletions I would like to make. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Thursday, February 22, 2018 9:02 PM To: gnso-rpm-data(a)icann.org Subject: Re: [Gnso-rpm-data] Google Doc Closes in 12 Hours & Suggested Agenda Hello everyone, Please note that the Google Doc has been changed to “view only”. Kindly find attached the PDF document that records the redlines/comments made by 05:00 UTC today on the Google Doc. At the time of the meeting, I will revert the Google Doc back to “comment” to facilitate the discussions. Thank you! Ariel From: Ariel Liang <ariel.liang(a)icann.org> Date: Thursday, February 22, 2018 at 11:45 AM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Google Doc Closes in 12 Hours & Suggested Agenda Hello everyone, Kindly note that staff have incorporated Sub Team’s suggestions and comments for Section 5 (brought back the deleted questions on page 34-35 and relocated the introductory text above these questions as a comment): https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN…. We will close “commenting/suggesting” on the Data Request Google doc at around 05:00 UTC on Friday, 23 February. We will also distribute a PDF document that records the redlines and comments made on the Google Doc for reference. As a friendly reminder (esp. for Kristine, Susan, and Rebecca), with respect to the hypothetical question on page 35-36, kindly provide your summary of rationale on why such question would be useful or not useful. Tomorrow’s meeting would be the last one before we forward the updated Data Table to the responding vendors. By EOB tomorrow, the updated Data Table will need to be shared with the full WG and sent to Procurement. The remaining part of the Data Table is Section 4 (page 28 et seq) and Section 6 (currently blank). We would like to get the Sub Team’s input on this – Should we: Try to tackle the remaining part and develop some agreed suggested questions? Incorporate existing input and comment in Section 4 as is, leave Section 6 blank, and still include the instruction text previously drafted by Kurt/Rebecca/Kathy for Section 4? Leave Section 4 and Section 6 blank, and include the instruction text previously drafted by Kurt/Rebecca/Kathy for Section 4? Furthermore, if the Data Sub Team intend to incorporate any survey questions related to the Additional Marketplace RPMs, tomorrow’s meeting is the last opportunity to provide any high-level topics to be incorporated in the Data Table. If we do so, they must be within scope of the survey project. Last, it would be very useful if the Sub Team could do a quick review of the Data Table from the beginning, making sure everything is in order before it is distributed. Staff would like to suggest the following agenda for tomorrow’s discussion: Discussion of the remaining part of the Data Table (Section 4, Section 6); whether to include any high-level topics related to Additional Marketplace RPMs Last review/sanity check of the Data Table Timing of future meetings / ICANN61 (whether to repurpose one session for the Data Sub Team) AOB Thank you for your time and contributions! Best Regards, Mary, Julie, Berry, and Ariel

4 3

Google Doc Closes in 12 Hours & Suggested Agenda
by Ariel Liang Feb. 23, 2018

Feb. 23, 2018

Hello everyone, Kindly note that staff have incorporated Sub Team’s suggestions and comments for Section 5 (brought back the deleted questions on page 34-35 and relocated the introductory text above these questions as a comment): https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN…. We will close “commenting/suggesting” on the Data Request Google doc at around 05:00 UTC on Friday, 23 February. We will also distribute a PDF document that records the redlines and comments made on the Google Doc for reference. As a friendly reminder (esp. for Kristine, Susan, and Rebecca), with respect to the hypothetical question on page 35-36, kindly provide your summary of rationale on why such question would be useful or not useful. Tomorrow’s meeting would be the last one before we forward the updated Data Table to the responding vendors. By EOB tomorrow, the updated Data Table will need to be shared with the full WG and sent to Procurement. The remaining part of the Data Table is Section 4 (page 28 et seq) and Section 6 (currently blank). We would like to get the Sub Team’s input on this – Should we: 1. Try to tackle the remaining part and develop some agreed suggested questions? 2. Incorporate existing input and comment in Section 4 as is, leave Section 6 blank, and still include the instruction text previously drafted by Kurt/Rebecca/Kathy for Section 4? 3. Leave Section 4 and Section 6 blank, and include the instruction text previously drafted by Kurt/Rebecca/Kathy for Section 4? Furthermore, if the Data Sub Team intend to incorporate any survey questions related to the Additional Marketplace RPMs, tomorrow’s meeting is the last opportunity to provide any high-level topics to be incorporated in the Data Table. If we do so, they must be within scope of the survey project. Last, it would be very useful if the Sub Team could do a quick review of the Data Table from the beginning, making sure everything is in order before it is distributed. Staff would like to suggest the following agenda for tomorrow’s discussion: 1. Discussion of the remaining part of the Data Table (Section 4, Section 6); whether to include any high-level topics related to Additional Marketplace RPMs 2. Last review/sanity check of the Data Table 3. Timing of future meetings / ICANN61 (whether to repurpose one session for the Data Sub Team) 4. AOB Thank you for your time and contributions! Best Regards, Mary, Julie, Berry, and Ariel

6 6

Recordings, attendance & AC chat for Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 23 February 2018
by Terri Agnew Feb. 23, 2018

Feb. 23, 2018

Dear All, Please find the attendance and AC Chat of the call attached to this email. The MP3 and Adobe Connect recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 23 February 2018 at 17:00 UTC. Attendance of the call is posted on the agenda wiki page: https://community.icann.org/x/mRa8B MP3: https://audio.icann.org/gnso/gnso-rpm-review-23feb18-en.mp3 Adobe Connect recording: <https://participate.icann.org/p2ycaw9k7mi/?OWASP_CSRFTOKEN=1606aaffffef62ce 66f816ddb69d4c60c61173e4ac6e610e3fb9c1fa3d197b3a> https://participate.icann.org/p2ycaw9k7mi/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar ** Please let me know if your name has been left off the list ** Mailing list archives: <http://mm.icann.org/pipermail/gnso-rpm-data/> http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the sub team: https://community.icann.org/x/eI1EB Thank you. With kind regards, Terri

1 0

Action Items from 23 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 23, 2018

Feb. 23, 2018

All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (23 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Data Table: Michael to email the group his suggestions for deleting duplication on page 26, and make the changes to the Google Doc (DONE). Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. Additional RPMs: 1) Add questions 4 and 5 to the Data Table if they are not already covered. 2) Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. Staff will update the Data Table based on the above input and the actions from the discussion during the meeting, send the Data Table to the full Working Group, and send it to ICANN Procurement staff. Staff to send a note to the list asking whether to hold a meeting next Friday, or arranging a meeting in ICANN61 Notes: 1. Discussion of the remaining part of the Data Table -- Review of the Section 4 (Registrant Questions) See: https://docs.google.com/document/d/1EUXC03ccuYhRMa_X4hDCPrq88KkF6qBRkL6sCcN… If you attempted to register a domain name, did you receive an objection based on trademark to your choice of domain name?... Action: Fine to add, but delete from "based on..." and add "If you received an objection, what was the basis?" To identify people who may not know what the specific name of the process they triggered was..." Actions: Ask “Did you receive a warning or notice of possible trademark conflict?” These are guidance for the survey provider to provide “cues” for the respondents. Discussion: -- Put this in the guidance for the survey provider. -- Should we keep this narrow, focusing on trademark claims and Sunrise? I think objection, to a registrant -- just say "did you receive this notice" and keep the exact words of the notice in. Unless we put this more vague question at the end. -- Assuming we are looking for who received a claims notice and what did they do. "objection" could mean all sorts of things. We want to find out did they receive a claims notice and what they did in response. -- Disagree. Want to know about objections. Asking about the notice in column three, but there could be a wide variety of objections. -- Think we have a disagreement on methodology. Don't think we should put the text of the notice to them, but get them thinking about the context. The surveyor can help finalize this. Did you receive this before you completed the registration and we can ask what happened. If you asked if you received the notice then they might not be able to tell if the wording is the same. Let them think about it and their answers may reveal what they received. -- Agree with not using the word objection. Some people may not understand what an objection or trademark notice is. Ask them questions about the notice and find out if they misperceived the notice, or even knew if it was an objection. -- Page 26: Rather than "did you receive an objection" did you receive a warning notice. It is outside scope to start asking questions about post registration whether they got a cease and desist letter. I think we are trying to determine if they triggered a notice, did they understand it, what did they do. -- Be careful about using “objection”, “protest”, or “challenge” since that hasn't happened yet. The rights holder hasn't made the objection. -- Go to page 29 -- captures where we could come out. Three buckets of people. Ask all of the questions about the language of the claims notice. Realize that people don't just sit in one of these buckets. We aren't the survey operators, we are trying to explain what we are seeking. -- Didn't want to lose potential registrants who tried to register but couldn't get through. Did they receive a warning or notice of possible trademark conflict. -- Left with whether or not we want to ask about other RPMs. Sounds like keeping it to trademark and Sunrise. -- At some point we do need to ask if what they received was a Trademark Claims Notice or something else. -- Notes for survey provider: preparing the survey people for the different kinds of things they might want to elicit. Prepare the respondents. The option that this is the type of objection/notice I received, or other. Include "cues". Section 6: Actions: -- Lori/Kathy to draft a note regarding the public interest group section to be included in the data table. -- Add questions 4 and 5 if they are not already covered. Discussion: -- Have done some thinking about it. -- Don't understand why we are doing a separate set of questions for public interest groups and trade associations. -- The thinking was that we wanted to capture as many groups as possible. One of the challenges in looking at the table, except for very large non-profits most would not be able to answer the questions. It would depend on who we reached out to. So, unless we reach out to very large non-profits (such as Red Cross, March of Dimes, YMCA, etc.) you aren't going to get answers to the questions. -- Problematic that we would be asking different questions of that group since we wouldn't be able to compare to the rest of the groups. -- Include a note to the survey provider that NGOs may need a bit more coaching and background. So they can help us identify who could be reached for the survey. Teese out a classification of users among the public interest groups. -- Another General Note to keep the inconsistency in the Claims Notices, Trademark Notification, NORNs. -- Redundancy in Section 3, page 21, third bullet point in 3rd column; 4th column, primary method. Michael to email the group about his suggestions and make the change to the Google Doc 2. Check the Additional Marketplace RPMs and see whether Qs fall within scope: Action: Add questions 4 and 5 to the Data Table -- Question 1 and 3 do not belong to the survey (out of scope) -- Question 4 * Could possibly ask the ROs, but not in the survey as it is not related Sunrise and Trademark Claims * These questions have impact on Sunrise and Trademark Claims. Two for ROs, one for registrar, and one for TM owners. Make sense to tag these onto the survey * GNSO Council did not give sign off to add these questions from additional marketplace RPMs. * Questions can only be added if it is specifically in scope for the RFP, fitting into the Sunrise and Trademark Claims question table. * ACTION: Question 4 to add to the TM owner section, related to Sunrise -- Question 5 * It is more germane * ACTION: Within scope, and can be added to the table, related to Trademark Claims, aim at ROs and Registrars -- No other questions in the additional RPMs need to be included in the survey? -- No more time for this survey to go through the additional RPMs thoroughly. In early stages of the PDP, questions were sent to ROs that ran these additional RPMs. Donuts and one/two others provided feedback. -- ACTION: Staff to color code the additional RPMs document and indicate what questions went onto the survey table, what did not. -- Timing of future meeting. ICANN61 Planning / repurposing a session for the Data Sub Team? -- Participation in ICANN61 * Susan, Kristin * Kurt (leaving Wed AM) * Michael (remote participation)

1 0

Additional Marketplace RPMs - Survey Incorporation
by Ariel Liang Feb. 16, 2018

Feb. 16, 2018

Hello everyone, Following up on today’s call, staff would like to provide some additional clarification regarding the potential incorporation of survey questions related to the Additional Marketplace RPMs offered by some registry operators on a voluntary basis. Since the scope of work stated in the RFP<https://www.icann.org/en/system/files/files/rfp-rpm-survey-29jan18-en.pdf#p…> is to assess the efficacy of Sunrise and Trademark Claims (specifically, we stated that the Data Table should be considered as the framework for scoping the project), any potential survey question about the Additional Marketplace RPMs should not expand the scope of the project. Change of the scope would have timeline and budget impact to the RFP, as well as the PDP in general. If the Sub Team could identify questions that won’t expand the scope of the project, it would be doable for vendors to submit proposals taking into account the data collection needs for Additional Marketplace RPMs. Requirements related to the Additional Marketplace RPMs need to be incorporated ASAP in the RFP documents, no later than 23 February when we pass the updated Data Table to the Procurement Team. Due to the time limitation, the Sub Team may not be able to develop detailed guidance or propose suggested survey questions for the Additional Marketplace RPMs. Staff would recommend the Sub Team to quickly run through these Additional Marketplace RPMs, identify high-level topics for data collection (must be within the scope), and include this guidance in the Data Table. If necessary, the Sub Team can organize an extended call on Friday, 23 February (90 minutes or so) to allow more time going through the additional RPMs, as well as reviewing the Data Table one last time. Please let us know if a longer call is needed and we will update the calendar invitation. Thank you all, and have a great weekend! Best Regards, Mary, Julie, Berry, and Ariel

1 0

Action Items from 16 February 2018 RPM Data Sub Team Call
by Julie Hedlund Feb. 16, 2018

Feb. 16, 2018

All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (16 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Note also that the next call will be on Friday, 23 February at 1700 UTC. Best Regards, Julie Julie Hedlund, Policy Director Action Items: Staff will update Section 5. Survey of Potential Registrants beginning on page 33 in the Google Doc based on the discussion during today’s call. With respect to hypotheticals (question on 35-36 in the Google Doc) Sub Team members should volunteer to produce two documents and circulate them on the list prior to the 23 February call: Short summary of what they hope to get from the use of hypotheticals; Short summary of why not to use a hypothetical. Kristine has volunteered for the second item. RPM Data Sub Team meeting at ICANN61: The RPM Co-Chairs will discuss whether one of the slots could be repurposed for the RPM Data Sub Team. Section 3, Survey of TM and Brand Owners: Sub Team members should review and comment on the list concerning the final anecdotal and data questions for the question “What is the evidence of harm under the existing [exact match] system?”, page 22-23. Notes: 5. Survey of Potential Registrants. Page 33 -- Inserted note to the survey providers: Language is helpful and clear. Page 34 -- Have you attempted to register a domain name in one of these new domains? [If no to both, terminate the survey] -- Thought we weren't going to suggest terminating the survey. But if they aren't considering registering and haven't tried to register then they aren't a potential registrant. Or do we still want them to look at the claims notice? If they continue with the survey, it might be good to find if the process seems confusing. -- Leave the language as is, and also give the survey provider latitude to change it. Page 34-35 -- The wording of this set of questions should be discussed with the surveyor... [etc.] -- Gather information multiple different ways. -- Question: In other cases we've been more specific; have we given enough direction here? -- Thought we were leaving in all of the variations of the questions? Examples -- deleted in the Google doc: "If, when registering your domain, you received a notice with the following wording, would you believe you were: legally allowed to continue with the registration, not legally allowed to continue, not be sure if you were legally allowed to continue?" etc. -- Bring back the deleted questions and include them in the note: The wording of this set of questions should be discussed with the surveyor. We would like to present the language of the notice to people in all relevant groups (1-3) and ask comprehension questions. The questions should be as neutral and non-leading as possible. This may involve beginning with “explain in your own words” and then asking more directed questions about perceptions of legal rights and risks, as well as “what would you do next if you received the notice when you attempted to register?” (e.g., stop, continue, consult someone else, etc.) and “why or why not?” – Anecdotal question. Other sample questions: If, when registering your domain, you received a notice with the following wording, would you believe you were: a. legally allowed to continue with the registration b. not legally allowed to continue b. not be sure if you were legally allowed to continue? If, when registering your domain, you received a notice with the following wording, would you believe you would: a. definitely get sued if you continued b. might or might not get sued c. definitely would not get sued if you continued? If, when registering your domain, you received a notice with the following wording, would you believe that: a. you had a legal right to continue with the registration? b. you might or might not have a legal right to continue with the registration? c. you had no legal right to continue with the registration? What would you do if you received a notice with the following wording: a. continue with the registration b. not continue with the registration c. consult someone else [who] something else [explain] Page 35-36: Consider offering Hypothetical: Famous maker of computers, Horse, and scenarios at extremes such as horse.computers and horse.farm -- if consumer attempting to register these received notice, what would they do? -- Did not have consensus on this. Guidance on what to present to the full WG. -- Trying to get to a general understanding of what is trademark law, but not sure offering hypotheticals is the best way to do this. Perhaps in a more direct fashion? -- Is that the purpose of the Hypothetical? Not automatically opposed to hypotheticals, but need to understand what info we're trying to get at & whether there's a less subjective way to get that info. -- The idea is to try to figure out if people would be scared away from non-infringing uses of TM terms. -- Could ask the survey provider if this helps. -- It actually sounds like we don't all have the same understanding of what info the hypotheticals are trying to elicit. -- But the problem is we can never know what intent is and even if someone says I think it's ok to get "apple.farm" if their intent is bad faith (which we would not know). -- There is no right answer. -- How useful is the data? Could also get adverse inferences. Not in agreement on what we would get by using a hypothetical. Okay if we are trying to elicite specific information that is actionable. -- Anything that helps us to make the claims notice is helpful. ACTION: Leave this suggestion (the hypothetical) and ask for a volunteer to make a short summary of what they hope to get from the hypothetical (Rebecca?). Kristine would be willing to write why not to use a hypothetical. Susan has offered to help. 2. Timing of future meetings & ICANN61 -- Try to knock the Additional Marketplace RPMs on the head as quickly as possible. -- Have a big session in PR to get it done. Make the room set up as a working session. Make it clear that people shouldn't be there if they aren't going to be working. Not a presentation of findings. Make sure we have remote participation. -- Could be difficult to schedule an additional meeting given conflicts, and “pop-up” rooms do not have remote access. -- The RPM WG has 4 session slots for a total of 6 hours; the Co-Chairs will discuss whether one of the slots could be repurposed for the RPM Data Sub Team.

3 3