- Gnso-rpm-data - lists.icann.org

REMINDER: Proposed Agenda and Documents: Meeting 20 June at 1730 UTC
by Julie Hedlund June 19, 2018

June 19, 2018

Dear Data Sub Team Members, In preparation for Wednesday’s meeting (20 June) at 17:30 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 3. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

1 0

RProposed Agenda and Documents: Meeting 20 June at 1730 UTC
by Julie Hedlund June 18, 2018

June 18, 2018

Dear Data Sub Team Members, In preparation for Wednesday’s meeting (20 June) at 17:30 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 3. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

1 0

Recordings, attendance & AC chat from the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Monday, 18 June 2018
by Julie Bisland June 18, 2018

June 18, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Monday, 18 June 2018 at 16:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/aIMpBQ MP3: https://audio.icann.org/gnso/gnso-rpm-review-18jun18-en.mp3 AC Recording: https://participate.icann.org/p7tsywnxxt7/<https://participate.icann.org/p7tsywnxxt7/?OWASP_CSRFTOKEN=a3029c3a5e9ca4be…> The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Julie

1 0

Actions & Notes: RPM Data Sub Team 18 June 2018
by Ariel Liang June 18, 2018

June 18, 2018

Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 18 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/aIMpBQ Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ACTION ITEM * Staff to email the full WG the schedule of upcoming meetings, including Sub Team calls (DONE) NOTES General Comments * AG will collaborate with ICANN staff to distribute the registrant survey among ICANN community groups that may include potential respondents. * There would be a filter question at the beginning, asking the respondents whether they have registered domains in the new gTLD. If not, they will be filtered to complete the potential registrant survey. Hypothetical questions are posted to both actual registrants/potential registrants. * The way that AG broke out the surveys is correct. * Is there a way to protect the responses to earlier questions being “contaminated”? You have a forward track and you cannot go back. Consider setting up the survey in a way that does not allow the respondents to go back and revise their responses to earlier questions? * There is concern about the single use case, as we may not receive representative responses. * Registrant survey targets 50 complete responses, and potential registrant survey targets 200 complete responses. If we get 200 potential registrants, sub group of 50 is acceptable. * The three domain name examples at the very beginning of the registrant survey are not new gTLDs. Any examples we use must be live. Discussions of the Hypothetical Questions (Doc sent by Kristine) * The respondents do not necessarily need to have already seen a Claims Notice. * Rebecca likes the scenarios/examples proposed by Kristine, but is concerned with asking the respondent multiple times for the various scenarios. This may not match the registrant’s actual experience registering a domain name. * The key is to show a Claims Notice and ask respondents to provide responses to specific questions. virtue.food example would be helpful to elicit thoughts from registrants. We cannot have a survey without some imagined domain name. Other than that, supports examples from Kristine. * Is there a threshold question? The current filter of the hypothetical question is that the respondent attempted to register for a new gTLD domain name but did not complete the registration. * Survey takers would be sophisticated enough to understand that people register different domain names for different uses. * We would be showing a Claims Notice specifically for VIRTUE. * From the way the survey is being set up, isn't the idea to describe the sequence/process that a registrant/potential registrant goes through in order to see how they interact with the domain registration process? That seems to staff to be what is typically described as a "use case". * If we give respondents a particular example, there would be a huge effect on the survey responses. We should at least give a couple of examples. * Regarding Kristine’s proposal, the issue is with regard to survey administration. It can be troublesome to present survey takers a whole list of examples/scenarios. Respondents may shape their responses by comparing these different examples/scenarios. The response we get would not be representative. Happy to add a xerox example. We need to do some subgrouping and random rotation, making sure one respondent will get one xerox example and one virtue example. Simulate the survey to what the registrant would be experiencing. * Don't understand why all the examples could not be presented to the respondents at once. * Strongly oppose to using an actual brand, especially a famous mark. * A survey does best when it replicates the actual reality. What is the best way to get people’s thought process via this survey by simulating the scenario that they are registering a domain name in the new gTLDs? * AG agrees with Rebecca's points. AG proposes to include a screening question: if you are going to register a domain name, what would you most likely register the domain name for? Based on respondents’ answers to the screening question, port/funnel them into a grid question. They would see some, but not all of the options in the grid. * Would the virtue.food example be an arbitrary example? * Funneling could work, but concerned whether it would be too limiting. Can we open the registrant survey up to more than 50 respondents? Can we ask this hypothetical question also to the potential registrants? * AG would ask this hypothetical question to both registrant and potential registrants (targeting a total of 250 responses). Can certainly target more than 50 registrants too. * In the scenario of virtue.food, when we present the Claims Notice of virtue, we need to know what the trademark is registered for. The reason for the registrants to proceed or not to proceed depends on what the TM is registered for. * An actual Claims Notice for https://domainnamewire.com/2014/01/30/trademark-claims-notice/. Virtue is registered for "medical equipment". * There is no chance that an English word is not already registered as a TM, but we can use a TM example of limited rights. We will need to fictionalize the Claims Notice for “virtue”, especially the goods/classes/jurisdiction elements. Do not set people up for an actual cybersquatting experience. Example, "virtue" for doors? * Sub Team is coalescing on an agreement to use virtue.food as an example, show a fictionalized Claims Notice for virtue, use a funnel to direct respondents to questions, ask them what they would do with the domain name based on their own experience. Procedural Matters * On Wed, 20 June, first 30 min is a brief meeting for the full WG to get on the same page in preparation for ICANN62; the remaining 60 min (17:30-18:30 UTC) would be used for the Data Sub Team. * Goal is to finish off the surveys by the Wed meeting for AG to revise/finalize the questions.

1 0

Re: [Gnso-rpm-data] [Ext] RE: REMINDER: Proposed Agenda and Documents: Meeting 18 June at 1600 UTC
by Julie Hedlund June 18, 2018

June 18, 2018

Thanks so much Kristine. We’ll get these loaded into the Adobe Connect room. Best, Julie From: "Dorrain, Kristine" <dorraink(a)amazon.com> Date: Monday, June 18, 2018 at 10:18 AM To: Julie Hedlund <julie.hedlund(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] RE: [Gnso-rpm-data] REMINDER: Proposed Agenda and Documents: Meeting 18 June at 1600 UTC Thanks Julie! All, I’m planning to be on the call since we moved it up (my 5 yo may join me so I’ll adjust my apology to being “mostly” there). In advance of our call, though I admit, not very far in advance, I have put together two straw documents. They’re rough but ready for discussion. Last week I committed to thinking about two things: How we could streamline the sunrise duration questions, which is currently two grids. (not for discussion today, but we’ve been intending to circle back to these eventually) How to reorganize the Claims Notice questions to be situation/use case specific without diving into hypotheticals (I think we’re picking up here today.) [Here, you will notice I removed the reference to virtue.food as I still don’t think it adds anything, but as I said before, I’m not going to die on that hill.] I’m not wedded to these docs but they do reflect my current best suggestion for how to get this information. We will definitely need to review the specific line items and tweak and wordsmith these and of course, we’ll need AG to evaluate the likelihood these will elicit the information we want. Talk to you all soon, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Julie Hedlund Sent: Monday, June 18, 2018 6:45 AM To: gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] REMINDER: Proposed Agenda and Documents: Meeting 18 June at 1600 UTC Dear Data Sub Team Members, In preparation for today’s (Monday) meeting (18 June) at 16:00 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/18K-zhP541IGCR0bXJpBtdmWdpfQ6BbKhFZaaZsf… [docs.google.com] 3. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… [docs.google.com] 4. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

1 0

REMINDER: Proposed Agenda and Documents: Meeting 18 June at 1600 UTC
by Julie Hedlund June 18, 2018

June 18, 2018

Dear Data Sub Team Members, In preparation for today’s (Monday) meeting (18 June) at 16:00 UTC for 60 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/18K-zhP541IGCR0bXJpBtdmWdpfQ6BbKhFZaaZsf… 3. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 4. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

2 1

Re: [Gnso-rpm-data] [Ext] RE: RPM Data Sub Team Meeting Options: Monday, 18 June
by Julie Hedlund June 17, 2018

June 17, 2018

Thanks Kristine! Best, Julie From: "Dorrain, Kristine" <dorraink(a)amazon.com> Date: Friday, June 15, 2018 at 5:31 PM To: Julie Hedlund <julie.hedlund(a)icann.org>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] RE: RPM Data Sub Team Meeting Options: Monday, 18 June I can do either option 2 or option 3. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Julie Hedlund Sent: Friday, June 15, 2018 12:03 PM To: Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] RPM Data Sub Team Meeting Options: Monday, 18 June Importance: High Hello everyone, As we noted on the call and per the notes below, several Sub Team members are not available at the suggested time of 17:00-18:30 UTC on Monday, 18 June, and for others the time is not ideal. Please let us know at your earliest convenience if you would be able to attend the call if it is moved to the any of the following options: OPTION 1: 1 hour earlier, to 16:00-17:30 UTC at 90 minutes OPTION 2: 1 hour earlier to 16:00-17:00 UTC at 60 minutes OPTION 3: 2 hours earlier to 15:00-16:30 UTC at 90 minutes Thank you very much for your assistance and please be on the lookout for a meeting notice on Sunday, or very early Monday. We hope you have a nice weekend! Best, Mary, Ariel, Berry, and Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of Ariel Liang <ariel.liang(a)icann.org> Date: Friday, June 15, 2018 at 2:56 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ [community.icann.org] Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b Change 3 years to 5 years. Q2c Change 3 years to 5 years Q2e "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. Suggestions for options to be included in a multiple-choice question I would not be sued if I continued I would not be subject to an action to take the domain if I continued No one else had a legal right to the name I had legitimate or legal right to the domain It does not seem like too much trouble to continue I consulted with an attorney and it was fine I was committed to the domain name for person/business reasons It matched investments names I already own in a portfolio (going with "theme" registrants) I didn't understand the notice Something else [explain] There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. Registrants with big portfolio might actually prefer a narrative option. Q2g There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) The structure is a bit confusing Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) Re virtue.food - oppose to specify as people may give a false hypothetical May want to specify "I plan to start a business or already have a business under that name" If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. We did give examples on other surveys to target non-expert. How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: Would you be nervous to register the domain name? -- provide options If yes, what actions would you take? -- provide options If no, what is the reason? -- provide options Does trademark law matter? Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. Don't need to limit this question for the people who have not completed the registration. Proposed new order: Show trademark claims notice. Q2g(iv) – confidence? Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] Q2h(i) – Why you abandoned? Q2g(iii) – trademark law knowledge We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. Those general Qs should appear after the more specific responses to virtue.food Q2h(i) We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] AG suggestion: Keep the hypo question for the potential registrant survey In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? Is it possible that most people taking the survey are legacy domain name registrants? We account for those potential respondents and try to filter them in the survey. It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

1 1

Surveying potential registrants (Re: Actions & Notes: RPM Data Sub Team 15 June 2018)
by Mary Wong June 16, 2018

June 16, 2018

Dear all, Thank you to Rebecca, Kristine and Michael for continuing the discussion. At the end of the last sub team call, the following appeared to be the decision points under consideration for the actual and potential registrant surveys: * The “actual registrant” and “potential registrant” surveys are two separate surveys; however, for the “actual registrant” target group, it may be helpful to also have a section that mirrors part of the “potential registrant” survey, so as to capture the views of those who succeeded in registering one/some domain(s) but decided not to proceed with other attempted registrations. * In both the “actual registrant” and “potential registrant” surveys, it will be important (possibly even necessary) to make clear that the respondent knows what a Claims Notice looks like – this may entail actually showing the respondent (during the survey?) a copy of the Claims Notice. * While “use cases” and illustrative examples using a hypothetical domain name can be considered but may not ultimately be necessary, what remains critical is eliciting responses where the respondent understands what a Claims Notice is and knows what it looks like. * Some reordering/reformatting of the questions may be necessary in order to present respondents with a logical process flow and to facilitate responses. If it will be of assistance, do you wish staff to request that Greg and/or Stacey provide a more specific description, on one of your next calls with them, as to what methods they and their subcontractor will be using to identify, target and reach the category of respondents that we are describing as “actual registrants” (i.e. those that succeeded in registering domain names especially in the new gTLDs) and “potential registrants” (i.e. those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future)? Thanks and cheers Mary, Julie, Ariel & Berry From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Saturday, June 16, 2018 at 06:38 To: "Dorrain, Kristine" <dorraink(a)amazon.com>, "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks for taking the time to go through all this, Kristine – especially since my attendance today was somewhat sporadic. I do understand the scope of what we are trying to get, but I agree that we should ask specific questions about actual responses to the Claims Notice, and not ask hypotheticals. So, I can go along with asking registrants, applicants who did not obtain a registration, and potential domain name applicants what they understand the Claims Notice to say, what their reaction to it would be, etc. However, I think we need to ask the same question of registrants in two classes: Those who have received Trademark Claims Notices, and those who have not. More importantly, in asking, parsing, analyzing and attempting to draw any conclusions from the results, we must be sure that AG draws clear distinctions between the different groups being asked these questions, their answers, and the significance of their answers in light of their different levels and types of experience and concern. I am therefore agreeable, but extremely cautious in light of past surveys and analyses. Michael R. From: Dorrain, Kristine [mailto:dorraink@amazon.com] Sent: Friday, June 15, 2018 2:49 PM To: Michael Graham (ELCA) <migraham(a)expedia.com>; Tushnet, Rebecca <rtushnet(a)law.harvard.edu>; Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Hi Michael, I do understand your concern but I think we ARE providing the Analysis Group with clear instructions about who to survey and how to ask the questions – and they’ve told us how they will get the participants. That’s the point of the calls and that’s indeed what we’re discussing here, now. Correct me if I’m misunderstanding, but I take your suggestion to mean you oppose surveying potential registrants at all. I don’t think we are at liberty to do that and I’ll explain: 1. We have charter questions that ask how understandable the Claims Notice is and we have some (at least anecdotal, abandonment) information that people may have been scared away by the Claims Notice. 2. As a result, we are expressly asked by the full WG to figure out if the Claims Notice is scary or needs to be re-worded. 3. We are also asked to figure out how people from different places of the world perceive the Claims Notice, as presented and worded (ideally, in a variety of languages- I have not seen how AG plans to address this) ^^These, as I understand it, are not negotiable. These are the pieces of information the WG asked to get survey data on. Therefore, we MUST present the Claims Notice to some people (ideally, people who would actually register a domain name) to learn how people react to it. Now, we can for sure get some of this information from people who succeeded in registering domain names – they completed the process. But we are also looking to get this information from people who opted not to proceed for whatever reason, when presented with a claims notice. These are “potential registrants.” And yes, for people who actually didn’t try yet or didn’t proceed, these questions are somewhat hypothetical, but what we’re attempting to get is a REACTION and an idea of how it might affect future behavior, not past data. Think of this part as market research, not historical data. The “hypothetical” disagreement is just in the level detail, as I understand it. I’ve been an outspoken critic of hypothetical questions. My solution is to give some general use cases (you want a domain name for a blog or for your business or as an investment) and then ask if the Claims Notice would be scary or a deterrent and why. Rebecca likes the idea of being more specific (providing an example name at least, maybe more of a detailed scenario). If I’ve misunderstood, I hope you’ll set me straight. Best, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Michael Graham (ELCA) Sent: Friday, June 15, 2018 2:42 PM To: Tushnet, Rebecca <rtushnet(a)law.harvard.edu<mailto:rtushnet@law.harvard.edu>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Understood. However, this sort of research requires a clear identification of the universe to be studied, the wording that will be used, and the environment in which it will be asked. I think we need some clear parameters and questions, and I don’t think we have those yet. Hopefully after our call today, AG can provide some guidance. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:27 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 As I said on the call, this is standard comprehension research, relied on by courts and marketing departments. When you want to know what members of a group take away from a message, you show it to them and ask them about it, even if they haven't seen it before. It is precisely the kind of data driven research this working group is supposed to conduct. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 5:12:17 PM To: Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Rebecca. From your explanation, then, we do not want to ask hypothetical questions, but want to ask what subjects’ understanding of the Claims Notice is? That makes sense if asked of actual registrants or applicants who received the notices. I don’t think it makes sense, however, to ask this question of those who have not gone through the process. Then it becomes a hypothetical: If you received this notice, what does it tell you? Without the context of involvement in the process I don’t think “potential registrants” would provide relevant or useful information. I’ll go back to the questions after the weekend to see how and who they address the issue of effectiveness of the actual Claims Notice to. That will better answer my concerns I hope. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:01 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 Surveying potential registrants (defined in the survey as those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future) for their understanding of the meaning of the claims notice is not hypothetical--it is a standard reception survey of the target audience. The question is not generally what issues kept some potential registrants away--that is a good deal broader than our task, I believe. The question is what people considering new gTLD registrations take away from the Claims Notice when they read it. That is one of the key questions this working group has about the functioning of the Claims Notice. Past and future new gTLD registrants are our universe. Depending on their past experiences, including receipt of claims notices (if we get any such respondents) we may have different questions for different subgroups. But they are the target group as a whole, and should get the same questions, except for those that don't apply to them. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org<mailto:gnso-rpm-data-bounces@icann.org>> on behalf of Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 4:50:32 PM To: Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Ariel. I will review the Tape (as they say), but believe I disagree with the last point unless the meaning is that we would send a single survey to both registrants and “potential registrants” (although I would like to understand better how this last universe will be identified/selected) but according to which group they fall into, they will follow separate branches – registrants being asked whether they received Claims Notices and what they did/how they reacted; and potential registrants being asked why they did not apply for registration – i.e. what hypothetical issues could arise that kept them away. I do not think we should mix them or that we should ask purely hypothetical questions of anyone since these will only obtain opinions, beliefs, and perceptions – not what the effects of the RPMs actually have been. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, June 15, 2018 11:56 AM To: gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM * Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 * 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years * If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b * Change 3 years to 5 years. Q2c * Change 3 years to 5 years Q2e * "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. * We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. * Suggestions for options to be included in a multiple-choice question - I would not be sued if I continued - I would not be subject to an action to take the domain if I continued - No one else had a legal right to the name - I had legitimate or legal right to the domain - It does not seem like too much trouble to continue - I consulted with an attorney and it was fine - I was committed to the domain name for person/business reasons - It matched investments names I already own in a portfolio (going with "theme" registrants) - I didn't understand the notice - Something else [explain] * There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. * Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. * Registrants with big portfolio might actually prefer a narrative option. Q2g * There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) * The structure is a bit confusing * Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not * Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) * Re virtue.food - oppose to specify as people may give a false hypothetical * May want to specify "I plan to start a business or already have a business under that name" * If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. * We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. * We did give examples on other surveys to target non-expert. * How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? * We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: - Would you be nervous to register the domain name? -- provide options - If yes, what actions would you take? -- provide options - If no, what is the reason? -- provide options - Does trademark law matter? * Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. * Don't need to limit this question for the people who have not completed the registration. * Proposed new order: - Show trademark claims notice. - Q2g(iv) – confidence? - Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] - Q2h(i) – Why you abandoned? - Q2g(iii) – trademark law knowledge * We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... * One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. * Those general Qs should appear after the more specific responses to virtue.food Q2h(i) * We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? * This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions * Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. * Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. * Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. * There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions * Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions * Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. * If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? * I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. * I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 * The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. * Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. * We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. * The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] * AG suggestion: - Keep the hypo question for the potential registrant survey - In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? * Is it possible that most people taking the survey are legacy domain name registrants? * We account for those potential respondents and try to filter them in the survey. * It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

3 2

Re: [Gnso-rpm-data] [Ext] Re: Surveying potential registrants (Re: Actions & Notes: RPM Data Sub Team 15 June 2018)
by Mary Wong June 16, 2018

June 16, 2018

Hello Rebecca and everyone – please accept apologies for any lack of clarity; my note was not intended to supersede or suggest a direction for any of the ongoing discussions among and between this Sub Team and the Analysis Group. Rather, and based on a review of the call recording and the ongoing email discussions, it was an attempt to summarize where the discussion seemed to be. Please see some additional clarifying notes in blue below. I am sorry if, in an attempt to provide a quick summary that staff had hoped might facilitate your next discussion, the message may instead have been mistaken as to its intent and meaning. Please feel free to disregard the summary as a result. Best regards, Mary From: "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu> Date: Saturday, June 16, 2018 at 11:14 To: Mary Wong <mary.wong(a)icann.org>, "Michael Graham (ELCA)" <migraham(a)expedia.com>, "Dorrain, Kristine" <dorraink(a)amazon.com>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] Re: Surveying potential registrants (Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018) Mary, can you explain what you mean by decision points? MW: All that phrase was intended to convey is that these are the topics that the Sub Team discussed on the call for which the Analysis Group will likely need to either clarify or revise certain questions and their sequencing, or for which Sub Team members may agree that more specific clarity is needed. The thought was that a summary may be helpful as there seemed to be some confusion and lack of clarity as to what was presented by the Analysis Group and why certain questions were either included or phrased in a certain way or sequence. What you have presented below does not completely track with what the AG presented or my understanding of what's under discussion: For point one, "it may be helpful to have a section...." is inconsistent with the questions with which we were presented, where the two groups (which will actually be constituted by their responses to the first few questions of the survey) got a number of the same questions. Michael apparently doesn't want to survey potential registrants, but that's a very different suggestion from completely rewriting the survey tracks. MW: This was, again, an attempt to summarize the main point under discussion on this topic. It was not a staff attempt to predict or suggest what the final decision as between the Sub Team and the Analysis Group will be. As you noted, Michael had a specific question about doing two surveys which Kristine answered as part of the subsequent email discussion. There was no attempt by staff to rewrite anything or to suggest a different type or number of surveys, we are merely trying to provide a quick summary/refresher to facilitate the next call and discussion. For point two "possibly even necessary"--our mandate is, among other things, to figure out if the Claims Notice is understood by its recipients. I can't see any scenario where "possibly" is in the picture: it will absolutely be necessary to let people read the thing we're asking if they understand, or understood if they saw it in the past. (Among other things, a person who says they got a Claims Notice and then realizes that they don't recognize what they're seeing should not be counted as someone who got a Claims Notice.) I didn't understand anyone on the call to suggest that we shouldn't show the Claims Notice and I'm surprised that's entered the conversation. MW: Again, this is an attempt to summarize what the discussion was about, and to ensure that there is a clear understanding among the Sub Team and the Analysis Group that the Claims Notice must be shown as part of the survey process. I strongly disagree with your characterization of point three; we are apparently debating what it means to "understand what a Claims Notice is." Knowing what it looks like is one thing; understanding what they think it means is one of our goals in the WG. I agree with Kristine: we are charged with figuring out how recipients understand it, which can only be answered by asking what they think it means. Perhaps this should be reframed as "answering the relevant charter questions about the intelligibility of the Claims Notice." MW: I apologize if the choice of summary language was imprecise or gave the wrong impression. Again, this was a quick summary of the points that were discussed. It was not a suggestion about framing or phrasing the specific survey questions – that is for the Analysis Group to do as the professional contractor retained for this purpose. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Mary Wong <mary.wong(a)icann.org> Sent: Friday, June 15, 2018 10:33 PM To: Michael Graham (ELCA); Dorrain, Kristine; Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org Subject: Surveying potential registrants (Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018) Dear all, Thank you to Rebecca, Kristine and Michael for continuing the discussion. At the end of the last sub team call, the following appeared to be the decision points under consideration for the actual and potential registrant surveys: * The “actual registrant” and “potential registrant” surveys are two separate surveys; however, for the “actual registrant” target group, it may be helpful to also have a section that mirrors part of the “potential registrant” survey, so as to capture the views of those who succeeded in registering one/some domain(s) but decided not to proceed with other attempted registrations. * In both the “actual registrant” and “potential registrant” surveys, it will be important (possibly even necessary) to make clear that the respondent knows what a Claims Notice looks like – this may entail actually showing the respondent (during the survey?) a copy of the Claims Notice. * While “use cases” and illustrative examples using a hypothetical domain name can be considered but may not ultimately be necessary, what remains critical is eliciting responses where the respondent understands what a Claims Notice is and knows what it looks like. * Some reordering/reformatting of the questions may be necessary in order to present respondents with a logical process flow and to facilitate responses. If it will be of assistance, do you wish staff to request that Greg and/or Stacey provide a more specific description, on one of your next calls with them, as to what methods they and their subcontractor will be using to identify, target and reach the category of respondents that we are describing as “actual registrants” (i.e. those that succeeded in registering domain names especially in the new gTLDs) and “potential registrants” (i.e. those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future)? Thanks and cheers Mary, Julie, Ariel & Berry From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Saturday, June 16, 2018 at 06:38 To: "Dorrain, Kristine" <dorraink(a)amazon.com>, "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks for taking the time to go through all this, Kristine – especially since my attendance today was somewhat sporadic. I do understand the scope of what we are trying to get, but I agree that we should ask specific questions about actual responses to the Claims Notice, and not ask hypotheticals. So, I can go along with asking registrants, applicants who did not obtain a registration, and potential domain name applicants what they understand the Claims Notice to say, what their reaction to it would be, etc. However, I think we need to ask the same question of registrants in two classes: Those who have received Trademark Claims Notices, and those who have not. More importantly, in asking, parsing, analyzing and attempting to draw any conclusions from the results, we must be sure that AG draws clear distinctions between the different groups being asked these questions, their answers, and the significance of their answers in light of their different levels and types of experience and concern. I am therefore agreeable, but extremely cautious in light of past surveys and analyses. Michael R. From: Dorrain, Kristine [mailto:dorraink@amazon.com] Sent: Friday, June 15, 2018 2:49 PM To: Michael Graham (ELCA) <migraham(a)expedia.com>; Tushnet, Rebecca <rtushnet(a)law.harvard.edu>; Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Hi Michael, I do understand your concern but I think we ARE providing the Analysis Group with clear instructions about who to survey and how to ask the questions – and they’ve told us how they will get the participants. That’s the point of the calls and that’s indeed what we’re discussing here, now. Correct me if I’m misunderstanding, but I take your suggestion to mean you oppose surveying potential registrants at all. I don’t think we are at liberty to do that and I’ll explain: 1. We have charter questions that ask how understandable the Claims Notice is and we have some (at least anecdotal, abandonment) information that people may have been scared away by the Claims Notice. 2. As a result, we are expressly asked by the full WG to figure out if the Claims Notice is scary or needs to be re-worded. 3. We are also asked to figure out how people from different places of the world perceive the Claims Notice, as presented and worded (ideally, in a variety of languages- I have not seen how AG plans to address this) ^^These, as I understand it, are not negotiable. These are the pieces of information the WG asked to get survey data on. Therefore, we MUST present the Claims Notice to some people (ideally, people who would actually register a domain name) to learn how people react to it. Now, we can for sure get some of this information from people who succeeded in registering domain names – they completed the process. But we are also looking to get this information from people who opted not to proceed for whatever reason, when presented with a claims notice. These are “potential registrants.” And yes, for people who actually didn’t try yet or didn’t proceed, these questions are somewhat hypothetical, but what we’re attempting to get is a REACTION and an idea of how it might affect future behavior, not past data. Think of this part as market research, not historical data. The “hypothetical” disagreement is just in the level detail, as I understand it. I’ve been an outspoken critic of hypothetical questions. My solution is to give some general use cases (you want a domain name for a blog or for your business or as an investment) and then ask if the Claims Notice would be scary or a deterrent and why. Rebecca likes the idea of being more specific (providing an example name at least, maybe more of a detailed scenario). If I’ve misunderstood, I hope you’ll set me straight. Best, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Michael Graham (ELCA) Sent: Friday, June 15, 2018 2:42 PM To: Tushnet, Rebecca <rtushnet(a)law.harvard.edu<mailto:rtushnet@law.harvard.edu>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Understood. However, this sort of research requires a clear identification of the universe to be studied, the wording that will be used, and the environment in which it will be asked. I think we need some clear parameters and questions, and I don’t think we have those yet. Hopefully after our call today, AG can provide some guidance. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:27 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 As I said on the call, this is standard comprehension research, relied on by courts and marketing departments. When you want to know what members of a group take away from a message, you show it to them and ask them about it, even if they haven't seen it before. It is precisely the kind of data driven research this working group is supposed to conduct. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 5:12:17 PM To: Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Rebecca. From your explanation, then, we do not want to ask hypothetical questions, but want to ask what subjects’ understanding of the Claims Notice is? That makes sense if asked of actual registrants or applicants who received the notices. I don’t think it makes sense, however, to ask this question of those who have not gone through the process. Then it becomes a hypothetical: If you received this notice, what does it tell you? Without the context of involvement in the process I don’t think “potential registrants” would provide relevant or useful information. I’ll go back to the questions after the weekend to see how and who they address the issue of effectiveness of the actual Claims Notice to. That will better answer my concerns I hope. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:01 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 Surveying potential registrants (defined in the survey as those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future) for their understanding of the meaning of the claims notice is not hypothetical--it is a standard reception survey of the target audience. The question is not generally what issues kept some potential registrants away--that is a good deal broader than our task, I believe. The question is what people considering new gTLD registrations take away from the Claims Notice when they read it. That is one of the key questions this working group has about the functioning of the Claims Notice. Past and future new gTLD registrants are our universe. Depending on their past experiences, including receipt of claims notices (if we get any such respondents) we may have different questions for different subgroups. But they are the target group as a whole, and should get the same questions, except for those that don't apply to them. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org<mailto:gnso-rpm-data-bounces@icann.org>> on behalf of Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 4:50:32 PM To: Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Ariel. I will review the Tape (as they say), but believe I disagree with the last point unless the meaning is that we would send a single survey to both registrants and “potential registrants” (although I would like to understand better how this last universe will be identified/selected) but according to which group they fall into, they will follow separate branches – registrants being asked whether they received Claims Notices and what they did/how they reacted; and potential registrants being asked why they did not apply for registration – i.e. what hypothetical issues could arise that kept them away. I do not think we should mix them or that we should ask purely hypothetical questions of anyone since these will only obtain opinions, beliefs, and perceptions – not what the effects of the RPMs actually have been. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, June 15, 2018 11:56 AM To: gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM * Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 * 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years * If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b * Change 3 years to 5 years. Q2c * Change 3 years to 5 years Q2e * "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. * We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. * Suggestions for options to be included in a multiple-choice question - I would not be sued if I continued - I would not be subject to an action to take the domain if I continued - No one else had a legal right to the name - I had legitimate or legal right to the domain - It does not seem like too much trouble to continue - I consulted with an attorney and it was fine - I was committed to the domain name for person/business reasons - It matched investments names I already own in a portfolio (going with "theme" registrants) - I didn't understand the notice - Something else [explain] * There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. * Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. * Registrants with big portfolio might actually prefer a narrative option. Q2g * There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) * The structure is a bit confusing * Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not * Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) * Re virtue.food - oppose to specify as people may give a false hypothetical * May want to specify "I plan to start a business or already have a business under that name" * If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. * We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. * We did give examples on other surveys to target non-expert. * How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? * We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: - Would you be nervous to register the domain name? -- provide options - If yes, what actions would you take? -- provide options - If no, what is the reason? -- provide options - Does trademark law matter? * Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. * Don't need to limit this question for the people who have not completed the registration. * Proposed new order: - Show trademark claims notice. - Q2g(iv) – confidence? - Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] - Q2h(i) – Why you abandoned? - Q2g(iii) – trademark law knowledge * We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... * One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. * Those general Qs should appear after the more specific responses to virtue.food Q2h(i) * We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? * This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions * Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. * Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. * Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. * There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions * Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions * Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. * If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? * I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. * I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 * The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. * Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. * We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. * The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] * AG suggestion: - Keep the hypo question for the potential registrant survey - In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? * Is it possible that most people taking the survey are legacy domain name registrants? * We account for those potential respondents and try to filter them in the survey. * It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

1 0

Recordings, attendance & AC chat from the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 15 June 2018
by Michelle DeSmyter June 16, 2018

June 16, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 15 June 2018 at 16:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/ZYMpBQ MP3: https://audio.icann.org/gnso/gnso-review-rpms-pdp-sub-team-data-15jun18-en.… AC Recording: https://participate.icann.org/p7igmayvcel/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar [gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Michelle

1 0