Rebecca: Since I'm trying to revise the "harm" question-samples, I want to understand your question and comments. My understanding (and I can stand corrected) is that "data" seeks empirically quantifiable answers (e.g. "How many UDRP actions have you filed?" "How many NORNs have you received?") whereas "anecdotal" seeks narrative responses. Obviously, the fact that we receive 5 anecdotal responses stating the same thing may become data, but that would be in the analysis, and not in the questions. So what I presumed was that by categorizing questions as "data" or "anecdotal" we are referring to the type of answers we expect to receive from survey takers. Michael R. Graham From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Tushnet, Rebecca Sent: Friday, February 16, 2018 1:59 PM To: Julie Hedlund <julie.hedlund@icann.org>; gnso-rpm-data@icann.org Subject: Re: [Gnso-rpm-data] Action Items from 16 February 2018 RPM Data Sub Team Call Thanks for sending these notes. I will take up the drafting of a few sentences. I'm not sure the list of questions is more helpful than the basic question we're trying to answer but I am willing to put them back in as examples of what we came up with but don't think are necessarily good wording for eliciting reactions from nonlawyers. One comment: I am increasingly puzzled by the way "anecdotal" is being used across the board, including for the evidence of harm and as inserted in part 5. In a properly sampled, representative survey, the plural of anecdote actually is data. It seems to me that, in particular, contemplated long form answers to the survey are being tossed into the "anecdote" pile for no particularly good reason. Could someone explain their reasoning on these? Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Gnso-rpm-data <gnso-rpm-data-bounces@icann.org<mailto:gnso-rpm-data-bounces@icann.org>> on behalf of Julie Hedlund <julie.hedlund@icann.org<mailto:julie.hedlund@icann.org>> Sent: Friday, February 16, 2018 4:08:47 PM To: gnso-rpm-data@icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Action Items from 16 February 2018 RPM Data Sub Team Call All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting today (16 February 2018). The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records. Note also that the next call will be on Friday, 23 February at 1700 UTC. Best Regards, Julie Julie Hedlund, Policy Director Action Items: 1. Staff will update Section 5. Survey of Potential Registrants beginning on page 33 in the Google Doc based on the discussion during today's call. 2. With respect to hypotheticals (question on 35-36 in the Google Doc) Sub Team members should volunteer to produce two documents and circulate them on the list prior to the 23 February call: * Short summary of what they hope to get from the use of hypotheticals; * Short summary of why not to use a hypothetical. Kristine has volunteered for the second item. 1. RPM Data Sub Team meeting at ICANN61: The RPM Co-Chairs will discuss whether one of the slots could be repurposed for the RPM Data Sub Team. 2. Section 3, Survey of TM and Brand Owners: Sub Team members should review and comment on the list concerning the final anecdotal and data questions for the question "What is the evidence of harm under the existing [exact match] system?", page 22-23. Notes: 5. Survey of Potential Registrants. Page 33 -- Inserted note to the survey providers: Language is helpful and clear. Page 34 -- Have you attempted to register a domain name in one of these new domains? [If no to both, terminate the survey] -- Thought we weren't going to suggest terminating the survey. But if they aren't considering registering and haven't tried to register then they aren't a potential registrant. Or do we still want them to look at the claims notice? If they continue with the survey, it might be good to find if the process seems confusing. -- Leave the language as is, and also give the survey provider latitude to change it. Page 34-35 -- The wording of this set of questions should be discussed with the surveyor... [etc.] -- Gather information multiple different ways. -- Question: In other cases we've been more specific; have we given enough direction here? -- Thought we were leaving in all of the variations of the questions? Examples -- deleted in the Google doc: "If, when registering your domain, you received a notice with the following wording, would you believe you were: legally allowed to continue with the registration, not legally allowed to continue, not be sure if you were legally allowed to continue?" etc. -- Bring back the deleted questions and include them in the note: The wording of this set of questions should be discussed with the surveyor. We would like to present the language of the notice to people in all relevant groups (1-3) and ask comprehension questions. The questions should be as neutral and non-leading as possible. This may involve beginning with "explain in your own words" and then asking more directed questions about perceptions of legal rights and risks, as well as "what would you do next if you received the notice when you attempted to register?" (e.g., stop, continue, consult someone else, etc.) and "why or why not?" - Anecdotal question. Other sample questions: If, when registering your domain, you received a notice with the following wording, would you believe you were: a. legally allowed to continue with the registration b. not legally allowed to continue b. not be sure if you were legally allowed to continue? If, when registering your domain, you received a notice with the following wording, would you believe you would: a. definitely get sued if you continued b. might or might not get sued c. definitely would not get sued if you continued? If, when registering your domain, you received a notice with the following wording, would you believe that: a. you had a legal right to continue with the registration? b. you might or might not have a legal right to continue with the registration? c. you had no legal right to continue with the registration? What would you do if you received a notice with the following wording: a. continue with the registration b. not continue with the registration c. consult someone else [who] something else [explain] Page 35-36: Consider offering Hypothetical: Famous maker of computers, Horse, and scenarios at extremes such as horse.computers and horse.farm -- if consumer attempting to register these received notice, what would they do? -- Did not have consensus on this. Guidance on what to present to the full WG. -- Trying to get to a general understanding of what is trademark law, but not sure offering hypotheticals is the best way to do this. Perhaps in a more direct fashion? -- Is that the purpose of the Hypothetical? Not automatically opposed to hypotheticals, but need to understand what info we're trying to get at & whether there's a less subjective way to get that info. -- The idea is to try to figure out if people would be scared away from non-infringing uses of TM terms. -- Could ask the survey provider if this helps. -- It actually sounds like we don't all have the same understanding of what info the hypotheticals are trying to elicit. -- But the problem is we can never know what intent is and even if someone says I think it's ok to get "apple.farm" if their intent is bad faith (which we would not know). -- There is no right answer. -- How useful is the data? Could also get adverse inferences. Not in agreement on what we would get by using a hypothetical. Okay if we are trying to elicite specific information that is actionable. -- Anything that helps us to make the claims notice is helpful. ACTION: Leave this suggestion (the hypothetical) and ask for a volunteer to make a short summary of what they hope to get from the hypothetical (Rebecca?). Kristine would be willing to write why not to use a hypothetical. Susan has offered to help. 2. Timing of future meetings & ICANN61 -- Try to knock the Additional Marketplace RPMs on the head as quickly as possible. -- Have a big session in PR to get it done. Make the room set up as a working session. Make it clear that people shouldn't be there if they aren't going to be working. Not a presentation of findings. Make sure we have remote participation. -- Could be difficult to schedule an additional meeting given conflicts, and "pop-up" rooms do not have remote access. -- The RPM WG has 4 session slots for a total of 6 hours; the Co-Chairs will discuss whether one of the slots could be repurposed for the RPM Data Sub Team.