June 2018 - Gnso-rpm-data - lists.icann.org

Surveying potential registrants (Re: Actions & Notes: RPM Data Sub Team 15 June 2018)
by Mary Wong June 16, 2018

June 16, 2018

Dear all, Thank you to Rebecca, Kristine and Michael for continuing the discussion. At the end of the last sub team call, the following appeared to be the decision points under consideration for the actual and potential registrant surveys: * The “actual registrant” and “potential registrant” surveys are two separate surveys; however, for the “actual registrant” target group, it may be helpful to also have a section that mirrors part of the “potential registrant” survey, so as to capture the views of those who succeeded in registering one/some domain(s) but decided not to proceed with other attempted registrations. * In both the “actual registrant” and “potential registrant” surveys, it will be important (possibly even necessary) to make clear that the respondent knows what a Claims Notice looks like – this may entail actually showing the respondent (during the survey?) a copy of the Claims Notice. * While “use cases” and illustrative examples using a hypothetical domain name can be considered but may not ultimately be necessary, what remains critical is eliciting responses where the respondent understands what a Claims Notice is and knows what it looks like. * Some reordering/reformatting of the questions may be necessary in order to present respondents with a logical process flow and to facilitate responses. If it will be of assistance, do you wish staff to request that Greg and/or Stacey provide a more specific description, on one of your next calls with them, as to what methods they and their subcontractor will be using to identify, target and reach the category of respondents that we are describing as “actual registrants” (i.e. those that succeeded in registering domain names especially in the new gTLDs) and “potential registrants” (i.e. those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future)? Thanks and cheers Mary, Julie, Ariel & Berry From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Saturday, June 16, 2018 at 06:38 To: "Dorrain, Kristine" <dorraink(a)amazon.com>, "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks for taking the time to go through all this, Kristine – especially since my attendance today was somewhat sporadic. I do understand the scope of what we are trying to get, but I agree that we should ask specific questions about actual responses to the Claims Notice, and not ask hypotheticals. So, I can go along with asking registrants, applicants who did not obtain a registration, and potential domain name applicants what they understand the Claims Notice to say, what their reaction to it would be, etc. However, I think we need to ask the same question of registrants in two classes: Those who have received Trademark Claims Notices, and those who have not. More importantly, in asking, parsing, analyzing and attempting to draw any conclusions from the results, we must be sure that AG draws clear distinctions between the different groups being asked these questions, their answers, and the significance of their answers in light of their different levels and types of experience and concern. I am therefore agreeable, but extremely cautious in light of past surveys and analyses. Michael R. From: Dorrain, Kristine [mailto:dorraink@amazon.com] Sent: Friday, June 15, 2018 2:49 PM To: Michael Graham (ELCA) <migraham(a)expedia.com>; Tushnet, Rebecca <rtushnet(a)law.harvard.edu>; Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Hi Michael, I do understand your concern but I think we ARE providing the Analysis Group with clear instructions about who to survey and how to ask the questions – and they’ve told us how they will get the participants. That’s the point of the calls and that’s indeed what we’re discussing here, now. Correct me if I’m misunderstanding, but I take your suggestion to mean you oppose surveying potential registrants at all. I don’t think we are at liberty to do that and I’ll explain: 1. We have charter questions that ask how understandable the Claims Notice is and we have some (at least anecdotal, abandonment) information that people may have been scared away by the Claims Notice. 2. As a result, we are expressly asked by the full WG to figure out if the Claims Notice is scary or needs to be re-worded. 3. We are also asked to figure out how people from different places of the world perceive the Claims Notice, as presented and worded (ideally, in a variety of languages- I have not seen how AG plans to address this) ^^These, as I understand it, are not negotiable. These are the pieces of information the WG asked to get survey data on. Therefore, we MUST present the Claims Notice to some people (ideally, people who would actually register a domain name) to learn how people react to it. Now, we can for sure get some of this information from people who succeeded in registering domain names – they completed the process. But we are also looking to get this information from people who opted not to proceed for whatever reason, when presented with a claims notice. These are “potential registrants.” And yes, for people who actually didn’t try yet or didn’t proceed, these questions are somewhat hypothetical, but what we’re attempting to get is a REACTION and an idea of how it might affect future behavior, not past data. Think of this part as market research, not historical data. The “hypothetical” disagreement is just in the level detail, as I understand it. I’ve been an outspoken critic of hypothetical questions. My solution is to give some general use cases (you want a domain name for a blog or for your business or as an investment) and then ask if the Claims Notice would be scary or a deterrent and why. Rebecca likes the idea of being more specific (providing an example name at least, maybe more of a detailed scenario). If I’ve misunderstood, I hope you’ll set me straight. Best, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Michael Graham (ELCA) Sent: Friday, June 15, 2018 2:42 PM To: Tushnet, Rebecca <rtushnet(a)law.harvard.edu<mailto:rtushnet@law.harvard.edu>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Understood. However, this sort of research requires a clear identification of the universe to be studied, the wording that will be used, and the environment in which it will be asked. I think we need some clear parameters and questions, and I don’t think we have those yet. Hopefully after our call today, AG can provide some guidance. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:27 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 As I said on the call, this is standard comprehension research, relied on by courts and marketing departments. When you want to know what members of a group take away from a message, you show it to them and ask them about it, even if they haven't seen it before. It is precisely the kind of data driven research this working group is supposed to conduct. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 5:12:17 PM To: Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Rebecca. From your explanation, then, we do not want to ask hypothetical questions, but want to ask what subjects’ understanding of the Claims Notice is? That makes sense if asked of actual registrants or applicants who received the notices. I don’t think it makes sense, however, to ask this question of those who have not gone through the process. Then it becomes a hypothetical: If you received this notice, what does it tell you? Without the context of involvement in the process I don’t think “potential registrants” would provide relevant or useful information. I’ll go back to the questions after the weekend to see how and who they address the issue of effectiveness of the actual Claims Notice to. That will better answer my concerns I hope. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:01 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 Surveying potential registrants (defined in the survey as those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future) for their understanding of the meaning of the claims notice is not hypothetical--it is a standard reception survey of the target audience. The question is not generally what issues kept some potential registrants away--that is a good deal broader than our task, I believe. The question is what people considering new gTLD registrations take away from the Claims Notice when they read it. That is one of the key questions this working group has about the functioning of the Claims Notice. Past and future new gTLD registrants are our universe. Depending on their past experiences, including receipt of claims notices (if we get any such respondents) we may have different questions for different subgroups. But they are the target group as a whole, and should get the same questions, except for those that don't apply to them. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org<mailto:gnso-rpm-data-bounces@icann.org>> on behalf of Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 4:50:32 PM To: Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Ariel. I will review the Tape (as they say), but believe I disagree with the last point unless the meaning is that we would send a single survey to both registrants and “potential registrants” (although I would like to understand better how this last universe will be identified/selected) but according to which group they fall into, they will follow separate branches – registrants being asked whether they received Claims Notices and what they did/how they reacted; and potential registrants being asked why they did not apply for registration – i.e. what hypothetical issues could arise that kept them away. I do not think we should mix them or that we should ask purely hypothetical questions of anyone since these will only obtain opinions, beliefs, and perceptions – not what the effects of the RPMs actually have been. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, June 15, 2018 11:56 AM To: gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM * Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 * 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years * If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b * Change 3 years to 5 years. Q2c * Change 3 years to 5 years Q2e * "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. * We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. * Suggestions for options to be included in a multiple-choice question - I would not be sued if I continued - I would not be subject to an action to take the domain if I continued - No one else had a legal right to the name - I had legitimate or legal right to the domain - It does not seem like too much trouble to continue - I consulted with an attorney and it was fine - I was committed to the domain name for person/business reasons - It matched investments names I already own in a portfolio (going with "theme" registrants) - I didn't understand the notice - Something else [explain] * There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. * Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. * Registrants with big portfolio might actually prefer a narrative option. Q2g * There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) * The structure is a bit confusing * Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not * Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) * Re virtue.food - oppose to specify as people may give a false hypothetical * May want to specify "I plan to start a business or already have a business under that name" * If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. * We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. * We did give examples on other surveys to target non-expert. * How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? * We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: - Would you be nervous to register the domain name? -- provide options - If yes, what actions would you take? -- provide options - If no, what is the reason? -- provide options - Does trademark law matter? * Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. * Don't need to limit this question for the people who have not completed the registration. * Proposed new order: - Show trademark claims notice. - Q2g(iv) – confidence? - Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] - Q2h(i) – Why you abandoned? - Q2g(iii) – trademark law knowledge * We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... * One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. * Those general Qs should appear after the more specific responses to virtue.food Q2h(i) * We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? * This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions * Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. * Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. * Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. * There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions * Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions * Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. * If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? * I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. * I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 * The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. * Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. * We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. * The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] * AG suggestion: - Keep the hypo question for the potential registrant survey - In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? * Is it possible that most people taking the survey are legacy domain name registrants? * We account for those potential respondents and try to filter them in the survey. * It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

3 2

Re: [Gnso-rpm-data] [Ext] Re: Surveying potential registrants (Re: Actions & Notes: RPM Data Sub Team 15 June 2018)
by Mary Wong June 16, 2018

June 16, 2018

Hello Rebecca and everyone – please accept apologies for any lack of clarity; my note was not intended to supersede or suggest a direction for any of the ongoing discussions among and between this Sub Team and the Analysis Group. Rather, and based on a review of the call recording and the ongoing email discussions, it was an attempt to summarize where the discussion seemed to be. Please see some additional clarifying notes in blue below. I am sorry if, in an attempt to provide a quick summary that staff had hoped might facilitate your next discussion, the message may instead have been mistaken as to its intent and meaning. Please feel free to disregard the summary as a result. Best regards, Mary From: "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu> Date: Saturday, June 16, 2018 at 11:14 To: Mary Wong <mary.wong(a)icann.org>, "Michael Graham (ELCA)" <migraham(a)expedia.com>, "Dorrain, Kristine" <dorraink(a)amazon.com>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] Re: Surveying potential registrants (Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018) Mary, can you explain what you mean by decision points? MW: All that phrase was intended to convey is that these are the topics that the Sub Team discussed on the call for which the Analysis Group will likely need to either clarify or revise certain questions and their sequencing, or for which Sub Team members may agree that more specific clarity is needed. The thought was that a summary may be helpful as there seemed to be some confusion and lack of clarity as to what was presented by the Analysis Group and why certain questions were either included or phrased in a certain way or sequence. What you have presented below does not completely track with what the AG presented or my understanding of what's under discussion: For point one, "it may be helpful to have a section...." is inconsistent with the questions with which we were presented, where the two groups (which will actually be constituted by their responses to the first few questions of the survey) got a number of the same questions. Michael apparently doesn't want to survey potential registrants, but that's a very different suggestion from completely rewriting the survey tracks. MW: This was, again, an attempt to summarize the main point under discussion on this topic. It was not a staff attempt to predict or suggest what the final decision as between the Sub Team and the Analysis Group will be. As you noted, Michael had a specific question about doing two surveys which Kristine answered as part of the subsequent email discussion. There was no attempt by staff to rewrite anything or to suggest a different type or number of surveys, we are merely trying to provide a quick summary/refresher to facilitate the next call and discussion. For point two "possibly even necessary"--our mandate is, among other things, to figure out if the Claims Notice is understood by its recipients. I can't see any scenario where "possibly" is in the picture: it will absolutely be necessary to let people read the thing we're asking if they understand, or understood if they saw it in the past. (Among other things, a person who says they got a Claims Notice and then realizes that they don't recognize what they're seeing should not be counted as someone who got a Claims Notice.) I didn't understand anyone on the call to suggest that we shouldn't show the Claims Notice and I'm surprised that's entered the conversation. MW: Again, this is an attempt to summarize what the discussion was about, and to ensure that there is a clear understanding among the Sub Team and the Analysis Group that the Claims Notice must be shown as part of the survey process. I strongly disagree with your characterization of point three; we are apparently debating what it means to "understand what a Claims Notice is." Knowing what it looks like is one thing; understanding what they think it means is one of our goals in the WG. I agree with Kristine: we are charged with figuring out how recipients understand it, which can only be answered by asking what they think it means. Perhaps this should be reframed as "answering the relevant charter questions about the intelligibility of the Claims Notice." MW: I apologize if the choice of summary language was imprecise or gave the wrong impression. Again, this was a quick summary of the points that were discussed. It was not a suggestion about framing or phrasing the specific survey questions – that is for the Analysis Group to do as the professional contractor retained for this purpose. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Mary Wong <mary.wong(a)icann.org> Sent: Friday, June 15, 2018 10:33 PM To: Michael Graham (ELCA); Dorrain, Kristine; Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org Subject: Surveying potential registrants (Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018) Dear all, Thank you to Rebecca, Kristine and Michael for continuing the discussion. At the end of the last sub team call, the following appeared to be the decision points under consideration for the actual and potential registrant surveys: * The “actual registrant” and “potential registrant” surveys are two separate surveys; however, for the “actual registrant” target group, it may be helpful to also have a section that mirrors part of the “potential registrant” survey, so as to capture the views of those who succeeded in registering one/some domain(s) but decided not to proceed with other attempted registrations. * In both the “actual registrant” and “potential registrant” surveys, it will be important (possibly even necessary) to make clear that the respondent knows what a Claims Notice looks like – this may entail actually showing the respondent (during the survey?) a copy of the Claims Notice. * While “use cases” and illustrative examples using a hypothetical domain name can be considered but may not ultimately be necessary, what remains critical is eliciting responses where the respondent understands what a Claims Notice is and knows what it looks like. * Some reordering/reformatting of the questions may be necessary in order to present respondents with a logical process flow and to facilitate responses. If it will be of assistance, do you wish staff to request that Greg and/or Stacey provide a more specific description, on one of your next calls with them, as to what methods they and their subcontractor will be using to identify, target and reach the category of respondents that we are describing as “actual registrants” (i.e. those that succeeded in registering domain names especially in the new gTLDs) and “potential registrants” (i.e. those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future)? Thanks and cheers Mary, Julie, Ariel & Berry From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Michael Graham (ELCA)" <migraham(a)expedia.com> Date: Saturday, June 16, 2018 at 06:38 To: "Dorrain, Kristine" <dorraink(a)amazon.com>, "Tushnet, Rebecca" <rtushnet(a)law.harvard.edu>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks for taking the time to go through all this, Kristine – especially since my attendance today was somewhat sporadic. I do understand the scope of what we are trying to get, but I agree that we should ask specific questions about actual responses to the Claims Notice, and not ask hypotheticals. So, I can go along with asking registrants, applicants who did not obtain a registration, and potential domain name applicants what they understand the Claims Notice to say, what their reaction to it would be, etc. However, I think we need to ask the same question of registrants in two classes: Those who have received Trademark Claims Notices, and those who have not. More importantly, in asking, parsing, analyzing and attempting to draw any conclusions from the results, we must be sure that AG draws clear distinctions between the different groups being asked these questions, their answers, and the significance of their answers in light of their different levels and types of experience and concern. I am therefore agreeable, but extremely cautious in light of past surveys and analyses. Michael R. From: Dorrain, Kristine [mailto:dorraink@amazon.com] Sent: Friday, June 15, 2018 2:49 PM To: Michael Graham (ELCA) <migraham(a)expedia.com>; Tushnet, Rebecca <rtushnet(a)law.harvard.edu>; Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Hi Michael, I do understand your concern but I think we ARE providing the Analysis Group with clear instructions about who to survey and how to ask the questions – and they’ve told us how they will get the participants. That’s the point of the calls and that’s indeed what we’re discussing here, now. Correct me if I’m misunderstanding, but I take your suggestion to mean you oppose surveying potential registrants at all. I don’t think we are at liberty to do that and I’ll explain: 1. We have charter questions that ask how understandable the Claims Notice is and we have some (at least anecdotal, abandonment) information that people may have been scared away by the Claims Notice. 2. As a result, we are expressly asked by the full WG to figure out if the Claims Notice is scary or needs to be re-worded. 3. We are also asked to figure out how people from different places of the world perceive the Claims Notice, as presented and worded (ideally, in a variety of languages- I have not seen how AG plans to address this) ^^These, as I understand it, are not negotiable. These are the pieces of information the WG asked to get survey data on. Therefore, we MUST present the Claims Notice to some people (ideally, people who would actually register a domain name) to learn how people react to it. Now, we can for sure get some of this information from people who succeeded in registering domain names – they completed the process. But we are also looking to get this information from people who opted not to proceed for whatever reason, when presented with a claims notice. These are “potential registrants.” And yes, for people who actually didn’t try yet or didn’t proceed, these questions are somewhat hypothetical, but what we’re attempting to get is a REACTION and an idea of how it might affect future behavior, not past data. Think of this part as market research, not historical data. The “hypothetical” disagreement is just in the level detail, as I understand it. I’ve been an outspoken critic of hypothetical questions. My solution is to give some general use cases (you want a domain name for a blog or for your business or as an investment) and then ask if the Claims Notice would be scary or a deterrent and why. Rebecca likes the idea of being more specific (providing an example name at least, maybe more of a detailed scenario). If I’ve misunderstood, I hope you’ll set me straight. Best, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Michael Graham (ELCA) Sent: Friday, June 15, 2018 2:42 PM To: Tushnet, Rebecca <rtushnet(a)law.harvard.edu<mailto:rtushnet@law.harvard.edu>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Understood. However, this sort of research requires a clear identification of the universe to be studied, the wording that will be used, and the environment in which it will be asked. I think we need some clear parameters and questions, and I don’t think we have those yet. Hopefully after our call today, AG can provide some guidance. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:27 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 As I said on the call, this is standard comprehension research, relied on by courts and marketing departments. When you want to know what members of a group take away from a message, you show it to them and ask them about it, even if they haven't seen it before. It is precisely the kind of data driven research this working group is supposed to conduct. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 5:12:17 PM To: Tushnet, Rebecca; Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: RE: Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Rebecca. From your explanation, then, we do not want to ask hypothetical questions, but want to ask what subjects’ understanding of the Claims Notice is? That makes sense if asked of actual registrants or applicants who received the notices. I don’t think it makes sense, however, to ask this question of those who have not gone through the process. Then it becomes a hypothetical: If you received this notice, what does it tell you? Without the context of involvement in the process I don’t think “potential registrants” would provide relevant or useful information. I’ll go back to the questions after the weekend to see how and who they address the issue of effectiveness of the actual Claims Notice to. That will better answer my concerns I hope. Michael R. From: Tushnet, Rebecca [mailto:rtushnet@law.harvard.edu] Sent: Friday, June 15, 2018 2:01 PM To: Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>>; Ariel Liang <ariel.liang(a)icann.org<mailto:ariel.liang@icann.org>>; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: Actions & Notes: RPM Data Sub Team 15 June 2018 Surveying potential registrants (defined in the survey as those who either attempted but did not complete a new gTLD registration, or would consider a new gTLD registration in the future) for their understanding of the meaning of the claims notice is not hypothetical--it is a standard reception survey of the target audience. The question is not generally what issues kept some potential registrants away--that is a good deal broader than our task, I believe. The question is what people considering new gTLD registrations take away from the Claims Notice when they read it. That is one of the key questions this working group has about the functioning of the Claims Notice. Past and future new gTLD registrants are our universe. Depending on their past experiences, including receipt of claims notices (if we get any such respondents) we may have different questions for different subgroups. But they are the target group as a whole, and should get the same questions, except for those that don't apply to them. Rebecca Tushnet Frank Stanton Professor of First Amendment Law, Harvard Law School 703 593 6759 ________________________________ From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org<mailto:gnso-rpm-data-bounces@icann.org>> on behalf of Michael Graham (ELCA) <migraham(a)expedia.com<mailto:migraham@expedia.com>> Sent: Friday, June 15, 2018 4:50:32 PM To: Ariel Liang; gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: Re: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Thanks, Ariel. I will review the Tape (as they say), but believe I disagree with the last point unless the meaning is that we would send a single survey to both registrants and “potential registrants” (although I would like to understand better how this last universe will be identified/selected) but according to which group they fall into, they will follow separate branches – registrants being asked whether they received Claims Notices and what they did/how they reacted; and potential registrants being asked why they did not apply for registration – i.e. what hypothetical issues could arise that kept them away. I do not think we should mix them or that we should ask purely hypothetical questions of anyone since these will only obtain opinions, beliefs, and perceptions – not what the effects of the RPMs actually have been. Michael R. From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: Friday, June 15, 2018 11:56 AM To: gnso-rpm-data(a)icann.org<mailto:gnso-rpm-data@icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM * Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 * 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years * If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b * Change 3 years to 5 years. Q2c * Change 3 years to 5 years Q2e * "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. * We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. * Suggestions for options to be included in a multiple-choice question - I would not be sued if I continued - I would not be subject to an action to take the domain if I continued - No one else had a legal right to the name - I had legitimate or legal right to the domain - It does not seem like too much trouble to continue - I consulted with an attorney and it was fine - I was committed to the domain name for person/business reasons - It matched investments names I already own in a portfolio (going with "theme" registrants) - I didn't understand the notice - Something else [explain] * There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. * Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. * Registrants with big portfolio might actually prefer a narrative option. Q2g * There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) * The structure is a bit confusing * Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not * Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) * Re virtue.food - oppose to specify as people may give a false hypothetical * May want to specify "I plan to start a business or already have a business under that name" * If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. * We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. * We did give examples on other surveys to target non-expert. * How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? * We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: - Would you be nervous to register the domain name? -- provide options - If yes, what actions would you take? -- provide options - If no, what is the reason? -- provide options - Does trademark law matter? * Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. * Don't need to limit this question for the people who have not completed the registration. * Proposed new order: - Show trademark claims notice. - Q2g(iv) – confidence? - Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] - Q2h(i) – Why you abandoned? - Q2g(iii) – trademark law knowledge * We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... * One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. * Those general Qs should appear after the more specific responses to virtue.food Q2h(i) * We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? * This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions * Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. * Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. * Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. * There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions * Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions * Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. * If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? * I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. * I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 * The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. * Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. * We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. * The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] * AG suggestion: - Keep the hypo question for the potential registrant survey - In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? * Is it possible that most people taking the survey are legacy domain name registrants? * We account for those potential respondents and try to filter them in the survey. * It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

1 0

Recordings, attendance & AC chat from the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Friday, 15 June 2018
by Michelle DeSmyter June 16, 2018

June 16, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Friday, 15 June 2018 at 16:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/ZYMpBQ MP3: https://audio.icann.org/gnso/gnso-review-rpms-pdp-sub-team-data-15jun18-en.… AC Recording: https://participate.icann.org/p7igmayvcel/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar [gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Michelle

1 0

Actions & Notes: RPM Data Sub Team 15 June 2018
by Ariel Liang June 15, 2018

June 15, 2018

Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM * Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 * 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years * If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b * Change 3 years to 5 years. Q2c * Change 3 years to 5 years Q2e * "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. * We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. * Suggestions for options to be included in a multiple-choice question * I would not be sued if I continued * I would not be subject to an action to take the domain if I continued * No one else had a legal right to the name * I had legitimate or legal right to the domain * It does not seem like too much trouble to continue * I consulted with an attorney and it was fine * I was committed to the domain name for person/business reasons * It matched investments names I already own in a portfolio (going with "theme" registrants) * I didn't understand the notice * Something else [explain] * There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. * Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. * Registrants with big portfolio might actually prefer a narrative option. Q2g * There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) * The structure is a bit confusing * Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not * Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) * Re virtue.food - oppose to specify as people may give a false hypothetical * May want to specify "I plan to start a business or already have a business under that name" * If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. * We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. * We did give examples on other surveys to target non-expert. * How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? * We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: * Would you be nervous to register the domain name? -- provide options * If yes, what actions would you take? -- provide options * If no, what is the reason? -- provide options * Does trademark law matter? * Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. * Don't need to limit this question for the people who have not completed the registration. * Proposed new order: * Show trademark claims notice. * Q2g(iv) – confidence? * Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] * Q2h(i) – Why you abandoned? * Q2g(iii) – trademark law knowledge * We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... * One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. * Those general Qs should appear after the more specific responses to virtue.food Q2h(i) * We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? * This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions * Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. * Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. * Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. * There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions * Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions * Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. * If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? * I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. * I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 * The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. * Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. * We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. * The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] * AG suggestion: * Keep the hypo question for the potential registrant survey * In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? * Is it possible that most people taking the survey are legacy domain name registrants? * We account for those potential respondents and try to filter them in the survey. * It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

4 7

RPM Data Sub Team Meeting Options: Monday, 18 June
by Julie Hedlund June 15, 2018

June 15, 2018

Hello everyone, As we noted on the call and per the notes below, several Sub Team members are not available at the suggested time of 17:00-18:30 UTC on Monday, 18 June, and for others the time is not ideal. Please let us know at your earliest convenience if you would be able to attend the call if it is moved to the any of the following options: OPTION 1: 1 hour earlier, to 16:00-17:30 UTC at 90 minutes OPTION 2: 1 hour earlier to 16:00-17:00 UTC at 60 minutes OPTION 3: 2 hours earlier to 15:00-16:30 UTC at 90 minutes Thank you very much for your assistance and please be on the lookout for a meeting notice on Sunday, or very early Monday. We hope you have a nice weekend! Best, Mary, Ariel, Berry, and Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of Ariel Liang <ariel.liang(a)icann.org> Date: Friday, June 15, 2018 at 2:56 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ [community.icann.org] Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b Change 3 years to 5 years. Q2c Change 3 years to 5 years Q2e "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. Suggestions for options to be included in a multiple-choice question I would not be sued if I continued I would not be subject to an action to take the domain if I continued No one else had a legal right to the name I had legitimate or legal right to the domain It does not seem like too much trouble to continue I consulted with an attorney and it was fine I was committed to the domain name for person/business reasons It matched investments names I already own in a portfolio (going with "theme" registrants) I didn't understand the notice Something else [explain] There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. Registrants with big portfolio might actually prefer a narrative option. Q2g There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) The structure is a bit confusing Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) Re virtue.food - oppose to specify as people may give a false hypothetical May want to specify "I plan to start a business or already have a business under that name" If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. We did give examples on other surveys to target non-expert. How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: Would you be nervous to register the domain name? -- provide options If yes, what actions would you take? -- provide options If no, what is the reason? -- provide options Does trademark law matter? Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. Don't need to limit this question for the people who have not completed the registration. Proposed new order: Show trademark claims notice. Q2g(iv) – confidence? Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] Q2h(i) – Why you abandoned? Q2g(iii) – trademark law knowledge We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. Those general Qs should appear after the more specific responses to virtue.food Q2h(i) We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] AG suggestion: Keep the hypo question for the potential registrant survey In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? Is it possible that most people taking the survey are legacy domain name registrants? We account for those potential respondents and try to filter them in the survey. It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

3 2

Re: [Gnso-rpm-data] [Ext] RE: RPM Data Sub Team Meeting Options: Monday, 18 June
by Julie Hedlund June 15, 2018

June 15, 2018

Thanks Phil! Best, Julie From: "Corwin, Philip" <pcorwin(a)verisign.com> Date: Friday, June 15, 2018 at 3:30 PM To: Julie Hedlund <julie.hedlund(a)icann.org>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Ext] RE: RPM Data Sub Team Meeting Options: Monday, 18 June Option 2 would work for me Philip S. Corwin Policy Counsel VeriSign, Inc. 12061 Bluemont Way Reston, VA 20190 703-948-4648/Direct 571-342-7489/Cell "Luck is the residue of design" -- Branch Rickey From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Julie Hedlund Sent: Friday, June 15, 2018 3:03 PM To: Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: [EXTERNAL] [Gnso-rpm-data] RPM Data Sub Team Meeting Options: Monday, 18 June Importance: High Hello everyone, As we noted on the call and per the notes below, several Sub Team members are not available at the suggested time of 17:00-18:30 UTC on Monday, 18 June, and for others the time is not ideal. Please let us know at your earliest convenience if you would be able to attend the call if it is moved to the any of the following options: OPTION 1: 1 hour earlier, to 16:00-17:30 UTC at 90 minutes OPTION 2: 1 hour earlier to 16:00-17:00 UTC at 60 minutes OPTION 3: 2 hours earlier to 15:00-16:30 UTC at 90 minutes Thank you very much for your assistance and please be on the lookout for a meeting notice on Sunday, or very early Monday. We hope you have a nice weekend! Best, Mary, Ariel, Berry, and Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of Ariel Liang <ariel.liang(a)icann.org> Date: Friday, June 15, 2018 at 2:56 PM To: "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: [Gnso-rpm-data] Actions & Notes: RPM Data Sub Team 15 June 2018 Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 15 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/ZYMpBQ [community.icann.org] Have a great weekend! Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEM Staff to ask Sub Team members’ availability to join the Monday, 18 June meeting at 17:00 UTC; if low availability, staff to propose an alternative time. (Done) NOTES Q2 2nd option: the first new gTLD was launched in July 2013, so change 3 years to 5 years If the following questions do not apply to the registrants, redirect them to the potential registrant survey section. Q2b Change 3 years to 5 years. Q2c Change 3 years to 5 years Q2e "If Yes, did you register the domain name for which you received a Claims Notice?" is a core question. We want to know why they went ahead despite receiving a Claims Notice. We want to know qualitatively why they did so. It goes with the same set of questions below. Same rationale may apply to the second time they receive the Claims Notice, or more. We spent a lot of time asking why people did NOT proceed, and there could be a lot of reasons. We do have a set of questions asking people why they DO proceed. If they did so, they may have made some serious decisions here, and we need to know why. Perhaps provide some multiple-choice options listing possible reasons why they proceed in order to dive deeper. Suggestions for options to be included in a multiple-choice question I would not be sued if I continued I would not be subject to an action to take the domain if I continued No one else had a legal right to the name I had legitimate or legal right to the domain It does not seem like too much trouble to continue I consulted with an attorney and it was fine I was committed to the domain name for person/business reasons It matched investments names I already own in a portfolio (going with "theme" registrants) I didn't understand the notice Something else [explain] There are portfolio registrants and they may have received many notices. May ask a quantitative question, "How many times this has happened to you?" and let them choose via multiple-choice options. Follow up with a question "For any of the domain name that you decide to proceed with registration, check all the reasons that apply". Bearing in mind that some people could have big portfolios. Is there anyway to encourage the survey taker to write a narrative and explain themselves? If we do get the response, it would be very helpful. Registrants with big portfolio might actually prefer a narrative option. Q2g There is an agreement to funnel this type of questions into potential registrant survey. Q2g(i) The structure is a bit confusing Need to be clear that this question should apply to anyone who is presented a Claims Notice, regardless whether he/she completed the registration or not Which of the following best describes your understanding of the purpose of the Claims Notice? Don’t know/Not sure -- we need a "what a heck" option there too Q2g(ii) – Q2h(i) Re virtue.food - oppose to specify as people may give a false hypothetical May want to specify "I plan to start a business or already have a business under that name" If we specify the business aspect, how would that help us? We don't know the background of the registrant why they proceed or not proceed. We don't know what the registrants are thinking, so we are asking them, at least we will get some insight and useful answers if we give them an example to think about. If we just give them a notice, they would not know what to do with that. We did give examples on other surveys to target non-expert. How do we know there is no brand using virtue.food. How do we know survey takers’ level of knowledge whether it would be an issue or not? We can ask clarifying questions at the end, but ask examples first to provide some context. You may need to place the “demographic” type of questions first to get into the mindset of the survey takers, and we can get a sense of the trend: Would you be nervous to register the domain name? -- provide options If yes, what actions would you take? -- provide options If no, what is the reason? -- provide options Does trademark law matter? Concerned if asking those questions first may have a bad effect on the survey takers, e.g., asking them about trademark law knowledge. It may be super hard to ask each survey taker to consider each scenario. Worth asking trademark only after asking some hypothetical questions, otherwise it would trigger some negative reaction. Don't need to limit this question for the people who have not completed the registration. Proposed new order: Show trademark claims notice. Q2g(iv) – confidence? Would you consult with an attorney? [maybe also “would you talk with domain name registrar, etc?] Q2h(i) – Why you abandoned? Q2g(iii) – trademark law knowledge We are making assumption that the user is going to assume the claims has something to do with the trademark. Most people don't know the difference between trademark and copyright. That's a mistake in the premise of each Q that the analysis group needs to resolve. Maybe this is the FIRST question and there is a matrix. so different paths of questioning need to be clarified... One possibility is to ask registrants what they imagine their use would be. If we will have enough respondents, we could sub group them. Those general Qs should appear after the more specific responses to virtue.food Q2h(i) We are asking why they did not proceed, and it is not with respect to virtue.food. Where are we asking them what they are doing with virtue.food? This question is for people who have abandoned. Everyone should be asked for this question. It is doubtful that we will get a lot of registrant that receives claims notice. Discussion about the open-ended questions Narrative responses can become cumbersome. It is fine to have a few in the survey, but the more complicated question is, the more loaded the responses would be. To have a lot of these questions would make it difficult for the respondent to respond, and can lower response rate and increase dropout rate. Some open text questions may be better for interview survey. Form survey would not have the opportunity to redirect the answer. Wide range of responses in open text field can make the analysis of the responses difficult and it would be hard to summarize the results in a simple way. There is difference between "Other___" and the larger open text field asking very open-ended question. Discussion about hypothetical questions Reference page 28-29 of the RFP Appendix regarding the comments for/against hypothetical questions Getting into use cases is different from hypothetical. There are general categories that people fall into. Give people the general categories. If we ask hypothetical questions of Registrants, I would like to return to TM/Brand owner survey to include some hypos. What would be the difference? I feel like that is a different kind of request for information. That would be something to ask in the design phase of a project not necessarily an evaluation phase. I'm ok with the page 29 scenario since we get an understanding of what the respondent is thinking when they answer. Consider the alternative to hypothetical listed on page 29 The questions about actual events and actions will provide metrics; hypos will prove opinions. We must somehow separate the two if we include them. Why the hypothetical question falls into the registrant survey? Q2g, Q2g(i), Q2g(ii) are inspired by the potential registrant survey questions. We want to know the set of people who register domain name, and when they see the trademark claims notices, what they are doing with them. To provide use cases is because it is hard for registrant to understand in the abstract. There is no way to ask such question without something to think about. The only way to get an answer to the question is by showing claims notice and see what they think. If you got a claims notice.... [Qs] AND here is a claims notice... [ Q] AG suggestion: Keep the hypo question for the potential registrant survey In the registrant survey, ask respondents did they receive claims survey, what they did to it, what was the purpose of their registration, etc. elicit the insight without providing a hypothetical scenario? Is it possible that most people taking the survey are legacy domain name registrants? We account for those potential respondents and try to filter them in the survey. It does not make a lot of sense to differentiate the registrants and potential registrants in two different surveys. There are a lot of overlapping paths. It should be one survey with branching paths. The people who said “I have received a claims notice, but didn't have the use case” should be given the opportunity to provide useful responses.

3 3

REMINDER: Proposed Agenda and Documents: Meeting 15 June at 1600 UTC
by Julie Hedlund June 15, 2018

June 15, 2018

Dear Data Sub Team Members, In preparation for today’s (Friday) meeting (15 June) at 16:00 UTC for 90 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/18K-zhP541IGCR0bXJpBtdmWdpfQ6BbKhFZaaZsf… 3. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 4. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

1 0

Proposed Agenda and Documents: Meeting 15 June at 1600 UTC
by Julie Hedlund June 13, 2018

June 13, 2018

Dear Data Sub Team Members, In preparation for Friday’s meeting (15 June) at 16:00 UTC for 90 minutes, please see the proposed agenda and Google docs: 1. Review agenda 2. Discussion of the proposed Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/18K-zhP541IGCR0bXJpBtdmWdpfQ6BbKhFZaaZsf… 3. Discussion of the proposed Potential Registrant survey questions and suggested changes by Analysis Group: https://docs.google.com/document/d/1iY4KsWpioC8Sacs-pWurfnKxhOsiQA7YaSo1wux… 4. AOB Please also see attached the RFP appendix that includes the original questions as developed by the Sub Team, which is also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry

1 0

Recordings, attendance & AC chat from Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Wednesday, 13 June 2018
by Julie Bisland June 13, 2018

June 13, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Wednesday, 13 June 2018 at 16:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/YoMpBQ MP3: https://audio.icann.org/gnso/gnso-rpm-review-data-13jun18-en.mp3 AC Recording: https://participate.icann.org/p1zyhuplbun/<https://participate.icann.org/p1zyhuplbun/?OWASP_CSRFTOKEN=a3029c3a5e9ca4be…> The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar [gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Julie

1 0

Actions & Notes: RPM Data Sub Team 13 June 2018
by Ariel Liang June 13, 2018

June 13, 2018

Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 13 June 2018 (1600 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/YoMpBQ Please note that the staff action items have been completed (see below). Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEMS * Staff to check which Q4 sub question(s) address the same intent as Q12. (DONE) * Based on notes, Q4m seems to have similar intent as Q12. The Sub Team want to know the benefits and disadvantages if registries provider either the Claims Period or Sunrise Period, instead of both. AG would word this question similarly to Q4k. * Staff to check the notes related to the grid questions and see what the conclusion/outcome was. (DONE) * Based on notes, Sub Team suggested that similar grid questions asked to ROs should be asked to Registrars. The grid questions should also leave some open-ended options for respondents to provide narrative responses. The grid questions in the RO survey are Q16/Q17, as well as Q4i/Q4j in the registrar survey. AG would like to request the Sub Team to provide suggestions and include additional options to Q16/Q17 to the Google Doc. AG could maintain the grid format of the current questions, but to also include a separate question to elicit anecdotal responses. * Staff to circulate Berry’s information about the domain name registration process. (DONE) * Information from Berry: Registrant enters the domain at the Registrar where it is queried as to whether its available for registration. If the potential Registrant places the domain in the cart with intent of registering it, the Registry will also check if it appears on the Domain Names List (DNL). If it does, the Registry sends a CNIS code to the Registrar that notifies the Registrant of the claims notice. If registered, the Registrar returns a registration confirmation to the TMCH. This is reference to the TMCH Technical Specification as documented on IETF. * Kristine to review the grid questions (Q16/Q17 in RO survey, Q4i/Q4j and Q13/Q14 in registrar survey) and propose ideas to revise the grid. NOTES Registrar Survey Q12 * A sub question in Q4 captures the intent of this one, asking if there should be either sunrise or claims rather than both. Move that sub question down and replace Q12 with it? * ACTION ITEM: Staff to check which Q4 sub question(s) address the same intent. * Staff Comment: Based on notes, Q4m seems to have similar intent as Q12. The Sub Team want to know the benefits and disadvantages if registries provider either the Claims Period or Sunrise Period, instead of both. AG would word this question similarly to Q4k. Q13/Q14 * There are similar questions discussed in the Registry survey. Do these questions capture the original intent? * The grid format would need to be extrapolated across all the surveys. * ACTION ITEM: Staff to check the notes related to the grid questions and see what the conclusion/outcome was. * Staff Comment: Based on notes, Sub Team suggested that similar grid questions asked to ROs should be asked to Registrars. The grid questions should also leave some open-ended options for respondents to provide narrative responses. The grid questions in the RO survey are Q16/Q17, as well as Q4i/Q4j in the registrar survey. AG would like to request the Sub Team to provide suggestions and include additional options to Q16/Q17 to the Google Doc. AG could maintain the grid format of the current questions, but to also include a separate question to elicit anecdotal responses. * ACTION ITEM: Kristine to review the grid questions (Q16/Q17 in RO survey, Q4i/Q4j and Q13/Q14 in registrar survey) and propose ideas to revise the grid. Q15 * The original questions are to understand whether there is any impact to abandonment rate when the registrant receives claims notice. The original questions refer to the problems related to pre ordering domains. * Original questions are (page 10 – RFP Appendix): * Do you have any records of the “abandonment rate” (i.e., domain name applicants who request the registration of a particular domain but do not go through to complete the payment)? * If so, what are the rates of abandonment for legacy TLDs and ccTLDs? * What is the abandonment rate for a New gTLD during the Claims period – both for names which receive Claims Notices and those which do not? And after the Claims period? * We are trying to understand whether the claims notice is only triggered when someone is actually trying to pay for the domain. * Rephrase the question to: At what point during the purchase process is the claims notice downloaded? Use multiple choice or dropdown menu to provide options. * During Sunrise registration, the record of trademark would be downloaded. Claims notice would not trigger the download of anything. * Information from Berry: Registrant enters the domain at the Registrar where it is queried as to whether its available for registration. If the potential Registrant places the domain in the cart with intent of registering it, the Registry will also check if it appears on the Domain Names List (DNL). If it does, the Registry sends a CNIS code to the Registrar that notifies the Registrant of the claims notice. If registered, the Registrar returns a registration confirmation to the TMCH. This is reference to the TMCH Technical Specification as documented on IETF. * ACTION ITEM: Staff to circulate Berry’s information about the domain name registration process. * The registrar would not be downloading any trademark record, so the question is invalid. Eliminate Q15. * This question may be completely redundant. The only caveat is that we need to leave the door open for someone else to explore an alternative business model. If you code in too much in the policy/procedures, then you don't leave much room for expansion. * What is the most beneficial point? Or lease beneficial point? Why it matters when the claims notice is generated? Don't think we can get to "beneficial" because different models will answer differently. * Rephrase the question to: At what point of the registration process do you typically query whether the domain name is registered in the TMCH? * We are trying to understand the theoretical abandonment rate. Whether the query to the TMCH would trigger the generation of claims notice. We want to get at the typical process with regard to abandonment. We want to get at the functional problems. If there is standard process, registrants themselves are the ones that query. Q16 * Q16 is lumped together with Q15 * The sequence of event is not coming through clearly. This question should go before Q15. General Comment * Put all the claims related questions in one section and it would flow better. * There are no questions about “abandonment” in this survey at all. In the RFP Appendix, questions on page 10 to the first section of page 11 are to understand the abandonment rate and how abandonment happens. These questions also aim to elicit anecdotes. * AG asked similar questions to elicit data regarding abandonment rate during the TMCH review, but the respondents did not provide answers, hence they suggested to remove these questions. * If we aren't going to get abandonment data, could we ask people yes/no if they have/can/want to provide it (to keep the survey short for people who won't share this info or don't have it)? * AG agreed to provide respondents option if they want to do a deeper dive in the survey about abandonment rate. Those who are motivated and interested can contribute without turning others off.

1 0