- Gnso-rpm-data - lists.icann.org

Actions & Notes: RPM Data Sub Team 06 June 2018
by Ariel Liang June 6, 2018

June 6, 2018

Dear All, Please see below the action items and notes captured by staff from the Data Sub Team call held on 06 June 2018 (1700 UTC). Staff have posted to the wiki space the action items and notes. Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording. The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/FysFBQ Best Regards, Ariel Ariel Xinyue Liang Policy Analyst | Washington, DC Internet Corporation for Assigned Names and Numbers (ICANN) ---------------- ACTION ITEMS For Sub Team: * Sub Team to provide comments/suggestions/questions on the draft surveys via Google docs to avoid the confusion that would result from multiple redlined documents. * Sub Team to discuss internally about Q12 and see whether specific additional questions/clarifications are needed. * Sub Team to suggest to AG additional options associated with Q12. * Sub Team to reference the elements in the original questions associated with Q16/17 and see whether/how to revise the wording. For AG: * AG to change 60 days to 30 days. For Staff: * Staff to extend the meeting on 07 June at 1600 UTC by 30 minutes to 120 minutes total (done – calendar invite updated and sent) * Staff to schedule a call on 08 June at 1600 UTC for 120 minutes (ongoing) * Staff to schedule a contingency call on Monday 11 June at 1600 UTC for 120 minutes NOTES Registry Operator Survey * General Comments: * Analysis Group (AG) made high level changes to reduce the number of open ended questions, remove leading questions, and simplify the language. Some original questions were removed. These high-level changes also apply to other surveys * Sub Team members Strongly favor allowing respondents to either not answer questions, or to indicate "Not Answering" in some gentle way. There is interest to aggregate the raw data, but some ROs may want to keep their survey responses confidential. Make it clear if the respondent skips certain questions, we still count their responses as valid. All questions will provide the option for the respondents to skip. * There should be one questionnaire going to one company, and we should avoid one RO sending back multiple responses if they manage multiple TLDs. * The current survey is quite long, so AG suggests dividing the survey into two parts, asking some ROs to respond one set, the other ROs to respond the other set. This approach may generate more responses in total. Sub Team feels that while this approach may be welcomed by the respondents, there may not be a big enough pool of ROs to do this. How valid the responses would be based on this approach? Sub Team don't want to send separate surveys to ROs but can parse the data once they are collected. * Sub Team would want to be able to quote responses that are text based. * Q1 * The question should ask the name of the respondent’s company/business? * This question can be removed; the intent is to it know the respondent. If this question remains, it should be optional. * Can we make the Q1 mandatory if the RO operates more than one TLD, so they can identify themselves? It may be useful to include such question. * The survey could ask ROs approximately how many TLDs that they operate, to provide more insight into the RO's operation. Or could provide a drop down, so large/medium/small ROs could identify themselves. * This question depends on how the RO selection is done. If specific ROs are invited to respond, it does not matter too much if they identify or not identify themselves. If a bulk invitation is sent to the RySG, for example, it would become more challenging, and the instruction needs to clarify that a RO only should complete one survey. * Staff believes that all registry operators will be asked to fill out the survey, i.e. no particular solicitation or invitation to some only * Q3/Q4 * This question may be reworded. Among the TLDs that you operate, what percentage approximately are community related, geo related, restricted by edibility terms? The intent of the question is to ask respondents to identify these specific TLDs that people have concerns with. * Perhaps lump in the IDN TLDs for the further consolidation of the questions. * Q7-Q9 * AG suggested to take out some of the pricing related questions to shorten the survey. Based on previous experience, such questions would be hard to answer by the ROs. Asking these questions in a survey instrument would be hard to illicit responses and may lead to higher dropout rate for the survey. * These are things (on premium pricing) that the Sub Team need to know. AG is willing to add questions back, with the caveat that it would raise potential issues of higher drop off rate. * Q7 * ROs do not know the names reserved in the TMCH. AG will revise the language. * Q8 * This question may potentially be removed if other pricing related questions are added in. * Q8a * This question may also be removed if other pricing related questions are added in. * Q9 * It has the same issue like Q7, as the RO has to establish premium pricing before launches. The RO does not know what is in the TMCH and wouldn't know whether they are eligible for Sunrise. ROs would not be able to answer this question. * The original question in the data table is a part of a series of questions. The Sub Team want to know whether the ROs consciously try to avoid situation where they price brand names as premium names. There are some allegations that ROs 'targeted' brand names. So the series questions aim to set the stage about sunrise pricing/premium pricing, and then lead to this question. * AG will redraft this question based on Sub Team input. * Q10 * This question should be placed before Q9, or merge with Q9. * Q11 * As some ROs read through the survey, they might get confused this question with the one related geo/community TLDs (e.g., Q22). Make it clear that they are not the same questions. * AG will consider merging Q11 and Q22 as they are related. * Q12 * It is a technical matter and want to make sure ROs understand the language when taking the survey. * The word "required" -- clarify that it is an already existing policy * References to the Spec may not be necessary. * ACTION ITEM: Sub Team to discuss internally about Q12 and see whether specific additional questions/clarifications are needed. * ACTION ITEM: Sub Team to suggest to AG additional options associated with Q12. * Q14 * ROs would not know the names in TMCH. So would Q14 be unanswerable? * This question tries to address the gap related to the quasi perpetual claims period. * Not sure whether this question is clear enough for the respondent. But from RO perspective, this would not be a confusing question. If a similar question is asked from the brand owners or other parties, the wording needs to be clarified. * The RO does not need to know what is in the TMCH in order to comply with the spec. * Q15 * ACTION ITEM: AG to change 60 days to 30 days. * Q16/Q17 * AG suggested a few outcomes based on the intent of the original question. * This question needs to be clarified so the respondent knows why they are being asked. Some respondent might believe that Sunrise period can be unlimited. * Should AG suggest different lengths of Sunrise period, or ask respondents what ideal length would be? * As to the comparison of 30 day and more than 30 day charts -- how can we make this comparative? One could see answering all 30 days as GREAT and all More than 30 days as GREAT -- which tells us nothing. * Could we also ask the general question: Do you believe increasing the 30 day period would improve any of these results? How many days would you suggest? * Q16/17 do not seem to reflect the balance concerning all parties in the original question. * Should this question be deleted? There is a potential to turn the original open-ended questions into quantifiable questions. It may be helpful to come up with some alternative wordings for this question, as data is needed. Sub Team want the respondents to provide data, but also give feedback, so they would know what to do differently. AG should read deeper into the questions and see what they can pull from the responses. * ACTION ITEM: Sub Team to reference the elements in the original questions associated with Q16/17 and see whether/how to revise the wording. Procedure Matters * Sub Team would need to homework in order to provide targeted input for AG. * We cannot sacrifice the quality of surveys just to meet the timeline. This is not the place to cut the corner, so we may need to have more calls. Taking the time now to ensure the BEST survey and metrics will dramatically reduce the time spent "discussing" reality based on narrative, perception, and preconceptions. * The RFP / contract includes the timeline. There is flexibility, but the contract contemplates that Analysis Group's work will be more or less completed by August. * ACTION ITEM: * Staff to extend the meeting on 07 June at 1600 UTC by 30 minutes to 120 minutes total (done – calendar invite updated and sent) * Staff to schedule a call on 08 June at 1600 UTC for 120 minutes (ongoing) * Staff to schedule a contingency call on Monday 11 June at 1600 UTC for 120 minutes * Sub Team to provide comments/suggestions/questions on the draft surveys via Google docs to avoid the confusion that would result from multiple redlined documents.

1 0

Recording, Attendance & AC Chat / RPM Sub Team for Data Call / Wednesday, 06 June 2018 at 17:00 UTC
by Andrea Glandon June 6, 2018

June 6, 2018

Dear All, Please find the attendance and AC chat of the call attached to this email, and the MP3 and AC recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Wednesday, 06 June 2018 at 17:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/FysFBQ MP3: https://audio.icann.org/gnso/gnso-rpm-review-data-06jun18-en.mp3 <https://urldefense.proofpoint.com/v2/url?u=https-3A__audio.icann.org_gnso_g…> AC Recording: https://participate.icann.org/p5lozwx59jj/<https://participate.icann.org/p5lozwx59jj/?OWASP_CSRFTOKEN=b26ccfc1d7ce5fa0…> The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar [gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__gnso.icann.org_en_grou…> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> Thank you. Kind Regards, Andrea Glandon

1 0

Re: [Gnso-rpm-data] Please postpone today's call
by Julie Hedlund June 6, 2018

June 6, 2018

Hi Susan, Lori, and Kristine (and Sub Team members), Thank you for your request and perhaps staff can provide some further explanation for why we are pressed for time. But first, staff should clarify that it was not expected that the Sub Team members would have read through and analyzed all of the materials from Analysis Group by today’s meeting. Given that the deadline for Analysis Group to provide the draft surveys was to be 05 June, that timeline in any case would not have provided enough time for the Sub Team members to review all of the materials. Instead, it was the staff’s understanding that the purpose of the calls today and tomorrow was to review the draft surveys and address the comments and questions from Analysis Group during the call. Second, it is the timeline, which is part of the RFP, that is driving this work. According to the timeline the Sub Team must provide its feedback to Analysis Group by this Friday, in order for Analysis Group to provide the final surveys to the WG by next Wednesday, 13 June, and then for Analysis Group to be able to start sending the surveys by 20 June. If we slip even by a few days we will not be able to meet the deadline of getting the surveys out by the end of the month. If we don't get the surveys out by the end of the month (which is predicated on AG getting the feedback this week), we won't get the results back by August and a report by September at the latest to review - which pushes out the development of initial recommendations. We do realize the challenges in this process and staff will do everything possible to help the review of the draft surveys go as smoothly as possible. However, given that today’s call is already scheduled with many Sub Team members expecting to attend, staff would suggest that the Sub Team can use the call as noted above to review the draft surveys and work with Analysis Group to address the questions. The work can continue on tomorrow’s call if needed. Following both calls staff will send the comments to the Sub Team to review before they are sent to Analysis Group on Friday. Kind regards, Mary, Ariel, Berry, and Julie From: Gnso-rpm-data <gnso-rpm-data-bounces(a)icann.org> on behalf of "Dorrain, Kristine via Gnso-rpm-data" <gnso-rpm-data(a)icann.org> Reply-To: "Dorrain, Kristine" <dorraink(a)amazon.com> Date: Wednesday, June 6, 2018 at 11:05 AM To: Lori Schulman <lschulman(a)inta.org>, Susan Payne <susan.payne(a)valideus.com>, Ariel Liang <ariel.liang(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Subject: Re: [Gnso-rpm-data] Please postpone today's call Hi everyone, I also support postponing the call. As I am one of the west-coasters, I just came in to the email which I cannot get through before our scheduled call today, particularly in light of the RySG call this morning. I don’t think this time will be well-used if we’re looking at the docs together for the first time. Best, Kristine From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Lori Schulman Sent: Wednesday, June 06, 2018 7:37 AM To: Susan Payne <susan.payne(a)valideus.com>; Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: Re: [Gnso-rpm-data] Please postpone today's call Dear All, I echo Susan’s concerns. I will not be making the call this afternoon as I believe that we cannot possibly be prepared. It’s 10:30 am my time and I cannot get through this by 1 pm given my other priorities at the moment. I strongly believe that we need a “rethink” on how we will proceed and do plan on attending tomorrow’s call in which have held slot. Lori Lori S. Schulman Senior Director, Internet Policy International Trademark Association (INTA) +1-202-704-0408, Skype: lsschulman From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Susan Payne Sent: Wednesday, June 06, 2018 10:29 AM To: Ariel Liang <ariel.liang(a)icann.org>; gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] Please postpone today's call Importance: High All Apologies in advance for being blunt. I am not seeking to cast blame on anyone, but I am extremely concerned that this exercise is being pushed forward with such haste. It is completely unreasonable and unrealistic to expect us all to have read all these documents, considered the extensive redlines and comments, and be in a position to discuss this meaningfully in the space of 4 hours from them landing in our Inboxes. Half of our subgroup are on the US West Coast – they are only just waking up so they don’t even get that long! I may have the advantage of this being the middle of my working day, but I still have multiple other calls on my time this afternoon. The meeting later this afternoon was set when we were due to receive these documents on Tuesday. Frankly, even that was a ludicrously short turn-around time, but that’s academic, since we didn’t receive the documents on Tuesday (and just to be clear, I’m not blaming AG for missing the deadline – they also have been given a wholly unrealistic timeline). I know that there is pressure to move this survey exercise along. I’ve been vocal myself about concerns on the WG timeline, but those concerns have been as much about the need to be realistic about what tasks should be undertaken and how long they will take. The RFP exercise to appoint AG has taken months (which is no surprise to me). It’s unfair and false economy to try to make up the time by truncating the preparatory work on the actual surveys. It’s wholly unreasonable to allow only 2-3 weeks for AG to review the initial input from the Subteam, create surveys, discuss and finalise with the Subteam, identify survey respondents, report to the full WG, and get these surveys deployed. We seem to be working towards a completely arbitrary date for reporting to the full WG – what is the reason for this 13 June deadline? We aren’t discussing on a WG call before, or during Panama, so what’s the justification for this? Whilst I have strongly questioned the value of this whole exercise from the outset, I, and the others on this subgroup, have also made a commitment to do what we can to try to ensure that some useful information will be generated. Without adequate consideration and input into the surveys before they are launched we are setting this whole exercise up for failure, which will be a shameful waste of the time spent to date and ICANN’s financial resources. Susan Payne Head of Legal Policy | Valideus Ltd E: susan.payne(a)valideus.com D: +44 20 7421 8255 T: +44 20 7421 8299 M: +44 7971 661175 From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: 06 June 2018 14:00 To: gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] Proposed Sunrise & Claims Survey Questions Dear Data Sub Team Members, In preparation for today’s meeting at 17:00 UTC, please find attached the redline and clean versions of the proposed Sunrise & Claims survey questions from the Analysis Group. Kindly see below the text provided by the Analysis Group to accompany the files. Apologies for the short time window for circulating the proposed survey questions, and we hope you will have a chance to review them prior to the call. Please see the proposed agenda for today’s meeting below: 1. Review agenda 2. Discussion of the proposed survey questions and suggested changes by Analysis Group 3. AOB All files attached to this email are also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ [community.icann.org] Best Regards, Mary, Julie, Ariel, and Berry ---------- Text from Analysis Group -------- Dear All, Attached please find our suggested revisions to the survey questions for each target group of the RPM survey (and, for reference, the Word version of the appendix to the RFP). We copied the questions from the RFP appendix into a Word document to follow a questionnaire format and have suggested edits to the survey in redline. We have also attached a clean version of each revised document. It may be easiest to review the clean version of our documents next to the RFP appendix, as the versions with tracked changes can be more difficult to follow. Please note that there may be minor formatting inconsistencies across the documents, as well as some minor differences in similar questions. We will clean these up this week, but wanted to provide you with the revised questions as soon as possible. As we discussed last week, the goal of our proposed edits is to limit the length of the survey while still allowing the capture of as much information as possible. To the extent possible, we converted open-ended questions to multiple choice. In some instances, we would like to request assistance with defining the multiple choice answers. In those instances, we have placed the request in double square brackets and highlighted the text in yellow. For some "anecdotal" questions, we could not think of an easy or clear way to translate the questions into closed-end responses. In those cases, we will allow open-text responses (marked by "open text field" responses). We removed some questions from several of the surveys, but believe more should still be removed from both the registry and trademark holder surveys. We have noted this in the relevant Word documents, and are hoping you can assist us in identifying several questions in each survey to remove. Best, Stacey and Greg Stacey Chan, Ph.D. | Manager Analysis Group, Inc. | Economic, Financial, and Strategy Consulting 1900 16th Street, Suite 1100, Denver, CO 80202 720-963-5307 (direct) | 720-963-5300 (main) stacey.chan(a)analysisgroup.com l www.analysisgroup.com [analysisgroup.com]

3 2

Please postpone today's call
by Susan Payne June 6, 2018

June 6, 2018

All Apologies in advance for being blunt. I am not seeking to cast blame on anyone, but I am extremely concerned that this exercise is being pushed forward with such haste. It is completely unreasonable and unrealistic to expect us all to have read all these documents, considered the extensive redlines and comments, and be in a position to discuss this meaningfully in the space of 4 hours from them landing in our Inboxes. Half of our subgroup are on the US West Coast – they are only just waking up so they don’t even get that long! I may have the advantage of this being the middle of my working day, but I still have multiple other calls on my time this afternoon. The meeting later this afternoon was set when we were due to receive these documents on Tuesday. Frankly, even that was a ludicrously short turn-around time, but that’s academic, since we didn’t receive the documents on Tuesday (and just to be clear, I’m not blaming AG for missing the deadline – they also have been given a wholly unrealistic timeline). I know that there is pressure to move this survey exercise along. I’ve been vocal myself about concerns on the WG timeline, but those concerns have been as much about the need to be realistic about what tasks should be undertaken and how long they will take. The RFP exercise to appoint AG has taken months (which is no surprise to me). It’s unfair and false economy to try to make up the time by truncating the preparatory work on the actual surveys. It’s wholly unreasonable to allow only 2-3 weeks for AG to review the initial input from the Subteam, create surveys, discuss and finalise with the Subteam, identify survey respondents, report to the full WG, and get these surveys deployed. We seem to be working towards a completely arbitrary date for reporting to the full WG – what is the reason for this 13 June deadline? We aren’t discussing on a WG call before, or during Panama, so what’s the justification for this? Whilst I have strongly questioned the value of this whole exercise from the outset, I, and the others on this subgroup, have also made a commitment to do what we can to try to ensure that some useful information will be generated. Without adequate consideration and input into the surveys before they are launched we are setting this whole exercise up for failure, which will be a shameful waste of the time spent to date and ICANN’s financial resources. Susan Payne Head of Legal Policy | Valideus Ltd E: susan.payne(a)valideus.com<mailto:susan.payne@valideus.com> D: +44 20 7421 8255 T: +44 20 7421 8299 M: +44 7971 661175 From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Ariel Liang Sent: 06 June 2018 14:00 To: gnso-rpm-data(a)icann.org Subject: [Gnso-rpm-data] Proposed Sunrise & Claims Survey Questions Dear Data Sub Team Members, In preparation for today’s meeting at 17:00 UTC, please find attached the redline and clean versions of the proposed Sunrise & Claims survey questions from the Analysis Group. Kindly see below the text provided by the Analysis Group to accompany the files. Apologies for the short time window for circulating the proposed survey questions, and we hope you will have a chance to review them prior to the call. Please see the proposed agenda for today’s meeting below: 1. Review agenda 2. Discussion of the proposed survey questions and suggested changes by Analysis Group 3. AOB All files attached to this email are also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ Best Regards, Mary, Julie, Ariel, and Berry ---------- Text from Analysis Group -------- Dear All, Attached please find our suggested revisions to the survey questions for each target group of the RPM survey (and, for reference, the Word version of the appendix to the RFP). We copied the questions from the RFP appendix into a Word document to follow a questionnaire format and have suggested edits to the survey in redline. We have also attached a clean version of each revised document. It may be easiest to review the clean version of our documents next to the RFP appendix, as the versions with tracked changes can be more difficult to follow. Please note that there may be minor formatting inconsistencies across the documents, as well as some minor differences in similar questions. We will clean these up this week, but wanted to provide you with the revised questions as soon as possible. As we discussed last week, the goal of our proposed edits is to limit the length of the survey while still allowing the capture of as much information as possible. To the extent possible, we converted open-ended questions to multiple choice. In some instances, we would like to request assistance with defining the multiple choice answers. In those instances, we have placed the request in double square brackets and highlighted the text in yellow. For some "anecdotal" questions, we could not think of an easy or clear way to translate the questions into closed-end responses. In those cases, we will allow open-text responses (marked by "open text field" responses). We removed some questions from several of the surveys, but believe more should still be removed from both the registry and trademark holder surveys. We have noted this in the relevant Word documents, and are hoping you can assist us in identifying several questions in each survey to remove. Best, Stacey and Greg Stacey Chan, Ph.D. | Manager Analysis Group, Inc. | Economic, Financial, and Strategy Consulting 1900 16th Street, Suite 1100, Denver, CO 80202 720-963-5307 (direct) | 720-963-5300 (main) stacey.chan(a)analysisgroup.com<mailto:stacey.chan@analysisgroup.com> l www.analysisgroup.com<http://www.analysisgroup.com>

4 3

Proposed Sunrise & Claims Survey Questions
by Ariel Liang June 6, 2018

June 6, 2018

Dear Data Sub Team Members, In preparation for today’s meeting at 17:00 UTC, please find attached the redline and clean versions of the proposed Sunrise & Claims survey questions from the Analysis Group. Kindly see below the text provided by the Analysis Group to accompany the files. Apologies for the short time window for circulating the proposed survey questions, and we hope you will have a chance to review them prior to the call. Please see the proposed agenda for today’s meeting below: 1. Review agenda 2. Discussion of the proposed survey questions and suggested changes by Analysis Group 3. AOB All files attached to this email are also uploaded on the agenda wiki page here: https://community.icann.org/x/FysFBQ Best Regards, Mary, Julie, Ariel, and Berry ---------- Text from Analysis Group -------- Dear All, Attached please find our suggested revisions to the survey questions for each target group of the RPM survey (and, for reference, the Word version of the appendix to the RFP). We copied the questions from the RFP appendix into a Word document to follow a questionnaire format and have suggested edits to the survey in redline. We have also attached a clean version of each revised document. It may be easiest to review the clean version of our documents next to the RFP appendix, as the versions with tracked changes can be more difficult to follow. Please note that there may be minor formatting inconsistencies across the documents, as well as some minor differences in similar questions. We will clean these up this week, but wanted to provide you with the revised questions as soon as possible. As we discussed last week, the goal of our proposed edits is to limit the length of the survey while still allowing the capture of as much information as possible. To the extent possible, we converted open-ended questions to multiple choice. In some instances, we would like to request assistance with defining the multiple choice answers. In those instances, we have placed the request in double square brackets and highlighted the text in yellow. For some "anecdotal" questions, we could not think of an easy or clear way to translate the questions into closed-end responses. In those cases, we will allow open-text responses (marked by "open text field" responses). We removed some questions from several of the surveys, but believe more should still be removed from both the registry and trademark holder surveys. We have noted this in the relevant Word documents, and are hoping you can assist us in identifying several questions in each survey to remove. Best, Stacey and Greg Stacey Chan, Ph.D. | Manager Analysis Group, Inc. | Economic, Financial, and Strategy Consulting 1900 16th Street, Suite 1100, Denver, CO 80202 720-963-5307 (direct) | 720-963-5300 (main) stacey.chan(a)analysisgroup.com<mailto:stacey.chan@analysisgroup.com> l www.analysisgroup.com

1 0

Re: [Gnso-rpm-data] [Ext] RE: Meeting Schedule re: / Doodle / RPMs Data Sub Team
by Julie Hedlund June 1, 2018

June 1, 2018

Hi Susan, With the WG call cancelled on Wednesday, we’ll now have 5 people (with Phil and Kathy) who can join on Wednesday. Also, the concern is that meeting Friday may not allow enough time for comments to be finalized by the Friday deadline. Thus, the Co-Chairs suggested going with Wednesday and Thursday. Kind regards, Julie From: Susan Payne <susan.payne(a)valideus.com> Date: Friday, June 1, 2018 at 1:31 PM To: Julie Hedlund <julie.hedlund(a)icann.org>, "gnso-rpm-data(a)icann.org" <gnso-rpm-data(a)icann.org> Cc: "gnso-secs(a)icann.org" <gnso-secs(a)icann.org> Subject: [Ext] RE: Meeting Schedule re: / Doodle / RPMs Data Sub Team Hi all, based on the responses to the Doodle Poll that does not seem ideal. Only 3 people indicated they can join the call on Weds, compared to 5 on Thurs and 6 on Fri. S Susan Payne Head of Legal Policy | Valideus Ltd E: susan.payne(a)valideus.com D: +44 20 7421 8255 T: +44 20 7421 8299 M: +44 7971 661175 From: Gnso-rpm-data [mailto:gnso-rpm-data-bounces@icann.org] On Behalf Of Julie Hedlund Sent: 01 June 2018 18:17 To: gnso-rpm-data(a)icann.org Cc: gnso-secs(a)icann.org Subject: [Gnso-rpm-data] Meeting Schedule re: / Doodle / RPMs Data Sub Team Dear Data Sub Team members, Thank you to those of you who completed the Doodle. After reviewing the responses to the Doodle, consulting with the RPM WG Co-Chairs, and considering the deadline to provide comments to Analysis Group by Friday, 08 June, it was decided to schedule two Data Sub Team calls for next week as follows: Wednesday, 06 June: 1700-1830 UTC (instead of a full WG meeting), and Thursday, 07 June: 1600-1730 UTC Note that if the work can be completed on Wednesday, we’ll cancel the Thursday call. Holding the meetings on Wednesday and Thursday (as opposed to Thursday and Friday) would ensure that comments can provided to Analysis Group as early as possible on Friday, 08 June. Best regards, Mary, Ariel, Berry, and Julie From: Ntfy-gnso-rpm-data <ntfy-gnso-rpm-data-bounces(a)icann.org> on behalf of Julie Bisland <julie.bisland(a)icann.org> Date: Thursday, May 31, 2018 at 7:19 PM To: "ntfy-gnso-rpm-data(a)icann.org" <ntfy-gnso-rpm-data(a)icann.org> Cc: "gnso-secs(a)icann.org" <gnso-secs(a)icann.org> Subject: [Ntfy-gnso-rpm-data] **REMINDER** / Doodle / RPMs Data Sub Team Dear Data Sub Team members, Please complete the following Doodle poll to identify an appropriate date and time for the next Data Sub Team call. Note that the times will automatically adjust for your time zone. Doodle: https://doodle.com/poll/7np4vqtxhwegutg2 [doodle.com] Kind regards, Julie Julie Bisland Secretariat Operations Coordinator for the GNSO Internet Corporation for Assigned Names and Numbers (ICANN) Email: julie.bisland(a)icann.org Skype ID: 79613b691cb8a6c1 Find out more about the GNSO by visiting the GNSO Newcomer pages [gnso.icann.org]

4 5

Meeting Schedule re: / Doodle / RPMs Data Sub Team
by Julie Hedlund June 1, 2018

June 1, 2018

Dear Data Sub Team members, Thank you to those of you who completed the Doodle. After reviewing the responses to the Doodle, consulting with the RPM WG Co-Chairs, and considering the deadline to provide comments to Analysis Group by Friday, 08 June, it was decided to schedule two Data Sub Team calls for next week as follows: Wednesday, 06 June: 1700-1830 UTC (instead of a full WG meeting), and Thursday, 07 June: 1600-1730 UTC Note that if the work can be completed on Wednesday, we’ll cancel the Thursday call. Holding the meetings on Wednesday and Thursday (as opposed to Thursday and Friday) would ensure that comments can provided to Analysis Group as early as possible on Friday, 08 June. Best regards, Mary, Ariel, Berry, and Julie From: Ntfy-gnso-rpm-data <ntfy-gnso-rpm-data-bounces(a)icann.org> on behalf of Julie Bisland <julie.bisland(a)icann.org> Date: Thursday, May 31, 2018 at 7:19 PM To: "ntfy-gnso-rpm-data(a)icann.org" <ntfy-gnso-rpm-data(a)icann.org> Cc: "gnso-secs(a)icann.org" <gnso-secs(a)icann.org> Subject: [Ntfy-gnso-rpm-data] **REMINDER** / Doodle / RPMs Data Sub Team Dear Data Sub Team members, Please complete the following Doodle poll to identify an appropriate date and time for the next Data Sub Team call. Note that the times will automatically adjust for your time zone. Doodle: https://doodle.com/poll/7np4vqtxhwegutg2 [doodle.com] Kind regards, Julie Julie Bisland Secretariat Operations Coordinator for the GNSO Internet Corporation for Assigned Names and Numbers (ICANN) Email: julie.bisland(a)icann.org Skype ID: 79613b691cb8a6c1 Find out more about the GNSO by visiting the GNSO Newcomer pages [gnso.icann.org]

2 1

Recording, attendance & Webex chat / RPM Sub Team for Data Call / Wednesday, 30 May 2018
by Julie Bisland May 31, 2018

May 31, 2018

Dear All, Please find the attendance and Webex Chat of the call attached to this email, and the MP3 recording below for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call held Wednesday, 30 May 2018 at 17:00 UTC. Attendance of the call is also posted on the agenda wiki page: https://community.icann.org/x/PyIFBQ MP3: https://audio.icann.org/gnso/gnso-rpm-review-data-30may18-en.mp3 The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page: https://gnso.icann.org/en/group-activities/calendar ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-data/ Main wiki page for the working group: https://community.icann.org/x/eI1EB Thank you. Kind regards, Julie

1 0

Actions & Notes 30 May 2018 RPM Data Sub Team Call
by Julie Hedlund May 30, 2018

May 30, 2018

Dear All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting on 30 May 2018. The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records at: https://community.icann.org/display/RARPMRIAGPWG/2018-05-30+Sub+Team+for+Da…. The slides for the presentation from Analysis Group are attached for reference. Best Regards, Mary, Ariel, Berry and Julie Action Items Staff will send a Doodle to schedule the Data Sub Team call/calls: for 1700 Wednesday, 06 June; 1600 Thursday, 07 June; and 1600 Friday, 08 June. Analysis Group will provide a redline of the guidance and draft questions [icann.org]and the instruments for how Analysis Group would put it in the field by COB Tuesday, 05 June. The Data Sub Team will review the redline of the document and provide feedback to Analysis Group no later than COB Friday, 08 June. Notes 1. Introduction/presentation of Analysis Group -- Keep in mind that this is a tight timeline. -- How can we get the word out? Is there a plan in place? Answer: We’ve begun discussions, but we haven’t put together a formalized plan. -- Re: responses from 50 new gTLD registry operators – are there that many? Or 50 different TLDs? Re: 200 responses for potential domain name registrants. Answer: re: potential registrants – 200 is generally a number that can give you a good statistical response; re: registry operators, could restrict it to one response to TLD operator. Note that for Amazon we have 52 names so you would get 52 of the same responses. -- On outreach – Staff has started discussing how to do that. Outreach may be different with each of the target groups. Registry Operators and Registrars – we could use ICANN staff who have those contacts. We welcome more suggestions. -- On concern in terms of yield rate, we should be very mindful of the length of the survey. -- Probably need be ruling out registry operators that are only .brand registry operators. -- Question: On the potential domain name registrants – who are these people? Answer: Research Now/SSI relies on panels of individuals. We will target English speaking potential domain name registrants. They get a remuneration for each survey that they complete. We will send screening questions to identify them. -- Question: On the 50 of each group – we want to make sure that when we get the results back we have statistically significant results; what would that be? How do we determine that? For trademark and brand owners, as well as registrars, how do we reach out to smaller businesses? Answer: Taking registrars as an example, we will work closely with ICANN staff closely to get into contact with as large a universe as possible, and follow up with those who haven’t responded, and then once we collect the data we will look to see if it is representative of the universe of registrars and if it isn’t we’ll think about how to reach others. The goals that have been set for data collection are preliminary, and there is the issue of statistical significant. We will use a mathematical formula that can vary from small to very large. It also depends on the structure of the question. Depends on how the answers vary too. We will need questions that make people describe who they are and then we might not to revisit whether we’ve collected enough data and what do we do to try to get more. -- Can learn from the INTA survey. We had concerns about the length of the questions and the schedule. We may run into some of the same challenges, and maybe we can find ways to alleviate them. -- A note about registries and registrars – new gTLD registries that are brands are probably not going to be in your survey, such as Sony or Playstation, have one registrant so they don’t have to use rights protection mechanisms. -- Question - will there be any testing on people who are independent? I think the data team can be a useful group to test on, but we are all very familiar with what we are trying to achieve and so may not see flaws that someone independent might see. Answer: It will be useful to have people who have the deep institutional practical knowledge as well as those who don’t. >From the chat: >From Susan Payne to Everyone: re Registrars - important to get those who actually handled the RPMs - some did not do so >From Kristine Dorrain to Everyone: @Susan, as I recall, we ask that as a screening question but I thought we then asked why they didn't handle RPMs...were they a barrier to entry? >From Susan Payne to Everyone: ah yes, thanks Kristine, well remembered >From Michael R. Graham to Everyone: @Lori -- I would also be glad to join a call to discuss the INTA Survey learnings. >From Mary Wong to Everyone: @Kathy, we can perhaps request that registrars pass on the survey to the resellers but I am not sure we have a good database of resellers at the moment. We can discuss this suggestion further with Greg, David and Stacey. >From Kristine Dorrain to Everyone: Just to be clear, while I agree with Susan's point (that Kathy echoed) indicating that the survey is not best directed to .BRANDS, actually .BRANDs are not exempt from TM Claims. It's not relevant for our purposes maybe, but I just wanted to correct that for the record. >From Susan Payne to Everyone: @Kathy - yes agreed that .Brand ROs are not the target ROs (the same company might of course be the target as a brand owner). For avoidance of doubt, .Brands do have to run TM Claims >From Kristine Dorrain to Everyone: As the Registrar is the contracted party and ICANN has no control over resellers, I am not sure trying to survey resellers is going to be beneficial or informative. I do not believe we formulated any questions for resellers. 2. Discuss the guidance and draft questions [icann.org] developed by the Data Sub Team -- This was a small group and we had a good mix of people. We had quasi charter questions that was a list of grievances or gripes. We went through and pull out what we thought the questions were getting at. We also tried to parse out what were the underlying concerns. -- One of things we came up with was of course we wanted data, but we also recognized that even brand owners don’t have good data collection mechanisms. The problem was we didn’t know why some people were having concerns, so the anecdotal questions try to drill down. So the original charter questions were really suggestions. We can infer or guess why someone entered that question. -- Starting with the data question, but then giving people off ramps to say that they want to give us more information. 3. Discuss work plan, survey methodologies, survey targets, outreach plan, and proposed survey design and questions -- For some of the groups the survey could take a long time to fill out. -- Suggest that what Analysis Group does is send a detailed, redlined, commented, response to this. Let the Sub Team read them and then have a line-by-line discussion. -- We were hoping that the anecdotal questions would help you understand what we want to get and then get the feedback from Analysis Group. -- There may be pain points that a number may not be able to convey. -- Note that the questions for the registrants and potential registrants have a different flavor. More like a traditional consumer survey, which will affect how they will be treated. -- The terminology “anecdotal” is unhelpful. Narrative is a better term. -- On the “anecdotal” questions – we had contemplated places where we had asked a more data-driven question having an off ramp to a narrative response. Thinking about brand owners, you can collect the data, but then can you tick a box that says I want to provide more information? It would shorten the survey. Answer: Telephone interviews have to be conducted via a strict script. Reliability of the information can go down.

1 0

Re: [Gnso-rpm-data] [Ntfy-gnso-rpm-data] ***REMINDER*** Meeting Invitation: Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Wednesday, 30 May 2018
by Ariel Liang May 29, 2018

May 29, 2018

Dear Data Sub Team members, Kindly see below the proposed agenda for the Data Sub Team meeting tomorrow (Wed, 30 May) at 17:00-19:00 UTC. 1. Review agenda 2. Introduction/presentation of Analysis Group 3. Discuss the guidance and draft questions<https://www.icann.org/en/system/files/files/rfp-rpm-sunrise-tm-claims-appen…> developed by the Data Sub Team 4. Discuss work plan, survey methodologies, survey targets, outreach plan, and proposed survey design and questions 5. AOB Best Regards, Mary, Julie, Ariel, and Berry From: Ntfy-gnso-rpm-data <ntfy-gnso-rpm-data-bounces(a)icann.org> on behalf of Julie Bisland <julie.bisland(a)icann.org> Date: Tuesday, May 29, 2018 at 10:05 AM To: "ntfy-gnso-rpm-data(a)icann.org" <ntfy-gnso-rpm-data(a)icann.org> Cc: "gnso-secs(a)icann.org" <gnso-secs(a)icann.org> Subject: [Ntfy-gnso-rpm-data] ***REMINDER*** Meeting Invitation: Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data call on Wednesday, 30 May 2018 Dear All, The next call for the Review of all Rights Protection Mechanisms (RPMs) Sub Team for Data is scheduled on Wednesday, 30 May 2018 at 17:00 UTC for 2 hour duration. 10:00 PDT, 13:00 EDT, 19:00 Paris CEST, 22:00 Karachi PKT , (Thursday) 02:00 Tokyo JST, 3:00 Melbourne AEDT For other times: https://tinyurl.com/y9ag7vtt [tinyurl.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__tinyurl.com_y9ag7vtt&d…> Agenda Wiki: https://community.icann.org/x/PyIFBQ [community.icann.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_…> New Connection Information New Web conference tool: Webex Please join the meeting room here: https://mytrials.webex.com/join/nathalie.peregrine [mytrials.webex.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__mytrials.webex.com_joi…> to join the online event., Click "Join Now". Three audio connection possibilities: * If you are planning on listening only, please join the WebEx room via VoIP by connecting your computer audio to the WebEx room. This can be done upon logging into the room. * If your cell phone connectivity is stable and reliable, please set up a dial out to your phone from the WebEx room. This can be done upon logging into the room. * If you are intending on speaking on the call and need an operator managed connection, please join the Verizon bridge (details located below) WebEx Audio: To receive a call back, provide your phone number when you join the event, or call the number below and enter the access code. Join by phone 1-240-454-0879 USA Toll 1-240-454-0879 USA Toll Access code: 928 164 170 Global call-in numbers<javascript:void(0)> To join via Verizon Bridge, see dial in numbers and participant passcode below: If you require a dial-out or to send apologies (do not send to full working group) please send an email request with your preferred contact number to gnso-secs(a)icann.org<mailto:gnso-secs@icann.org> Please let me know if you have any questions. Thank you. Kind regards, Terri Participant passcode: DATA For security reasons, the passcode will be required to join the conference. Dial in numbers: Country Toll Numbers Freephone/ Toll Free Number ARGENTINA 0800-777-0519 AUSTRALIA ADELAIDE: 61-8-8121-4842 1-800-657-260 AUSTRALIA BRISBANE: 61-7-3102-0944 1-800-657-260 AUSTRALIA CANBERRA: 61-2-6100-1944 1-800-657-260 AUSTRALIA MELBOURNE: 61-3-9010-7713 1-800-657-260 AUSTRALIA PERTH: 61-8-9467-5223 1-800-657-260 AUSTRALIA SYDNEY: 61-2-8205-8129 1-800-657-260 AUSTRIA 43-1-92-81-113 0800-005-259 BELGIUM 32-2-400-9861 0800-3-8795 BRAZIL RIO DE JANEIRO: 55-21-40421490 0800-7610651 BRAZIL SAO PAULO: 55-11-3958-0779 0800-7610651 CHILE 1230-020-2863 CHINA CHINA A: 86-400-810-4789 10800-712-1670 CHINA CHINA B: 86-400-810-4789 10800-120-1670 COLOMBIA 01800-9-156474 CROATIA 080-08-06-309 CZECH REPUBLIC 420-2-25-98-56-64 800-700-177 DENMARK 45-7014-0284 8088-8324 EGYPT 0800000-9029 ESTONIA 800-011-1093 FINLAND 358-9-5424-7162 0-800-9-14610 FRANCE LYON: 33-4-26-69-12-85 080-511-1496 FRANCE MARSEILLE: 33-4-86-06-00-85 080-511-1496 FRANCE PARIS: 33-1-70-70-60-72 080-511-1496 GERMANY 49-69-2222-20362 0800-664-4247 GREECE 30-80-1-100-0687 00800-12-7312 HONG KONG 852-3001-3863 800-962-856 HUNGARY 36-1-700-8856 06-800-12755 INDIA INDIA A: 000-800-852-1268 INDIA INDIA B: 000-800-001-6305 INDIA INDIA C: 1800-300-00491 INDONESIA 001-803-011-3982 IRELAND 353-1-246-7646 1800-992-368 ISRAEL 1-80-9216162 ITALY MILAN: 39-02-3600-6007 800-986-383 ITALY ROME: 39-06-8751-6018 800-986-383 ITALY TORINO: 39-011-510-0118 800-986-383 JAPAN OSAKA: 81-6-7878-2631 0066-33-132439 JAPAN TOKYO: 81-3-6868-2631 0066-33-132439 LATVIA 8000-3185 LUXEMBOURG 352-27-000-1364 8002-9246 MALAYSIA 1-800-81-3065 MEXICO GUADALAJARA (JAL): 52-33-3208-7310 001-866-376-9696 MEXICO MEXICO CITY: 52-55-5062-9110 001-866-376-9696 MEXICO MONTERREY: 52-81-2482-0610 001-866-376-9696 NETHERLANDS 31-20-718-8588 0800-023-4378 NEW ZEALAND 64-9-970-4771 0800-447-722 NORWAY 47-21-590-062 800-15157 PANAMA 011-001-800-5072065 PERU 0800-53713 PHILIPPINES 63-2-858-3716 1800-111-42453 POLAND 00-800-1212572 PORTUGAL 351-2-10054705 8008-14052 ROMANIA 40-31-630-01-79 RUSSIA 8-10-8002-0144011 SAUDI ARABIA 800-8-110087 SINGAPORE 65-6883-9230 800-120-4663 SLOVAK REPUBLIC 421-2-322-422-25 0800-002066 SOUTH AFRICA 080-09-80414 SOUTH KOREA 82-2-6744-1083 00798-14800-7352 SPAIN 34-91-414-25-33 800-300-053 SWEDEN 46-8-566-19-348 0200-884-622 SWITZERLAND 41-44-580-6398 0800-120-032 TAIWAN 886-2-2795-7379 00801-137-797 THAILAND 001-800-1206-66056 TURKEY 00-800-151-0516 UNITED ARAB EMIRATES 8000-35702370 UNITED KINGDOM BIRMINGHAM: 44-121-210-9025 0808-238-6029 UNITED KINGDOM GLASGOW: 44-141-202-3225 0808-238-6029 UNITED KINGDOM LEEDS: 44-113-301-2125 0808-238-6029 UNITED KINGDOM LONDON: 44-20-7108-6370 0808-238-6029 UNITED KINGDOM MANCHESTER: 44-161-601-1425 0808-238-6029 URUGUAY 000-413-598-3421 USA 1-517-345-9004 866-692-5726 VENEZUELA 0800-1-00-3702 VIETNAM 120-11751

1 0