Dear All, Below are the action items and notes staff captured from the RPM Data Sub Team meeting on 30 May 2018. The notes from the call are posted to the Sub Team wiki space, together with the call recording, transcript and Adobe Connect chat and attendance records at: https://community.icann.org/display/RARPMRIAGPWG/2018-05-30+Sub+Team+for+Dat.... The slides for the presentation from Analysis Group are attached for reference. Best Regards, Mary, Ariel, Berry and Julie Action Items Staff will send a Doodle to schedule the Data Sub Team call/calls: for 1700 Wednesday, 06 June; 1600 Thursday, 07 June; and 1600 Friday, 08 June. Analysis Group will provide a redline of the guidance and draft questions [icann.org]and the instruments for how Analysis Group would put it in the field by COB Tuesday, 05 June. The Data Sub Team will review the redline of the document and provide feedback to Analysis Group no later than COB Friday, 08 June. Notes 1. Introduction/presentation of Analysis Group -- Keep in mind that this is a tight timeline. -- How can we get the word out? Is there a plan in place? Answer: We’ve begun discussions, but we haven’t put together a formalized plan. -- Re: responses from 50 new gTLD registry operators – are there that many? Or 50 different TLDs? Re: 200 responses for potential domain name registrants. Answer: re: potential registrants – 200 is generally a number that can give you a good statistical response; re: registry operators, could restrict it to one response to TLD operator. Note that for Amazon we have 52 names so you would get 52 of the same responses. -- On outreach – Staff has started discussing how to do that. Outreach may be different with each of the target groups. Registry Operators and Registrars – we could use ICANN staff who have those contacts. We welcome more suggestions. -- On concern in terms of yield rate, we should be very mindful of the length of the survey. -- Probably need be ruling out registry operators that are only .brand registry operators. -- Question: On the potential domain name registrants – who are these people? Answer: Research Now/SSI relies on panels of individuals. We will target English speaking potential domain name registrants. They get a remuneration for each survey that they complete. We will send screening questions to identify them. -- Question: On the 50 of each group – we want to make sure that when we get the results back we have statistically significant results; what would that be? How do we determine that? For trademark and brand owners, as well as registrars, how do we reach out to smaller businesses? Answer: Taking registrars as an example, we will work closely with ICANN staff closely to get into contact with as large a universe as possible, and follow up with those who haven’t responded, and then once we collect the data we will look to see if it is representative of the universe of registrars and if it isn’t we’ll think about how to reach others. The goals that have been set for data collection are preliminary, and there is the issue of statistical significant. We will use a mathematical formula that can vary from small to very large. It also depends on the structure of the question. Depends on how the answers vary too. We will need questions that make people describe who they are and then we might not to revisit whether we’ve collected enough data and what do we do to try to get more. -- Can learn from the INTA survey. We had concerns about the length of the questions and the schedule. We may run into some of the same challenges, and maybe we can find ways to alleviate them. -- A note about registries and registrars – new gTLD registries that are brands are probably not going to be in your survey, such as Sony or Playstation, have one registrant so they don’t have to use rights protection mechanisms. -- Question - will there be any testing on people who are independent? I think the data team can be a useful group to test on, but we are all very familiar with what we are trying to achieve and so may not see flaws that someone independent might see. Answer: It will be useful to have people who have the deep institutional practical knowledge as well as those who don’t.
From the chat:
From Susan Payne to Everyone: re Registrars - important to get those who actually handled the RPMs - some did not do so
From Kristine Dorrain to Everyone: @Susan, as I recall, we ask that as a screening question but I thought we then asked why they didn't handle RPMs...were they a barrier to entry?
From Susan Payne to Everyone: ah yes, thanks Kristine, well remembered
From Michael R. Graham to Everyone: @Lori -- I would also be glad to join a call to discuss the INTA Survey learnings.
From Mary Wong to Everyone: @Kathy, we can perhaps request that registrars pass on the survey to the resellers but I am not sure we have a good database of resellers at the moment. We can discuss this suggestion further with Greg, David and Stacey.
From Kristine Dorrain to Everyone: Just to be clear, while I agree with Susan's point (that Kathy echoed) indicating that the survey is not best directed to .BRANDS, actually .BRANDs are not exempt from TM Claims. It's not relevant for our purposes maybe, but I just wanted to correct that for the record.
From Susan Payne to Everyone: @Kathy - yes agreed that .Brand ROs are not the target ROs (the same company might of course be the target as a brand owner). For avoidance of doubt, .Brands do have to run TM Claims
From Kristine Dorrain to Everyone: As the Registrar is the contracted party and ICANN has no control over resellers, I am not sure trying to survey resellers is going to be beneficial or informative. I do not believe we formulated any questions for resellers.
2. Discuss the guidance and draft questions [icann.org] developed by the Data Sub Team -- This was a small group and we had a good mix of people. We had quasi charter questions that was a list of grievances or gripes. We went through and pull out what we thought the questions were getting at. We also tried to parse out what were the underlying concerns. -- One of things we came up with was of course we wanted data, but we also recognized that even brand owners don’t have good data collection mechanisms. The problem was we didn’t know why some people were having concerns, so the anecdotal questions try to drill down. So the original charter questions were really suggestions. We can infer or guess why someone entered that question. -- Starting with the data question, but then giving people off ramps to say that they want to give us more information. 3. Discuss work plan, survey methodologies, survey targets, outreach plan, and proposed survey design and questions -- For some of the groups the survey could take a long time to fill out. -- Suggest that what Analysis Group does is send a detailed, redlined, commented, response to this. Let the Sub Team read them and then have a line-by-line discussion. -- We were hoping that the anecdotal questions would help you understand what we want to get and then get the feedback from Analysis Group. -- There may be pain points that a number may not be able to convey. -- Note that the questions for the registrants and potential registrants have a different flavor. More like a traditional consumer survey, which will affect how they will be treated. -- The terminology “anecdotal” is unhelpful. Narrative is a better term. -- On the “anecdotal” questions – we had contemplated places where we had asked a more data-driven question having an off ramp to a narrative response. Thinking about brand owners, you can collect the data, but then can you tick a box that says I want to provide more information? It would shorten the survey. Answer: Telephone interviews have to be conducted via a strict script. Reliability of the information can go down.