Dear Paul and everyone, To assist Lori and the Working Group chairs, staff intends to compile all the questions that are sent to this list so that, if they are not addressed on the call later, Lori will be able to review them following the call. This may be especially helpful if you raise a question about a particular data point or a slide from the Nielsen deck that we are not able to cover on the call. It will be helpful therefore if, like Paul, any questions you have about a specific result can also include a reference to the slide number. Please note also that both the Nielsen slides (which contain all the results) and the more succinct presentation of the results that Lori made to the CCT Review Team have been posted to the Working Group wiki space here: https://community.icann.org/x/GhghB. I believe that, for purposes of allowing for greater Working Group discussion during our call, Lori will be using the same presentation slides as those that were used for the CCT Review Team presentation. Thanks and cheers Mary From: <gnso-rpm-wg-bounces@icann.org> on behalf of Paul Tattersfield <gpmgroup@gmail.com> Date: Wednesday, August 30, 2017 at 18:15 To: George Kirikos <icann@leap.com> Cc: gnso-rpm-wg <gnso-rpm-wg@icann.org> Subject: Re: [gnso-rpm-wg] Critique of INTA survey Hi George & Lori, I think the survey may be useful as long as we recognise those filling out the survey are mostly from the largest members of the INTA. I think NTIA dill well getting such substantive replies from these participants. Theses replies may be helpful as the benefits and problems they have with new gTLDs aree likely to be very different from medium sized and smaller enterprises. It may be when considering just the largest members the statistical signifance of the survey changes? With this in mind I think it would be helpful if an extra column could be added to each of the tables on page 6. And in that column show the Total NTIA members percentages i.e. the percentages of the 6,600 next to the each of the %ages from the 33 that replied. I will not be on the call this evening as it will be 4.00am local time here in Europe, but I would be grateful Lori if you have time to talk on slide 22 as I can review the transcript. On slide 22 can you explain a bit more about the differences between the two columns and why the second column isn't simply the reverse of column 1? Best regards, Paul. On Wed, Aug 30, 2017 at 9:02 PM, George Kirikos <icann@leap.com<mailto:icann@leap.com>> wrote: Hi Lori, [It's "George", by the way. There's another person named "Georges" on this list] I already submitted my comments, nearly 24 hours ago, and already gave ample notice. The 2nd paragraph of my email even said: "and make these statements in advance of the call, so that Lori or INTA/Nielsen have a chance to rebut" The questions I ask will flow from that analysis, today's followup, and also Kurt's email (since he won't be there to ask them himself). There are no surprises here. No one's trying to ambush you. Here's the very first question I will ask: "In order to draw valid conclusions from a survey, the sample must be both randomly selected, and the sample size be of sufficient size. Isn't it correct that the INTA survey is (a) a self-selected, non-random and unrepresentative sample, and (b) far too small, in order to make any statistically reliable conclusions about the larger population it purports to represent?" Any answer but "Yes, you're correct George" will be difficult to defend. The emails (from both myself and Kurt) give clear examples demonstrating how unrepresentative it is, and even links to relevant calculators for determining how large a sample size needs to be for various error margins, etc. After that, assuming you answer "Yes", a natural followup would be "Do you withdraw the study?" The best answer to that would be "Yes". Note, that the invitation (very politely offered -- to save everyone time) was given to withdraw it 8 hours before the conference call. It's now 7 hours before the conference call. Sincerely, George Kirikos 416-588-0269<tel:416-588-0269> http://www.leap.com/[leap.com]<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.leap.com_&d=DwMFaQ&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=DJ69mAe-idEhpAMF1nu2x6c2w3xl7xb5cjS_7sB4h6Y&m=1L_1wnr9S7NdASDPFvl1Sd8PlD7Fx9lnhNusVAdbgTY&s=XkMx-RepWjV8Eirx5yMKY_5BJ_tRUS_9Ez7mXVvn9kM&e=> On Wed, Aug 30, 2017 at 3:45 PM, Lori Schulman <lschulman@inta.org<mailto:lschulman@inta.org>> wrote:
Georges,
I am going to ask that all tough questions be put in writing so I can review and discuss with the survey’s administrators if I can’t answer them myself. As you noted, there is a low response rate and the analysis is complex and may not apply to this group’s work. If you have your questions already formulated, please submit them to the list.
Lori
Lori S. Schulman
Senior Director, Internet Policy
International Trademark Association (INTA)
+1-202-704-0408<tel:%2B1-202-704-0408>, Skype: lsschulman
From: gnso-rpm-wg-bounces@icann.org<mailto:gnso-rpm-wg-bounces@icann.org> [mailto:gnso-rpm-wg-bounces@icann.org<mailto:gnso-rpm-wg-bounces@icann.org>] On Behalf Of George Kirikos Sent: Wednesday, August 30, 2017 3:29 PM
To: gnso-rpm-wg <gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org>> Subject: Re: [gnso-rpm-wg] Critique of INTA survey
Lori,
It is false to say the my comments have any bias or hostility. They are sound arguments. I was open to it being a scientifically valid survey, but then I read it, multiple times. So did Kurt (maybe not multiple times for him!?!?), who I have no affiliation with.
I don't know whether the working group chairs were aware of the study's deep flaws before they made the invitation to present it, or had even read it, but now they do. If they want to keep the schedule, I'll be there to ask the tough questions tonight, and let the PDP members that want to try to defend it do so.
Sincerely,
George Kirikos 416-588-0269<tel:416-588-0269> http://www.leap.com/[leap.com]<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.leap.com_&d=DwMFaQ&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=DJ69mAe-idEhpAMF1nu2x6c2w3xl7xb5cjS_7sB4h6Y&m=1L_1wnr9S7NdASDPFvl1Sd8PlD7Fx9lnhNusVAdbgTY&s=XkMx-RepWjV8Eirx5yMKY_5BJ_tRUS_9Ez7mXVvn9kM&e=>
On Wed, Aug 30, 2017 at 3:22 PM, Lori Schulman <lschulman@inta.org<mailto:lschulman@inta.org>> wrote:
Dear All,
This working group chairs requested that I present INTA’s survey results and that I what I intend to do. I am here to present existing data. It is up to the group to decide if there is any value here. George comments show immediate bias and hostility toward the work before we have even started a discussion. I have stated all along that the study was intended for another purpose and that we had challenges with conducting it. If the PDP WG wishes to exclude the findings that is for the group to decide. Everything we do is a learning. George, if you feel that this evening’s call will have little or no value to your participation, you have the option of not dialing in and listening to the recording at your convenience.
Lori
Lori S. Schulman
Senior Director, Internet Policy
International Trademark Association (INTA)
+1-202-704-0408<tel:%2B1-202-704-0408>, Skype: lsschulman
From: gnso-rpm-wg-bounces@icann.org<mailto:gnso-rpm-wg-bounces@icann.org> [mailto:gnso-rpm-wg-bounces@icann.org<mailto:gnso-rpm-wg-bounces@icann.org>] On Behalf Of George Kirikos Sent: Wednesday, August 30, 2017 3:05 PM To: gnso-rpm-wg <gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org>> Subject: Re: [gnso-rpm-wg] Critique of INTA survey
P.S. There are roughly 8 hours to go until our scheduled call. I would invite Lori and/or INTA to simply withdraw the paper from this PDP (and the other ICANN group to which it was presented), since ultimately it is not a scientifically valid study. Any conclusions from it are indefensible.
It would bring more credibility to INTA to withdraw it, in my opinion, recognizing it as deeply flawed now, rather than to attempt to defend it for 90 minutes tonight, and ultimately see it abandoned/ignored by the PDP. As a group, we're always seeking efficiencies --- withdrawing this paper and giving everyone back their Wednesday night appears to me to be "low hanging fruit" in that regard.
The sooner it's withdrawn, the more time folks will have to make arrangements to enjoy their Wednesday evening.
Sincerely,
George Kirikos 416-588-0269<tel:416-588-0269> http://www.leap.com/[leap.com]<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.leap.com_&d=DwMFaQ&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=DJ69mAe-idEhpAMF1nu2x6c2w3xl7xb5cjS_7sB4h6Y&m=1L_1wnr9S7NdASDPFvl1Sd8PlD7Fx9lnhNusVAdbgTY&s=XkMx-RepWjV8Eirx5yMKY_5BJ_tRUS_9Ez7mXVvn9kM&e=>
On Wed, Aug 30, 2017 at 2:13 PM, George Kirikos <icann@leap.com<mailto:icann@leap.com>> wrote:
Hi Kurt,
Thanks for mostly agreeing with my analysis. However:
On Wed, Aug 30, 2017 at 1:47 PM, Kurt Pritz <kurt@kjpritz.com<mailto:kurt@kjpritz.com>> wrote:
There was one conclusion I could draw. It states that UDRP and Sunrise were the favored rights protection mechanisms, used to a major or moderate extent by 67% and 64% of the respondents respectively. The next most utilized RPMs were Trademark Claims and URS (by 36% and 27% respectively). To me this says that, to those who are in-the-know, Sunrise is a highly-valued RPM and, therefore, should be continued. (Sorry, George) (see slides 15 and 51)
The first part of your conclusion is correct (obviously anyone who personally benefits from "front of the line" privileges is going to value it), but the second part (therefore, that it should be continued) is NOT correct. As a PDP, our job is to weigh the benefits against the costs of policy choices amongst ALL stakeholders, not just ones receiving benefits.
Thus, if that was "the one conclusion (you) could draw", and it's now debunked, then we're left with the truth, that no conclusions can be drawn from it --- it's for entertainment value only, i.e. it's an advocacy piece, marketing fluff, not a scientifically-valid survey that would endure any serious peer review from those in the field of statistics.
To be clear, I tried to keep yesterday's email as short as possible (remember, it was a response to a very long document), and didn't point out every flaw with the survey. To point out another, note that on page 6 it notes that 67% of responses were from USA and Canada. However, INTA's own website states that:
"63% of our member organizations are outside of North America."
This further reinforces my point that it was an unrepresentative sample. As we know from election polling, the survey companies make adjustments in weighting to attempt to compensate for the unrepresentative samples (e.g. if too many men were sampled relative to the known proportion, they'd lower the weights accordingly, etc.). No attempts were made to do this (nor could they credibly have done so, given the small sample size, and lack of randomness).
This is a classic case of "If you torture the data long enough, it will confess to anything."
Sincerely,
George Kirikos 416-588-0269<tel:416-588-0269> http://www.leap.com/[leap.com]<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.leap.com_&d=DwMFaQ&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=DJ69mAe-idEhpAMF1nu2x6c2w3xl7xb5cjS_7sB4h6Y&m=1L_1wnr9S7NdASDPFvl1Sd8PlD7Fx9lnhNusVAdbgTY&s=XkMx-RepWjV8Eirx5yMKY_5BJ_tRUS_9Ez7mXVvn9kM&e=>
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg