Dear All, Please find the attendance of the call attached to this email. The MP3, Adobe Connect recording and Adobe Connect chat below for the Review of all Rights Protection Mechanisms (RPMs) PDP Working Group call held Thursday, 31 August 2017at 03:00 UTC. Attendance of the call is posted on agenda wiki page: https://community.icann.org/x/GhghB MP3: http://audio.icann.org/gnso/gnso-rpm-review-31aug17-en.mp3 Adobe Connect recording: https://participate.icann.org/p259wa70n1j/ The recordings and transcriptions of the calls are posted on the GNSO Master Calendar page:http://gnso.icann.org/en/group-activities/calendar[gnso.icann.org]<https://urldefense.proofpoint.com/v2/url?u=http-3A__gnso.icann.org_en_group-2Dactivities_calendar&d=DwMF-g&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=8_WhWIPqsLT6TmF1Zmyci866vcPSFO4VShFqESGe_5iHWGlBLwwwehFBfjrsjWv9&m=VqOOufta7uIGpIGDJ5U1cepLELhCsPPoZZC9osq3Hrk&s=eBrYQigfSiOfET2VcOvNtKoUgJHCm4qXeW7zA-dR-is&e=> ** Please let me know if your name has been left off the list ** Mailing list archives: http://mm.icann.org/pipermail/gnso-rpm-wg/ Wiki page: https://community.icann.org/x/GhghB Thank you. Kind regards, Michelle Adobe Connect chat transcript for 31 August 2017: Michelle DeSmyter:Dear All, Welcome to the Review of All Rights Protection Mechanisms (RPMs) in all gTLDs PDP WG call on Thursday, 31 August 2017 at 03:00 UTC. Michelle DeSmyter:Agenda wiki page: https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_G... Maxim Alzoba (FAITID):Hello All Jonathan Frost (.CLUB):Hi Maxim Louise Marie Hurel:Hi everyone Jonathan Agmon:hello Kathy Kleiman:I've been on told for awhile for the conference bridge. Anyone else having the same experience? Kathy Kleiman:told -> hold :-) Michelle DeSmyter:Hi there Kathy, let me speak with the operator and find out what is going on Louise Marie Hurel:We can hear you well, Kathy Kathy Kleiman:Great! Kathy Kleiman:Mary, Michelle, let's give it the usual extra minute to two. Kathy Kleiman:Tx to all who have joined us for this call! Schulman:Hi Schulman:There's another interesting sound. Kathy Kleiman:Can that line be muted? George Kirikos:Hi folks. Louise Marie Hurel:haha Mary Wong:Hi all, please remember to MUTE your mic if you are not speaking, thanks! Kathy Kleiman:Hi Lori! Kathy Kleiman:Hi George. George Kirikos:Are any of the participants here from Asia/Pacific? Sara Bockey:I see at least one person from apac George Kirikos:That's good to know, Sara. Michael Flemming:These meeting times are greatly appreciated from us in APAC. Jonathan Agmon:@George I thought you knew everybody ;) Jonathan Agmon:I am in Singapore... Maxim Alzoba (FAITID):I'd recommend to read e-mail from Kurt Pritz about this survey. George Kirikos:Good idea, Maxim. Maxim Alzoba (FAITID):79% of surveyed is >1B revenue ... so it is not so relevant to small and mediumbusiness (page 6) George Kirikos:That's not 'randomized' -- that's a self-selected survey. Paul McGrady:@Maxim - do you have a survey to provide that does a better job teasing out data from small and medium businesses? Would love to see it. George Kirikos:Kurt explained the difference. Paul McGrady:@George - do you have any randomized surveys supporting your views? Would love to see them... George Kirikos:@Paul: answered on the mailing list already. Maxim Alzoba (FAITID):@Paul, unfortunately not. Our experience (in registrar) that not many companies of this size aware on RPMs at all Kathy Kleiman:CCTRT -> Competition, Consumer Trust and Consumer Choice Review Team Paul McGrady:@George - noted. The answer was no, you have no surveys supporting your various opinions. Jon Nevett:ICANN requested the study -- did it help fund it as well? Mary Wong:@Jon, I don't believe the INTA study was requested by ICANN. Maxim Alzoba (FAITID):@Paul, I m meant small and medium Paul McGrady:@Maxim - how could that be? We were all assured that there was a large market for new gTLDs before Round 1 launched... Jon Nevett:Lori just said that ICANN requested it Greg Shatan:George, your confusing the initial sample and the respondents to the sample. Two different concepts. Maxim Alzoba (FAITID):@Paul, it depends on who assured you George Kirikos:@Greg: we know it's not representative sample. They even acknowledge the size issue in their report. George Kirikos:What they don't acknowledge is the size of the invalidity. They understate it. Greg Shatan:Those Nielsen people must not know much about surveys. Kiran Malancharuvil:haha Greg. George Kirikos:@Greg: they obviously do know better. I pointed that out in my email. Why don't you acknowledge that 33 responses is very, very bad. Paul McGrady:@Maxim - Ironically, it was the ICANN Economic Survey that was the basis for the community momentum to expand. Kathy Kleiman 2:Slide 6 Philip Corwin:Regardless of any critique of its statistical methodology and didsappointment at the response rate, it may still provide useful anecdotal data. It's certainly better than no data at all. Paul McGrady:@Greg, ha! Greg Shatan:Higher would be better, but I will refrain from using florid overstatements. George Kirikos:@Phil: if it's presented simply as unrepresentative anecdotal data, I have no problems with that at all. Maxim Alzoba (FAITID):+1 George George Kirikos:But, re-read Kurt's diplomatic response -- that's not how it was presented. George Kirikos:Consantly referencing "INTA members" (i.e. the entire population), rather than results being only for the 33 participants. Mike Rodenbaugh:INTA has acknowledged that results are an indicator of a trend and not the trend itself, as Lori just said Paul McGrady:@Phil - agree. If our basis to submit ideas and data into this group is statistical perfection, that is going to exlcude a lot of inputs. Kathy Kleiman 2:@All: Lori's words are very clear and useful. She is trying to address the issues you have raised. George Kirikos:@Mike: "indicator of a trend" is a way to massage things, to try to say that it has some validity. That's just not correct. George Kirikos:A bad survey doesn't tell you anything (good or bad). Greg Shatan:Results are always only from the participants, George. Jonathan Agmon:At least there is a report to look at. George Kirikos:@Greg: the reason we do statistics is to make statements about a larger population, based on data from a sampled subset. Mike Rodenbaugh:@George I disagree, at least it is anecdotal evidence Maxim Alzoba (FAITID):@Paul, without knowing the composition of those averages - it goes not give enough info to understand if conclusions are correct Louise Marie Hurel:I disagree, George. I believe it is clear towards how the results can be interpreted Mike Rodenbaugh:showing cost impacts to large mark owners Kiran Malancharuvil:@kathy - agree that lori is doing very well addressing issues. seems like the biggest critics are more interested in the chat room. Paul McGrady:+1 Jonathan. Disheartened to see folks with no surveys to back their opinion attacking this survey, just because it doesn't fit their narrative. Kathy Kleiman 2:Slide 9 George Kirikos:@Paul: Attack my arguments, rather than attack my lack of surveys. George Kirikos:You can't argue against math. Greg Shatan:George, it's not the sample that responds. Responses to any kind of surveys have to deal with non-response bias, response bias, selection bias, self-selection bias, etc. Maxim Alzoba (FAITID):The isue is not the survey, it is about re-presentation, having a survey does not mean it is applicable George Kirikos:Statistics principles are pretty well understood. Mike Rodenbaugh:nobody is saying this is a statistically valid survey George Kirikos:@Greg: and size --- did the surrvey even attempt that adjustment? No. Philip Corwin:Interesting that the RPM regarded as most effective is the legacy UDRP and not any of the new RPMs. George Kirikos:Notice slides constantly refer to "INTA members" (sorry, they're not all numbered, e.g. "average total defense costs per members." Mike Rodenbaugh:INTA has 1000+ corporate members, mostly large businesses; the 33 responses are indicative of overall experience of those members George Kirikos:Notice Lori just said "Our members", not "the 33 members that responded". Maxim Alzoba (FAITID):33 out of 1000 is 3.3% Georges Nahitchevansky:George Give it a rest already and let Lori finish her presentation George Kirikos:I'm not interrupting her, Georges. Greg Shatan:You know what she was referring to. There was a lengthy introduction. Greg Shatan:I think George wants to be on Mystery Science Theatre 3000.... Sara Bockey:Agree with Georges. Enough. Let Lori do her persentation. John McElwaine:petty Rebecca L Tushnet:How can it be indicative of overall experience when it's not even a representative group of INTA brand owners? Georges Nahitchevansky:You are interrupting everyone else. You can make your points after she is done Scott Austin:+1GeorgeS Rebecca L Tushnet:I agree that it is useful anecdotal evidence, but it's not about a trend (it's not a time series, among other things) Kristine Dorrain - Amazon Registry Services:@George, do you contend that the user stories this "survey" (quotes provided for your benefit) are invalid simply because there aren't enough of them? Brian Winterfeldt:It is more data than we have had to date Rebecca - it is a window if you will - granted not a scientifically perfect one. Jonathan Agmon:+1 Georges Kiran Malancharuvil:+1 Georges Paul McGrady:@George, you note that statistics principles are pretty well understood. However, in your earlier email, you indicate you read the survey in horror "(I've read the report three times now,in horror)." What statistical principle does one use to know when to switch from careful reading to reading it in horror? Philip Corwin:Lori, I don't see anything in these slides regarding use of private RPMs such as DPML and whether that affected use of sunrise. Were any questions asked about that? George Kirikos:Folks keep asking me questions, but then say I should wait for Lori to finish. Make up your mind. Greg Shatan:Cute. Paul McGrady:@George, I think it is fine for you to go ahead and answer my question about reading the survey in horror... George Kirikos:@Paul: go read the rest of that email. That was the preface, and then I explained it. Greg Shatan:First you complain about being asked questions and then you don't answer them. Make up your mind. George Kirikos:@Greg: I won't be distracted by repetitive questions. George Kirikos:Go re-read what I wrote, and put it on the mailing list. Greg Shatan:I don't see that question in the chat. David McAuley:Well presented, thank you Lori J. Scott Evans:thank you lori Kiran Malancharuvil:thanks Lori Greg Shatan:I read what you wrote. Once was more than enough. Brian Winterfeldt:Thank you Lori! Sara Bockey:Thanks, Lori Cyntia King:Thnanks, Lori. Brian Winterfeldt:Great summary. Kristine Dorrain - Amazon Registry Services:Thank you Lori. John McElwaine:thanks Lori Maxim Alzoba (FAITID):is the full presentation available? Georges Nahitchevansky:Thank you Lori Paul McGrady:Thanks Lori! Jon Nevett:Thanks Lori! Greg Shatan:Thank you, Lori. Philip Corwin:Thanks very much for the presentation, Lori. Mary Wong:I can put up the study slides if that is hepful. Scott Austin:Thank you Lori Mary Wong:Nielsen slides: https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_dow... Jonathan Agmon:Thank you Lori! Mary Wong:Let us (staff) know if anyone wants the Nielsen slides up in AC. Louise Marie Hurel:Thanks, Lori! I agree with Rebecca. While the report brings interesting data, the discrepancy between the number of those who actually answered the survey challenges the idea of an actual "indicator of a trend". I believe is a question of "what are the next steps", the follow up Maxim Alzoba (FAITID):@Lori, It would be interesting to see colmposition of numbers from page 10 (of full report) on price of defensive actions J. Scott Evans:George. Quit asking loaded questions. Kiran Malancharuvil:lori addressed this in her presentation Kiran Malancharuvil:perhaps George should have listened George Kirikos:What was loaded? Greg Shatan:Jane, you ignorant.... George Kirikos:Don't put words into my mouth. J. Scott Evans:@george. You basically asked her to admit that all your criticism is valid and that the study is useless. Ask new questions, we have all seen you review and opinion. Greg Shatan:What words were put in your mouth? Cyntia King:I have had numerous conversatoins w/ large brand owners who's data is spread across their organization & is difficult to obtain, nevermind share. George Kirikos:Did I use the word "ignorant"? Rebecca L Tushnet:Greg, leaving out "slut" doesn't make that quote any less inappropriate. Greg Shatan:You clearly implied it. Brian Winterfeldt:Brand owners spent a lot of time and energy to give us this information and I believe it would be foolish to ignore it. Cyntia King:+1 Brian George Kirikos:Implied? I did no such thing. Paul McGrady:+1 Brian Greg Shatan:I'm shocked, shocked that there's gambling in this establishment. Paul McGrady:MIchael Graham Brian Winterfeldt:It's Michael Graham. Cyntia King:We should utilize the info as the best available even if imperfect statistically. Kiran Malancharuvil:+1 Brian Kathy Kleiman 2:@Maxim, the full presentation will be circulated to the full WG Greg Shatan:Agree with Cynthia, these are factually accurate responses at a bare minimum. How to extrapolate them is a reasonable question. Mary Wong:The full set of Nielsen slides, Lori's presentation today and Lori's previous presentation to the CCT-RT are all posted here: https://urldefense.proofpoint.com/v2/url?u=https-3A__community.icann.org_x_G... David McAuley:Thanks Mary Maxim Alzoba (FAITID):@Cyntia, usually conclusions are quite dependent on the presented info, and having not statistically correct info on the input means you have not correct conclusions George Kirikos:Exactly, Maxim. Louise Marie Hurel:@Lori, do you have a detailed list of countries (rather than regions) that contributed? Schulman:I could ask Cyntia King:@Maxim We're not curing cancer. We're tryng to make recommendations to ICANN about RPMs based on the best data available. Louise Marie Hurel:origin* Greg Shatan:What do you mean by "statistically correct", Maxim? What is incorrect about these results? Maxim Alzoba (FAITID):for example, costs of ICANN compliant costs of 2k USD ... having myself managing all compliants for 2 TLDs and 3 Registrars for few years ... I do not see it relevant Paul McGrady:@Maxim - but people draw conclusions all the time in these calls and on the list with no surveys, flawed or otherwise, backing them up. Are all of their conclusions therefore "not correct"? Or is data perfect only a burden for brand owners alone to carry? Louise Marie Hurel:Thanks, that would be great, @Lori Maxim Alzoba (FAITID):@Greg, the issue is that it lacks composition of averages and not representative Brian Winterfeldt:It actually took a lot of effort to get as many responses as we had given the time required to complete the survey! Greg Shatan:Imperfect though they may be, they are not incorrect. Every survey, even the most rigorous surveys have flaws. Maxim Alzoba (FAITID):I think it could be seen as a source of anecdotal data George Kirikos:@Greg: it's the extrapolation to say that the results of the small sample also apply to the larger population that is the issue. Maxim Alzoba (FAITID):+1 George George Kirikos:It's a flaw to do so, given the circumstances. Greg Shatan:Maxim, what do you mean by "composition of averages"? George Kirikos:As anecdotes, fine. Weighted accordingly. Georges Nahitchevansky:So all of George K's so called anecdotal information over the past months should be discarded by your logic as in many instances, as others have pointed out, the so called evidence is based on questionable and flawed analysis and speculation Greg Shatan:George, this does not have to be binary, thought you want it to be. It Cyntia King:Biz decisions are often made best on best available data. There is a tradeoff: time/money vs. perfect data that groups like this must make in order to make reasonable & timely decisions. George Kirikos:Georges: I'm not presenting any anecdotes as more than what they are, which Kurt even acknowledged INTA has been doing. Greg Shatan:This is not merely anecdotal. Maxim Alzoba (FAITID):@Greg, kit is about 150k USD costs ... Maxim Alzoba (FAITID):*it George Kirikos:At best it is, Greg. Do you really believe these results apply to ALL Trademark holders, worldwide? (non-INTA and INTA combined, the entire population of TM holders) Greg Shatan:You've lost me. Cyntia King:+1 Lori Greg Shatan:Again, you are trying to be binary. That's not what I said, and that's a good example of a loaded question. Mike Rodenbaugh:INTA corporate members are generally large businesses; it is fair to extrapolate the experience of 33 responders across the broader group of 1000 members. Schulman:Agree with Rodenbaugh Michael Flemming:Question: The average total defense costs per company on slide 13 (I think), is that inclusive of individual worker's salaries or only the cost of the defensive action itself? Greg Shatan:I think that's a fair way to look at it, Mike. Rebecca L Tushnet:MIke, that's not even true on the study's own numbers, as far as I can tell--billion dollar companies overresponded Mary Wong:@Michael, do you mean Slide 13 of the nielsen deck or this presentation? Michael Flemming:This one Michael Flemming:Thank you, Lori. Greg Shatan:Rebecca, Mike said extrapolate. He did not get into the methods that would be use to extrapolate. Mike Rodenbaugh:@Rebecca yes those companies have better resources to prepare these responses Philip Corwin:Q-What if anything does this study tell us about efficacy of the new RPMs and whether and how they should be modified? Mike Rodenbaugh:you may have a point as to some of the numbers, i.e. dollars spent; but we can extrapolate many of the 'percentage' numbers quite fairly Maxim Alzoba (FAITID):an average of 79% of companies >1B turnover might mean few companies spending a lot comparing to others Rebecca L Tushnet:Sure, Mike, but that means you can't extrapolate even to INTA brand owners Georges Nahitchevansky:And to the type of business involved Cyntia King:@Phiip orwin That tis the right question Rebecca L Tushnet:At least not without rejiggering the numbers to match the ratios Maxim Alzoba (FAITID):@Mike it would be not fair to extrapolate from that George Kirikos:Exactly, Rebecca. Rebecca L Tushnet:And correspondingly increase the confidence intervals George Kirikos:+1 Rebecca. George Kirikos:If one keeps torturing that data, though, with only 33 responses, essentially you can't say much that is statistically valid, though. Philip Corwin:Thanks Cyntia. I have alwaus found that if you don't ask the right question you will likely not get a useful answer. Mike Rodenbaugh:@Phil that is the core question; imho it at least shows some value to the RPMs -- and we have seen little to no actual evidence of any harm from them George Kirikos:i.e. "50% +/- 50%" will be the result. Georges Nahitchevansky:e.g. a finance company would probably have more fraud and phishing issues than a dog food company. So costs would be higher for the finance company J. Scott Evans:The conclusion of the survey falls in line with the economic surveys ICANN did in the early 2000's and echo the myriad of public comments submitted by brand owenrs throughout the new gTLD process Michael Flemming:Very helpful, Michael. Thank you very much! Brian Winterfeldt:+1 J. Scott Greg Shatan:The issues for different sectors are different, not necessarily less or more. Kiran Malancharuvil:yes j scott. Cyntia King:@J. Scott Evans Verey interesting Greg Shatan:J Scott -- Very good point, and informs how we look at these results. Maxim Alzoba (FAITID):it would be hard to build consensus on basis of this report Kristine Dorrain - Amazon Registry Services:We can't build consensus on this report because it only reflect data about brand owners. It doesn't mean the brand owner stories are wrong. Brian Winterfeldt:+1 Cyntia Maxim Alzoba (FAITID):it gives us anecdotal information George Kirikos:Exactly, Maxim. Kristine Dorrain - Amazon Registry Services:We will still discuss other stakeholders as well. Kristine Dorrain - Amazon Registry Services:this study doesn't mean we won't also hear from them. George Kirikos:Some folks just don't appear to understand statistics. Greg Shatan:It gives us more than anecdotal information, as much as you try to stuff it down. George Kirikos:Perhaps ICANN needs to hire people, on staff, for statistics expertise. Mary Wong:If it helps, note that this Working Group is also proposing more surveys - of regisries, registrars, brand owners, registrants - for more anecdotal evidence. Michael R Graham:The results of the survey are what they are. They are, frankly, far more than the information we have from other stakeholders and groups. Paul McGrady:@George - I take exception at your denegrating INTA Staff. Philip Corwin:It would have been very surprising if more than a thousand new gTLDs did not generate some additional costs for TM owners. But, again, the focus of this study was costs, while the focus of our work is efficacy, balance, and potential RPM modifications. George Kirikos:@Mary: it won't be anecdotal data --- if it's random, a large sample, etc. Mary Wong:@George, the proposal is for anecdotal evidence for the most part. Kristine Dorrain - Amazon Registry Services:I was not part of any discussion group where we decided we were only ever going to look at fully validated surveys with a margin of error of 0.5. we have always said we'd look at whatever data we could get. George Kirikos:@Paul: stop putting words in my mouth. Michael R Graham:Anecdotal evidence has its own value. David McAuley:Thanks Kathy, Lori, and all, good bye Maxim Alzoba (FAITID):bye all Michael R Graham:Thanks all. Jonathan Agmon:Thank you all J. Scott Evans:ciao George Kirikos:Bye folks. Philip Corwin:Bye all. have a great Labor Day! Brian Winterfeldt:By everyone. Thank you Kathy. Louise Marie Hurel:Bye all. Thanks. Kiran Malancharuvil:thanks John McElwaine: bye Scott Austin:Thanks all Greg Shatan:Bye all. Thanks all, esp. Lori. Schulman:It was my pleasure. Really...