Misleading Slides (was Re: URS Practitioners Survey Results and Presentation for ICANN62)
Hi folks, In preparation for today's call, I reviewed the slides, and compared them with the "raw" results in the 2nd PDF. It's important to note that on the agenda page on the wiki (https://62.schedule.icann.org/meetings/699479) only the summary presentation is linked to, and not the raw results (which we received by email). The presentation is highly misleading (by omission), in particular for page 6, where it purports to neutrally summarize all the respondents to the survey. But, the actual survey question it's reporting was (from page 1 of the raw results) "In how many URS proceedings have you been involved as Complainant or its representative?" The related question (from page 2) "2. In how many URS proceedings have you been involved as Respondent or its representative?" was very clear that there was not a single response from practitioners who've represented Respondents." By omitting the actual questions from the survey, the presentation makes it appear (to someone who has not done a deep dive into the raw results) that this was a balanced survey of practitioners who represent both complainants and respondents to URS disputes. That could not be further from the truth, and so the results must be taken with a grain of salt. This was the equivalent of doing a survey of criminal attorneys, with all the responses from criminal prosecutors and without any responses from criminal defense lawyers. Furthermore, if we go to the raw questions, we had explicitly contemplated receiving input from those who'd only been involved in 1 or 2 cases (i.e. the "answer choices" for the first two questions were: 1 to 2, 3 to 5, 5 to 10, 10 or more, none). Yet, somehow in the implementation of the survey, it only "selected URS practitioners who had handled 5 cases or more" (page 5 of the presentation). This further skewed the results, and was inconsistent with the intent of the survey's questions. By the way, the survey claims to have "34 practitioners" on the "final list", but page 5 actually lists 36 in the right hand column. There's no explanation for that discrepancy. In summary, this appears to be yet another unrepresentative small sample, that is ultimately destined for the dust bin (along with the previously discredited INTA study). ICANN staff should add the PDF with the raw results to the agenda page, lest anyone be misled by this presentation. Sincerely, George Kirikos 416-588-0269 http://www.leap.com/ On Thu, Jun 21, 2018 at 11:27 AM, Julie Hedlund <julie.hedlund@icann.org> wrote:
Dear RPM PDP Working Group members,
On behalf of the URS Practitioners Sub Team, please see the attached documents for your reference for the presentation that the Sub Team will provide to the Working Group at ICANN62 during the following session:
Wednesday, 27 June, 10:30-12:00 (Panama City UTC-5): RPM PDP WG Meeting Session 1: 1) Presentation from URS Practitioners Sub Team re: Survey Results; 2) Update from TMCH Data Sub Team; 3) Update from URS Documents Sub Team
see: https://62.schedule.icann.org/meetings/699479
The attachments are the results of the responses to the URS practitioners survey, and the slides for the Sub Team’s presentation.
Best regards,
Mary, Ariel, Berry, and Julie
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
participants (1)
-
George Kirikos