P.S. Sufficient sample sizes was a topic of discussion last year, in relation to the INTA study. Recall that an article in INTA's own journal discussed appropriate sample sizes: https://mm.icann.org/pipermail/gnso-rpm-wg/2017-August/002438.html "finding a survey of 25 pharmacists and doctors to be an insufficient sampling" Esther Dyson had a nice quote, "Always make new mistakes." Unfortunately, it seems the same mistakes are happening when it comes to the empirical work here. Sincerely, George Kirikos 416-588-0269 http://www.leap.com/ On Wed, Aug 1, 2018 at 8:52 AM, George Kirikos <icann@leap.com> wrote:
Hi folks,
Without going into detail on all the documents (which were sent just 5 hours before our scheduled call), I'd like to reiterate my concerns expressed in late June:
https://mm.icann.org/pipermail/gnso-rpm-wg/2018-June/003186.html
that the survey results of practitioners gives too much weight to those representing complainants, and also is too small a sample to have statistical significance. We spent considerable time as a group creating the survey questions. Instead of sending it to all practitioners who participated in a URS dispute (already a relatively small group), it was instead sent only to those who were "repeat customers" so to speak. That reduced the number of responses considerably, and also made it unrepresentative. Rather than attempting to draw conclusions from such a sample, the proper course should have been to go back and collect all the data (i.e. from those who participated in 1 or more URS disputes), rather than an unrepresentative and small sample.
To understand why it's unrepresentative, suppose we were studying the experiences of those who visited a restaurant, for a Yelp review, etc. It's highly likely that those who've only visited that restaurant one time will have had different impressions than those who have visited 5+ times. Those who didn't like the food or service on their first visit would not go back for a 2nd or 3rd meal. By not giving any weight at all to those unsatisfied visitors (by not surveying them), it creates an unrepresentative sample of those that remain, who on balance are expected to have liked the restaurant, given they've gone 5+ times to it.
To understand the true experience of the URS, we need to have more data from respondents/registrants themselves, an important stakeholder in this process (not just providers and representatives of complainants).
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Wed, Aug 1, 2018 at 8:09 AM, Julie Hedlund <julie.hedlund@icann.org> wrote:
Dear RPM PDP Working Group members,
Further to the message below, please see the following draft agenda and related materials for today’s meeting, 01 August, at 17:00 UTC for 90 minutes. A meeting invite has been sent separately.
Best,
Mary, Ariel, Berry, and Julie
Draft Agenda:
1. Review Agenda/SOIs
2. URS Sub Team Presentations:
a. URS Practitioners Sub Team
b. URS Providers Sub Team
c. URS Documents Sub Team
3. Upcoming Meetings/Next Steps
4. AOB
From: gnso-rpm-wg <gnso-rpm-wg-bounces@icann.org> on behalf of Julie Hedlund <julie.hedlund@icann.org> Date: Tuesday, July 24, 2018 at 1:46 PM To: "gnso-rpm-wg@icann.org" <gnso-rpm-wg@icann.org> Subject: [gnso-rpm-wg] Upcoming RPM PDP Working Group Meeting 01 August at 17:00 UTC
Dear RPM PDP Working Group members,
Some of you will have received invites to the URS Document Sub Team meeting tomorrow, 25 July, at 17:00 UTC for 90 minutes. The URS Providers and Practitioners Sub Teams also have been invited to join that call. The URS Sub Teams are preparing their updates to the full WG concerning issues they have identified based on the data they have gathered. Please note that the URS Providers and URS Practitioners Sub Teams do not have separate meetings scheduled this week.
In addition, the TMCH Data Sub Team is meeting tomorrow at 17:00 UTC to finalize the Sunrise and Claims surveys.
The full RPM Working Group will resume its weekly meetings beginning on Wednesday, 01 August at 17:00 UTC for 90 minutes. It is anticipated that during this meeting the URS Sub Teams will begin to present issues that they have identified for consideration and possible recommendations. You will shortly see a calendar invite for this meeting.
Best,
Mary, Ariel, Berry, and Julie
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg