Directly from INTA's website: What the TTAB has to say about sample size
Hi folks, Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147): (a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions" (b) "finding 57 respondents raised a question “as to the overall validity of the survey results" The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example: (c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling" (start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to) The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014. https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf C. Representative Samples (page number 1172) Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147 And here are the footnotes: 144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973). 145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”). 146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”). 147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents). (end of excerpt) Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said: "However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight" That's exactly the second problem experienced with this INTA survey, as previously discussed. Q.E.D. Have a nice day. Sincerely, George Kirikos 416-588-0269 http://www.leap.com/
George, This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective. Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed. Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc. On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky -- ******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you. *******************************************************************
Hi Jonathan, We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs. The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc. No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so. Sincerely, George Kirikos 416-588-0269 http://www.leap.com/ On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
George, I don't know how many times Lori and others have to tell you that no one is trying to extrapolate a trend from the 33 responses. (She mentioned it several times during the call... but I think your focus was on the chat, unfortunately for you and ultimately for us.) We have all agreed that these 33 responses should be taken as 33 companies' input into the process, through INTA. It's valuable information. Give it a rest. Your emails are badgering and unhelpful. Best, Kiran Kiran Malancharuvil Policy MarkMonitor 415.222.8318 (t) 415.419.9138 (m) www.markmonitor.com -----Original Message----- From: gnso-rpm-wg-bounces@icann.org [mailto:gnso-rpm-wg-bounces@icann.org] On Behalf Of George Kirikos Sent: Friday, September 01, 2017 2:40 AM To: gnso-rpm-wg <gnso-rpm-wg@icann.org> Subject: Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size Hi Jonathan, We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs. The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc. No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so. Sincerely, George Kirikos 416-588-0269 http://www.leap.com/ On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
While I often disagree with George, I don't think its helpful to tell him to give it a rest. If members feel that someone is acting in a manner that is incongruent with our standards of conduct, then please share those concerns with staff or the WG co-chairs so they can take action. There have been multiple emails from the co-chairs that participants are dropping off because of the hostile tone in our communications. This is not acceptable, nor should it be. It's easy to forget in this environment, because we all care so deeply, that in the end, we are all on the same team. Let's try to reflect that reality as much as we can. Best, Claudio On Fri, Sep 1, 2017 at 4:47 PM Kiran Malancharuvil via gnso-rpm-wg < gnso-rpm-wg@icann.org> wrote:
George,
I don't know how many times Lori and others have to tell you that no one is trying to extrapolate a trend from the 33 responses. (She mentioned it several times during the call... but I think your focus was on the chat, unfortunately for you and ultimately for us.) We have all agreed that these 33 responses should be taken as 33 companies' input into the process, through INTA. It's valuable information.
Give it a rest. Your emails are badgering and unhelpful.
Best,
Kiran
Kiran Malancharuvil Policy MarkMonitor 415.222.8318 (t) 415.419.9138 (m) www.markmonitor.com
-----Original Message----- From: gnso-rpm-wg-bounces@icann.org [mailto:gnso-rpm-wg-bounces@icann.org] On Behalf Of George Kirikos Sent: Friday, September 01, 2017 2:40 AM To: gnso-rpm-wg <gnso-rpm-wg@icann.org> Subject: Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size
Hi Jonathan,
We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs.
The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc.
No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky < jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9
(T.T.A.B.
Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
I kindly disagree with Claudio. It is entirely possible to be behaving in a way that is unhelpful to the forward momentum of the working group but without necessarily violating standards of conduct. I think it's perfectly acceptable, especially for those of us who are sensitive about how long this working group is taking, to ask those who have stated their opinions repeatedly that the inputs of other members of the working group be thrown into the trash to move on to more substantial of discussions. We have lots to do and the anti-trademark crowd's message that they think the inputs of others who disagree with their position are "trash" have already been made repeatedly clear. Those who worked hard to bring those inputs to the working group simply do not agree. Let's get on with our real work now. Sent from my iPhone On Sep 1, 2017, at 7:25 PM, claudio di gangi <ipcdigangi@gmail.com<mailto:ipcdigangi@gmail.com>> wrote: While I often disagree with George, I don't think its helpful to tell him to give it a rest. If members feel that someone is acting in a manner that is incongruent with our standards of conduct, then please share those concerns with staff or the WG co-chairs so they can take action. There have been multiple emails from the co-chairs that participants are dropping off because of the hostile tone in our communications. This is not acceptable, nor should it be. It's easy to forget in this environment, because we all care so deeply, that in the end, we are all on the same team. Let's try to reflect that reality as much as we can. Best, Claudio On Fri, Sep 1, 2017 at 4:47 PM Kiran Malancharuvil via gnso-rpm-wg <gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org>> wrote: George, I don't know how many times Lori and others have to tell you that no one is trying to extrapolate a trend from the 33 responses. (She mentioned it several times during the call... but I think your focus was on the chat, unfortunately for you and ultimately for us.) We have all agreed that these 33 responses should be taken as 33 companies' input into the process, through INTA. It's valuable information. Give it a rest. Your emails are badgering and unhelpful. Best, Kiran Kiran Malancharuvil Policy MarkMonitor 415.222.8318 (t) 415.419.9138 (m) www.markmonitor.com<http://www.markmonitor.com> -----Original Message----- From: gnso-rpm-wg-bounces@icann.org<mailto:gnso-rpm-wg-bounces@icann.org> [mailto:gnso-rpm-wg-bounces@icann.org<mailto:gnso-rpm-wg-bounces@icann.org>] On Behalf Of George Kirikos Sent: Friday, September 01, 2017 2:40 AM To: gnso-rpm-wg <gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org>> Subject: Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size Hi Jonathan, We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs. The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc. No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so. Sincerely, George Kirikos 416-588-0269 http://www.leap.com/ On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net<mailto:jonathan.matkowsky@riskiq.net>> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com<mailto:icann@leap.com>> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg ________________________________ The contents of this message may be privileged and confidential. If this message has been received in error, please delete it without reading it. Your receipt of this message is not intended to waive any applicable privilege. Please do not disseminate this message without the permission of the author. Any tax advice contained in this email was not intended to be used, and cannot be used, by you (or any other taxpayer) to avoid penalties under applicable tax laws and regulations.
+1 Paul George has nothing new to add. We understand his positions. He understands ours. Let's move on. Kiran Malancharuvil Policy Counselor MarkMonitor 415-419-9138 (m) Sent from my mobile, please excuse any typos.
On Sep 1, 2017, at 6:13 PM, icannlists <icannlists@winston.com> wrote:
I kindly disagree with Claudio. It is entirely possible to be behaving in a way that is unhelpful to the forward momentum of the working group but without necessarily violating standards of conduct. I think it's perfectly acceptable, especially for those of us who are sensitive about how long this working group is taking, to ask those who have stated their opinions repeatedly that the inputs of other members of the working group be thrown into the trash to move on to more substantial of discussions. We have lots to do and the anti-trademark crowd's message that they think the inputs of others who disagree with their position are "trash" have already been made repeatedly clear. Those who worked hard to bring those inputs to the working group simply do not agree.
Let's get on with our real work now.
Sent from my iPhone
On Sep 1, 2017, at 7:25 PM, claudio di gangi <ipcdigangi@gmail.com<mailto:ipcdigangi@gmail.com>> wrote:
While I often disagree with George, I don't think its helpful to tell him to give it a rest.
If members feel that someone is acting in a manner that is incongruent with our standards of conduct, then please share those concerns with staff or the WG co-chairs so they can take action. There have been multiple emails from the co-chairs that participants are dropping off because of the hostile tone in our communications. This is not acceptable, nor should it be.
It's easy to forget in this environment, because we all care so deeply, that in the end, we are all on the same team. Let's try to reflect that reality as much as we can.
Best, Claudio
On Fri, Sep 1, 2017 at 4:47 PM Kiran Malancharuvil via gnso-rpm-wg <gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org>> wrote: George,
I don't know how many times Lori and others have to tell you that no one is trying to extrapolate a trend from the 33 responses. (She mentioned it several times during the call... but I think your focus was on the chat, unfortunately for you and ultimately for us.) We have all agreed that these 33 responses should be taken as 33 companies' input into the process, through INTA. It's valuable information.
Give it a rest. Your emails are badgering and unhelpful.
Best,
Kiran
Kiran Malancharuvil Policy MarkMonitor 415.222.8318 (t) 415.419.9138 (m) www.markmonitor.com<http://www.markmonitor.com>
-----Original Message----- From: gnso-rpm-wg-bounces@icann.org<mailto:gnso-rpm-wg-bounces@icann.org> [mailto:gnso-rpm-wg-bounces@icann.org<mailto:gnso-rpm-wg-bounces@icann.org>] On Behalf Of George Kirikos Sent: Friday, September 01, 2017 2:40 AM To: gnso-rpm-wg <gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org>> Subject: Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size
Hi Jonathan,
We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs.
The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc.
No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net<mailto:jonathan.matkowsky@riskiq.net>> wrote: George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com<mailto:icann@leap.com>> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
________________________________ The contents of this message may be privileged and confidential. If this message has been received in error, please delete it without reading it. Your receipt of this message is not intended to waive any applicable privilege. Please do not disseminate this message without the permission of the author. Any tax advice contained in this email was not intended to be used, and cannot be used, by you (or any other taxpayer) to avoid penalties under applicable tax laws and regulations.
I agree with you generally except that the Trademark Reporter (on which I have sat on its editorial board in the distant past) collects very particular standards designed to meet very different evidentiary considerations than are at stake in what's at issue. And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further. If it doesn't for you, fine but frankly, participating regularly in this group has been nearly impossible given all the melodrama and pontification on a regular basis that is more harmful than it's worth to the group as a whole. If you need more anectodal evidence I'm happy to provide, and have told you as an INTA member that didn't participate in the survey that the majority reflects our views too, and the views of other INTA members and their customers, many of whom as smaller brand holders based on the lowest end of the revenue spectrum surveyed by INTA members, have spent upward of $150k in defensive registrations specifically related to the new gTLD launch, excluding the actual registration fees payable to the registrar. Best regards Jonathan On Fri, Sep 1, 2017 at 2:39 AM George Kirikos <icann@leap.com> wrote:
Hi Jonathan,
We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs.
The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc.
No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky -- ******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you. *******************************************************************
George. Sorry but this email below must have got caught in my draft emails from this morning, and just got sent. Anyway, my comments about this WG were not specifically intended to point fingers at you specifically so I owe you an apology. I just think we should give this a rest and move on as you made your point already and so have others. I'd like to get more active in this group but really encourage that we try to stay focused, make our points and then move on so we can progress. Hope you have a great weekend! Jonathan On Fri, Sep 1, 2017 at 8:33 PM jonathan matkowsky < jonathan.matkowsky@riskiq.net> wrote:
I agree with you generally except that the Trademark Reporter (on which I have sat on its editorial board in the distant past) collects very particular standards designed to meet very different evidentiary considerations than are at stake in what's at issue.
And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further. If it doesn't for you, fine but frankly, participating regularly in this group has been nearly impossible given all the melodrama and pontification on a regular basis that is more harmful than it's worth to the group as a whole.
If you need more anectodal evidence I'm happy to provide, and have told you as an INTA member that didn't participate in the survey that the majority reflects our views too, and the views of other INTA members and their customers, many of whom as smaller brand holders based on the lowest end of the revenue spectrum surveyed by INTA members, have spent upward of $150k in defensive registrations specifically related to the new gTLD launch, excluding the actual registration fees payable to the registrar.
Best regards Jonathan
On Fri, Sep 1, 2017 at 2:39 AM George Kirikos <icann@leap.com> wrote:
Hi Jonathan,
We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs.
The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc.
No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
-- Jonathan Matkowsky -- ******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you. *******************************************************************
It seems the point originally being considered was whether there was a significant enough participation in SunRise to warrant its continuance. Anecdotal evidence and opinions (and even poorly designed surveys) are of little help in answering the question. Seems to me that this WG could and should undertake a serious survey to find out the answer. Doing so will help to end to (or at least relieve) much of the positioning statements I see - most of which are opinion or anecdotally based. We are tasked to investigate and form a consensus based policy recommendation to be considered by the community and eventually board. ICANN has both the funds and expertise to assist in this project. Sincerely, Paul Keating, Esq.
On Sep 2, 2017, at 5:34 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
I agree with you generally except that the Trademark Reporter (on which I have sat on its editorial board in the distant past) collects very particular standards designed to meet very different evidentiary considerations than are at stake in what's at issue.
And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further. If it doesn't for you, fine but frankly, participating regularly in this group has been nearly impossible given all the melodrama and pontification on a regular basis that is more harmful than it's worth to the group as a whole.
If you need more anectodal evidence I'm happy to provide, and have told you as an INTA member that didn't participate in the survey that the majority reflects our views too, and the views of other INTA members and their customers, many of whom as smaller brand holders based on the lowest end of the revenue spectrum surveyed by INTA members, have spent upward of $150k in defensive registrations specifically related to the new gTLD launch, excluding the actual registration fees payable to the registrar.
Best regards Jonathan
On Fri, Sep 1, 2017 at 2:39 AM George Kirikos <icann@leap.com> wrote: Hi Jonathan,
We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs.
The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc.
No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you. ******************************************************************* _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
If it doesn't impede or slow down our work, I think Paul's suggestion is worthy of consideration. One approach could be for this WG to post a brief online Request for Information (RFI) that asks a series of questions about the utilization of Sunrise and the additonal marketplace RPMs for data collection purposes. The input and experience gained from the INTA survey can inform this process. This method was used successfully to obtain data on the 'domain name tasting' issue from several years ago. Alternatively, we can utilize the data that we already have in the staff report which indicates over 30,000 Sunrise transactions in 2014 alone, and other public resources that describe the level of costs imposed in specific TLDs, such as .sucks (millions of dollars), .porn, .adult, etc. This information reflects there was more money spent on Sunrise registrations (from the 2012 round) than in all previous gTLD Sunrise periods combined by an order of several multitudes. Best, Claudio On Sat, Sep 2, 2017 at 6:03 AM Paul Keating <paul@law.es> wrote:
It seems the point originally being considered was whether there was a significant enough participation in SunRise to warrant its continuance.
Anecdotal evidence and opinions (and even poorly designed surveys) are of little help in answering the question.
Seems to me that this WG could and should undertake a serious survey to find out the answer. Doing so will help to end to (or at least relieve) much of the positioning statements I see - most of which are opinion or anecdotally based.
We are tasked to investigate and form a consensus based policy recommendation to be considered by the community and eventually board.
ICANN has both the funds and expertise to assist in this project.
Sincerely, Paul Keating, Esq.
On Sep 2, 2017, at 5:34 AM, jonathan matkowsky < jonathan.matkowsky@riskiq.net> wrote:
I agree with you generally except that the Trademark Reporter (on which I have sat on its editorial board in the distant past) collects very particular standards designed to meet very different evidentiary considerations than are at stake in what's at issue.
And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further. If it doesn't for you, fine but frankly, participating regularly in this group has been nearly impossible given all the melodrama and pontification on a regular basis that is more harmful than it's worth to the group as a whole.
If you need more anectodal evidence I'm happy to provide, and have told you as an INTA member that didn't participate in the survey that the majority reflects our views too, and the views of other INTA members and their customers, many of whom as smaller brand holders based on the lowest end of the revenue spectrum surveyed by INTA members, have spent upward of $150k in defensive registrations specifically related to the new gTLD launch, excluding the actual registration fees payable to the registrar.
Best regards Jonathan
On Fri, Sep 1, 2017 at 2:39 AM George Kirikos <icann@leap.com> wrote:
Hi Jonathan,
We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs.
The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc.
No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
For the information of WG members, your co-chairs are currently working with policy staff to prepare a request to Council for the assistance of professionals in designing and disseminating surveys to meet our data needs. This will be consistent with the new directive that PDP WGs base their policy recommendations on reliable data to the extent feasible. We will provide further information as this develops. Philip S. Corwin, Founding Principal Virtualaw LLC 1155 F Street, NW Suite 1050 Washington, DC 20004 202-559-8597/Direct 202-559-8750/Fax 202-255-6172/Cell Twitter: @VlawDC "Luck is the residue of design" -- Branch Rickey From: gnso-rpm-wg-bounces@icann.org [mailto:gnso-rpm-wg-bounces@icann.org] On Behalf Of claudio di gangi Sent: Monday, September 04, 2017 2:03 PM To: Paul Keating; jonathan matkowsky Cc: gnso-rpm-wg Subject: Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size If it doesn't impede or slow down our work, I think Paul's suggestion is worthy of consideration. One approach could be for this WG to post a brief online Request for Information (RFI) that asks a series of questions about the utilization of Sunrise and the additonal marketplace RPMs for data collection purposes. The input and experience gained from the INTA survey can inform this process. This method was used successfully to obtain data on the 'domain name tasting' issue from several years ago. Alternatively, we can utilize the data that we already have in the staff report which indicates over 30,000 Sunrise transactions in 2014 alone, and other public resources that describe the level of costs imposed in specific TLDs, such as .sucks (millions of dollars), .porn, .adult, etc. This information reflects there was more money spent on Sunrise registrations (from the 2012 round) than in all previous gTLD Sunrise periods combined by an order of several multitudes. Best, Claudio On Sat, Sep 2, 2017 at 6:03 AM Paul Keating <paul@law.es<mailto:paul@law.es>> wrote: It seems the point originally being considered was whether there was a significant enough participation in SunRise to warrant its continuance. Anecdotal evidence and opinions (and even poorly designed surveys) are of little help in answering the question. Seems to me that this WG could and should undertake a serious survey to find out the answer. Doing so will help to end to (or at least relieve) much of the positioning statements I see - most of which are opinion or anecdotally based. We are tasked to investigate and form a consensus based policy recommendation to be considered by the community and eventually board. ICANN has both the funds and expertise to assist in this project. Sincerely, Paul Keating, Esq. On Sep 2, 2017, at 5:34 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net<mailto:jonathan.matkowsky@riskiq.net>> wrote: I agree with you generally except that the Trademark Reporter (on which I have sat on its editorial board in the distant past) collects very particular standards designed to meet very different evidentiary considerations than are at stake in what's at issue. And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further. If it doesn't for you, fine but frankly, participating regularly in this group has been nearly impossible given all the melodrama and pontification on a regular basis that is more harmful than it's worth to the group as a whole. If you need more anectodal evidence I'm happy to provide, and have told you as an INTA member that didn't participate in the survey that the majority reflects our views too, and the views of other INTA members and their customers, many of whom as smaller brand holders based on the lowest end of the revenue spectrum surveyed by INTA members, have spent upward of $150k in defensive registrations specifically related to the new gTLD launch, excluding the actual registration fees payable to the registrar. Best regards Jonathan On Fri, Sep 1, 2017 at 2:39 AM George Kirikos <icann@leap.com<mailto:icann@leap.com>> wrote: Hi Jonathan, We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs. The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc. No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so. Sincerely, George Kirikos 416-588-0269 http://www.leap.com/ On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net<mailto:jonathan.matkowsky@riskiq.net>> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com<mailto:icann@leap.com>> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg -- Jonathan Matkowsky ******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you. ******************************************************************* _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org<mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
Phil, thanks for the reminder about the effort that is underway on that front. Mary, I'm curious how staff was able to arrive at the figure of over 30,000 Sunrise transactions (as found in the staff report) for 2014? If it wasn't too difficult to do, can we derive those numbers for calendar years 2015, 2016, 2017? Best, Claudio On Mon, Sep 4, 2017 at 2:38 PM Phil Corwin <psc@vlaw-dc.com> wrote:
For the information of WG members, your co-chairs are currently working with policy staff to prepare a request to Council for the assistance of professionals in designing and disseminating surveys to meet our data needs. This will be consistent with the new directive that PDP WGs base their policy recommendations on reliable data to the extent feasible. We will provide further information as this develops.
*Philip S. Corwin, Founding Principal*
*Virtualaw LLC*
*1155 F Street, NW*
*Suite 1050*
*Washington, DC 20004*
*202-559-8597/Direct*
*202-559-8750/Fax*
*202-255-6172/Cell*
*Twitter: @VlawDC*
*"Luck is the residue of design" -- Branch Rickey*
*From:* gnso-rpm-wg-bounces@icann.org [mailto: gnso-rpm-wg-bounces@icann.org] *On Behalf Of *claudio di gangi *Sent:* Monday, September 04, 2017 2:03 PM *To:* Paul Keating; jonathan matkowsky *Cc:* gnso-rpm-wg
*Subject:* Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size
If it doesn't impede or slow down our work, I think Paul's suggestion is worthy of consideration.
One approach could be for this WG to post a brief online Request for Information (RFI) that asks a series of questions about the utilization of Sunrise and the additonal marketplace RPMs for data collection purposes. The input and experience gained from the INTA survey can inform this process. This method was used successfully to obtain data on the 'domain name tasting' issue from several years ago.
Alternatively, we can utilize the data that we already have in the staff report which indicates over 30,000 Sunrise transactions in 2014 alone, and other public resources that describe the level of costs imposed in specific TLDs, such as .sucks (millions of dollars), .porn, .adult, etc. This information reflects there was more money spent on Sunrise registrations (from the 2012 round) than in all previous gTLD Sunrise periods combined by an order of several multitudes.
Best,
Claudio
On Sat, Sep 2, 2017 at 6:03 AM Paul Keating <paul@law.es> wrote:
It seems the point originally being considered was whether there was a significant enough participation in SunRise to warrant its continuance.
Anecdotal evidence and opinions (and even poorly designed surveys) are of little help in answering the question.
Seems to me that this WG could and should undertake a serious survey to find out the answer. Doing so will help to end to (or at least relieve) much of the positioning statements I see - most of which are opinion or anecdotally based.
We are tasked to investigate and form a consensus based policy recommendation to be considered by the community and eventually board.
ICANN has both the funds and expertise to assist in this project.
Sincerely,
Paul Keating, Esq.
On Sep 2, 2017, at 5:34 AM, jonathan matkowsky < jonathan.matkowsky@riskiq.net> wrote:
I agree with you generally except that the Trademark Reporter (on which I have sat on its editorial board in the distant past) collects very particular standards designed to meet very different evidentiary considerations than are at stake in what's at issue.
And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further. If it doesn't for you, fine but frankly, participating regularly in this group has been nearly impossible given all the melodrama and pontification on a regular basis that is more harmful than it's worth to the group as a whole.
If you need more anectodal evidence I'm happy to provide, and have told you as an INTA member that didn't participate in the survey that the majority reflects our views too, and the views of other INTA members and their customers, many of whom as smaller brand holders based on the lowest end of the revenue spectrum surveyed by INTA members, have spent upward of $150k in defensive registrations specifically related to the new gTLD launch, excluding the actual registration fees payable to the registrar.
Best regards
Jonathan
On Fri, Sep 1, 2017 at 2:39 AM George Kirikos <icann@leap.com> wrote:
Hi Jonathan,
We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs.
The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc.
No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
--
Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
Just wanted to confirm we have numbers on the total number of Sunrise registrations (irrespective of price), ideally per Calendar year. The staff report identified over 30,000 in 2014 so it appears we do not need a survey, but defer to Mary. Best, Claudio On Tue, Sep 5, 2017 at 2:56 PM claudio di gangi <ipcdigangi@gmail.com> wrote:
Phil, thanks for the reminder about the effort that is underway on that front.
Mary, I'm curious how staff was able to arrive at the figure of over 30,000 Sunrise transactions (as found in the staff report) for 2014?
If it wasn't too difficult to do, can we derive those numbers for calendar years 2015, 2016, 2017?
Best, Claudio
On Mon, Sep 4, 2017 at 2:38 PM Phil Corwin <psc@vlaw-dc.com> wrote:
For the information of WG members, your co-chairs are currently working with policy staff to prepare a request to Council for the assistance of professionals in designing and disseminating surveys to meet our data needs. This will be consistent with the new directive that PDP WGs base their policy recommendations on reliable data to the extent feasible. We will provide further information as this develops.
*Philip S. Corwin, Founding Principal*
*Virtualaw LLC*
*1155 F Street, NW*
*Suite 1050*
*Washington, DC 20004*
*202-559-8597/Direct*
*202-559-8750/Fax*
*202-255-6172/Cell*
*Twitter: @VlawDC*
*"Luck is the residue of design" -- Branch Rickey*
*From:* gnso-rpm-wg-bounces@icann.org [mailto: gnso-rpm-wg-bounces@icann.org] *On Behalf Of *claudio di gangi *Sent:* Monday, September 04, 2017 2:03 PM *To:* Paul Keating; jonathan matkowsky *Cc:* gnso-rpm-wg
*Subject:* Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size
If it doesn't impede or slow down our work, I think Paul's suggestion is worthy of consideration.
One approach could be for this WG to post a brief online Request for Information (RFI) that asks a series of questions about the utilization of Sunrise and the additonal marketplace RPMs for data collection purposes. The input and experience gained from the INTA survey can inform this process. This method was used successfully to obtain data on the 'domain name tasting' issue from several years ago.
Alternatively, we can utilize the data that we already have in the staff report which indicates over 30,000 Sunrise transactions in 2014 alone, and other public resources that describe the level of costs imposed in specific TLDs, such as .sucks (millions of dollars), .porn, .adult, etc. This information reflects there was more money spent on Sunrise registrations (from the 2012 round) than in all previous gTLD Sunrise periods combined by an order of several multitudes.
Best,
Claudio
On Sat, Sep 2, 2017 at 6:03 AM Paul Keating <paul@law.es> wrote:
It seems the point originally being considered was whether there was a significant enough participation in SunRise to warrant its continuance.
Anecdotal evidence and opinions (and even poorly designed surveys) are of little help in answering the question.
Seems to me that this WG could and should undertake a serious survey to find out the answer. Doing so will help to end to (or at least relieve) much of the positioning statements I see - most of which are opinion or anecdotally based.
We are tasked to investigate and form a consensus based policy recommendation to be considered by the community and eventually board.
ICANN has both the funds and expertise to assist in this project.
Sincerely,
Paul Keating, Esq.
On Sep 2, 2017, at 5:34 AM, jonathan matkowsky < jonathan.matkowsky@riskiq.net> wrote:
I agree with you generally except that the Trademark Reporter (on which I have sat on its editorial board in the distant past) collects very particular standards designed to meet very different evidentiary considerations than are at stake in what's at issue.
And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further. If it doesn't for you, fine but frankly, participating regularly in this group has been nearly impossible given all the melodrama and pontification on a regular basis that is more harmful than it's worth to the group as a whole.
If you need more anectodal evidence I'm happy to provide, and have told you as an INTA member that didn't participate in the survey that the majority reflects our views too, and the views of other INTA members and their customers, many of whom as smaller brand holders based on the lowest end of the revenue spectrum surveyed by INTA members, have spent upward of $150k in defensive registrations specifically related to the new gTLD launch, excluding the actual registration fees payable to the registrar.
Best regards
Jonathan
On Fri, Sep 1, 2017 at 2:39 AM George Kirikos <icann@leap.com> wrote:
Hi Jonathan,
We're not a tribunal --- no one suggested that we are. Although, we do have something in common with a tribunal, namely that we as a group take a lot of input/data/evidence, and are supposed to weigh that evidence in an objective and scientific manner to arrive at truths (or policy decisions/recommendations), unbiased by our own prior beliefs.
The entire point of that post was to demarcate (via an independent source that is hard to dispute --- hard to dispute basic math/statistics --- a source I selected that INTA itself has published in their own journal, even) what is considered statistically valid findings, and what is considered merely anecdotal, what is considered insufficient sampling, etc.
No one is denying those 33 members of INTA who answered the survey had those experiences or opinions. What *is* in dispute is whether one should extract any truth about those experiences when talking about the larger populations, namely (1) all INTA members, and (b) all TM holders. Because of the issues with the study, it would not be credible to do so.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/
On Fri, Sep 1, 2017 at 3:53 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
--
Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.
*******************************************************************
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
Jonathan "And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further." This embodies exactly the point I and I believe George are trying to make. The study is not a study because it fails to meet any form of statistical standards. However you and others apparently believe it "carries extraordinary significance". I'm sorry but the lack of compliance with statistical norms means the survey results cannot be relied on for any purpose. There is not even a sufficient basis to believe it actually reflects the opinion of 30 some-odd trademark holders. So keep it if you wish but please let's all recognize its true worth. Sincerely, Paul Keating, Esq.
On Sep 2, 2017, at 5:34 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further
Paul,
From my perspective, the survey is the survey. Its limitations have been put forward clearly in writing and during Lori's presentation. I would like to believe that we can each make up our minds what weight to give to any document provided, not just by INTA but also by others.
What I find difficult to agree with is your statement that "[t]here is not even a sufficient basis to believe it actually reflects the opinion of 30 some-odd trademark holders." What "sufficient basis" you require? Are you saying that INTA fabricated the results? Are you not going too far there? Thanks, -----Original Message----- From: gnso-rpm-wg-bounces@icann.org [mailto:gnso-rpm-wg-bounces@icann.org] On Behalf Of Paul Keating Sent: Saturday, September 2, 2017 6:10 PM To: jonathan matkowsky <jonathan.matkowsky@riskiq.net> Cc: gnso-rpm-wg <gnso-rpm-wg@icann.org> Subject: Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size Jonathan "And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further." This embodies exactly the point I and I believe George are trying to make. The study is not a study because it fails to meet any form of statistical standards. However you and others apparently believe it "carries extraordinary significance". I'm sorry but the lack of compliance with statistical norms means the survey results cannot be relied on for any purpose. There is not even a sufficient basis to believe it actually reflects the opinion of 30 some-odd trademark holders. So keep it if you wish but please let's all recognize its true worth. Sincerely, Paul Keating, Esq.
On Sep 2, 2017, at 5:34 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg ************************************************************************************ This footnote confirms that this email message has been scanned by PineApp Mail-SeCure for the presence of malicious code, vandals & computer viruses. ************************************************************************************
The personal view of this co-chair is that the INTA report, while not based upon a large enough sample to have reliable statistical validity, nonetheless provides some useful anecdotal perspective. I was not particularly surprised by its findings that new gTLDs generated additional trademark defense costs, and that most new gTLD registrations were for defensive purposes, for and by the reporting entities. Indeed, I would have been surprised if the opposite had been reported. While the INTA report was seeking to answer the question of what expenses were generated by new gTLDs, and not evaluate the efficacy of the accompanying RPMs, I did think that this portion is particularly relevant to our work (from slide 14): RPM's are Helpful Two-thirds of the respondents surveyed feel that UDRPs and required sunrise periods have helped mitigate risks to a major/moderate extent. Of those who think that RPMs are effective the ranking is as follows: UDRP 67% Sunrise 64% Claims 36% URS 27% PDDRP/RRDRP/PICDRP 15% As noted in a prior email, your co-chairs are working with staff to develop a proposal for reliable data surveys that can assist in providing a sound basis for our ultimate answers to Charter questions. Best to all Philip S. Corwin, Founding Principal Virtualaw LLC 1155 F Street, NW Suite 1050 Washington, DC 20004 202-559-8597/Direct 202-559-8750/Fax 202-255-6172/Cell Twitter: @VlawDC "Luck is the residue of design" -- Branch Rickey -----Original Message----- From: gnso-rpm-wg-bounces@icann.org [mailto:gnso-rpm-wg-bounces@icann.org] On Behalf Of Jonathan_agmon icann Sent: Monday, September 04, 2017 4:28 AM To: Paul Keating; jonathan matkowsky Cc: gnso-rpm-wg Subject: Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size Paul,
From my perspective, the survey is the survey. Its limitations have been put forward clearly in writing and during Lori's presentation. I would like to believe that we can each make up our minds what weight to give to any document provided, not just by INTA but also by others.
What I find difficult to agree with is your statement that "[t]here is not even a sufficient basis to believe it actually reflects the opinion of 30 some-odd trademark holders." What "sufficient basis" you require? Are you saying that INTA fabricated the results? Are you not going too far there? Thanks, -----Original Message----- From: gnso-rpm-wg-bounces@icann.org [mailto:gnso-rpm-wg-bounces@icann.org] On Behalf Of Paul Keating Sent: Saturday, September 2, 2017 6:10 PM To: jonathan matkowsky <jonathan.matkowsky@riskiq.net> Cc: gnso-rpm-wg <gnso-rpm-wg@icann.org> Subject: Re: [gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size Jonathan "And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further." This embodies exactly the point I and I believe George are trying to make. The study is not a study because it fails to meet any form of statistical standards. However you and others apparently believe it "carries extraordinary significance". I'm sorry but the lack of compliance with statistical norms means the survey results cannot be relied on for any purpose. There is not even a sufficient basis to believe it actually reflects the opinion of 30 some-odd trademark holders. So keep it if you wish but please let's all recognize its true worth. Sincerely, Paul Keating, Esq.
On Sep 2, 2017, at 5:34 AM, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
And I don't think we should get distracted by the extent to which the survey reflects all INTA members as for many, it carries extraordinary significance without needing to get bogged down any further
gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg ************************************************************************************ This footnote confirms that this email message has been scanned by PineApp Mail-SeCure for the presence of malicious code, vandals & computer viruses. ************************************************************************************ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
Jonathan, While i can appreciate your enthusiasm, the goal is to move beyond individual opinion and attempt to locate fact based evidence. The INTA survey would fail any criteria taught in the most basic statistics class. The margin or error is so large as to render the study conclusions meaningless. It is all the more dangerous given it having been issued by "INTA". If it had been issued by any other entity i am sure you would have laughed it out of the room Sent from my iPad
On 1 Sep 2017, at 09:54, jonathan matkowsky <jonathan.matkowsky@riskiq.net> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com> wrote: Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you. ******************************************************************* _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
I don't even see the small sample size as the biggest issue. The biggest issue is how the report draws conclusions from the data that was provided that are not borne out by reality, for example when averages of spending or registrations are presented as generally applicable. Am 01.09.2017 um 11:48 schrieb Paul Keating:
Jonathan,
While i can appreciate your enthusiasm, the goal is to move beyond individual opinion and attempt to locate fact based evidence. The INTA survey would fail any criteria taught in the most basic statistics class. The margin or error is so large as to render the study conclusions meaningless. It is all the more dangerous given it having been issued by "INTA". If it had been issued by any other entity i am sure you would have laughed it out of the room
Sent from my iPad
On 1 Sep 2017, at 09:54, jonathan matkowsky <jonathan.matkowsky@riskiq.net <mailto:jonathan.matkowsky@riskiq.net>> wrote:
George,
This is not a tribunal subject to US federal law under the Latham Act or any other statute for that matter. The survey here speaks for itself and was very useful from our perspective.
Personally I wish we as an INTA member could have participated, but our opinion would have been consistent with what I saw to be the majority of those surveyed.
Jonathan Matkowsky, VP - IP & Brand Security RiskIQ, Inc.
On Thu, Aug 31, 2017 at 3:30 AM George Kirikos <icann@leap.com <mailto:icann@leap.com>> wrote:
Hi folks,
Before I went to bed, I made a note to myself to do a quick search to see what TM practitioners would do in their TTAB statistical studies. This morning, one of the first hits I found came from INTA's own website, see below. In particular, note the statements (from footnote 147):
(a) "characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions"
(b) "finding 57 respondents raised a question “as to the overall validity of the survey results"
The INTA survey sample size of 33 is far below even those two examples, and was closest to the next example:
(c) "finding a survey of 25 pharmacists and doctors to be an insufficient sampling"
(start of excerpt, sorry for the formatting, footnotes are #144 to #147; easier to read the PDF I link to)
The Trademark Reporter (The Law Journal of the International Trademark Association), September-October, 2014.
https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf
C. Representative Samples (page number 1172)
Probability and nonprobability methods may be used to select the sample from the universe of possible respondents. However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight.144 The number of respondents sampled must be large enough for the results to be reliable. The overall sample size for a survey will depend on the number of disputed marks tested and whether the survey includes any control groups. Surveys in Board proceedings often interview between 100 and 300 respondents about each mark or stimulus examined.145 In some instances, the Board has considered survey samples with fewer than 200 respondents to be small,146 and samples with fewer than 100 respondents routinely have been disfavored.147
And here are the footnotes:
144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5 (T.T.A.B. June 7, 2007) (“We also find that the survey results are questionable because the survey did not fairly sample the universe of possible respondents and is biased in MHP’s favor.”) and Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B. 1970) (“[T]he persons to be interviewed were not chosen on the basis of a sampling technique but solely because they were known to opposer . . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).
145. Facebook, Inc. v. Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013) (“Dr. Ford supervised . . . interviews: 270 in the test cell and 272 in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at *7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in the survey . . . with 200 consumers participating in one of two ‘test cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007) (“[S]urvey respondents in the test group (199 women age 18 and older from around the country) were shown a stimulus card . . .”); AVA Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783, 1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a card. . . . A control group of 100 respondents [was] shown a card . . .”). Note, the test group may include more respondents than the control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc., 2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was taken of 296 individuals, 200 of whom were exposed to applicant’s mark with the remaining 96 persons comprising a control group that was exposed to the fictitious mark . . .”).
146. 7-Eleven, Inc. v. Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162 survey respondents to be “small,” but according opposer’s survey some weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B. Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163, respectively.”).
147. Clear Choice Holdings LLC v. Implant Direct Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90 respondents for each mark tested to be “a small number”); Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL 1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as “anecdotal evidence” that did not lend themselves to statistical conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a question “as to the overall validity of the survey results”); Am. Home Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B. 1970) (finding a survey of 25 pharmacists and doctors to be an insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept. 2002) (finding it inappropriate to draw conclusions based on a survey with only three respondents).
(end of excerpt)
Of course, the above focused on sample size, but let's not forget the other part, about the non-representative nature of sample. Re-read the part above that said:
"However, if the sample of respondents is not representative of the universe from which it was selected, it will be accorded little weight"
That's exactly the second problem experienced with this INTA survey, as previously discussed.
Q.E.D.
Have a nice day.
Sincerely,
George Kirikos 416-588-0269 http://www.leap.com/ _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org <mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Jonathan Matkowsky
******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you.******************************************************************* _______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org <mailto:gnso-rpm-wg@icann.org> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
_______________________________________________ gnso-rpm-wg mailing list gnso-rpm-wg@icann.org https://mm.icann.org/mailman/listinfo/gnso-rpm-wg
-- Bei weiteren Fragen stehen wir Ihnen gerne zur Verfügung. Mit freundlichen Grüßen, Volker A. Greimann - Rechtsabteilung - Key-Systems GmbH Im Oberen Werk 1 66386 St. Ingbert Tel.: +49 (0) 6894 - 9396 901 Fax.: +49 (0) 6894 - 9396 851 Email: vgreimann@key-systems.net Web: www.key-systems.net / www.RRPproxy.net www.domaindiscount24.com / www.BrandShelter.com Folgen Sie uns bei Twitter oder werden Sie unser Fan bei Facebook: www.facebook.com/KeySystems www.twitter.com/key_systems Geschäftsführer: Alexander Siffrin Handelsregister Nr.: HR B 18835 - Saarbruecken Umsatzsteuer ID.: DE211006534 Member of the KEYDRIVE GROUP www.keydrive.lu Der Inhalt dieser Nachricht ist vertraulich und nur für den angegebenen Empfänger bestimmt. Jede Form der Kenntnisgabe, Veröffentlichung oder Weitergabe an Dritte durch den Empfänger ist unzulässig. Sollte diese Nachricht nicht für Sie bestimmt sein, so bitten wir Sie, sich mit uns per E-Mail oder telefonisch in Verbindung zu setzen. -------------------------------------------- Should you have any further questions, please do not hesitate to contact us. Best regards, Volker A. Greimann - legal department - Key-Systems GmbH Im Oberen Werk 1 66386 St. Ingbert Tel.: +49 (0) 6894 - 9396 901 Fax.: +49 (0) 6894 - 9396 851 Email: vgreimann@key-systems.net Web: www.key-systems.net / www.RRPproxy.net www.domaindiscount24.com / www.BrandShelter.com Follow us on Twitter or join our fan community on Facebook and stay updated: www.facebook.com/KeySystems www.twitter.com/key_systems CEO: Alexander Siffrin Registration No.: HR B 18835 - Saarbruecken V.A.T. ID.: DE211006534 Member of the KEYDRIVE GROUP www.keydrive.lu This e-mail and its attachments is intended only for the person to whom it is addressed. Furthermore it is not permitted to publish any content of this email. You must not use, disclose, copy, print or rely on this e-mail. If an addressing or transmission error has misdirected this e-mail, kindly notify the author by replying to this e-mail or contacting us by telephone.
participants (9)
-
claudio di gangi -
George Kirikos -
icannlists -
jonathan matkowsky -
Jonathan_agmon icann -
Kiran Malancharuvil -
Paul Keating -
Phil Corwin -
Volker Greimann