Dear All,

Please see below the action items and notes captured by staff from the Data Sub Team call held on 06 June 2018 (1700 UTC).  Staff have posted to the wiki space the action items and notes.  Please note that these will be high-level notes and are not meant as a substitute for the transcript or recording.  The recording, AC chat, and attendance records are posted on the wiki at: https://community.icann.org/x/FysFBQ

Best Regards,

Ariel

Ariel Xinyue Liang

Policy Analyst | Washington, DC 

Internet Corporation for Assigned Names and Numbers (ICANN) 

----------------

ACTION ITEMS

For Sub Team:

• Sub Team to provide comments/suggestions/questions on the draft surveys via Google docs to avoid the confusion that would result from multiple redlined documents.
• Sub Team to discuss internally about Q12 and see whether specific additional questions/clarifications are needed.
• Sub Team to suggest to AG additional options associated with Q12.
• Sub Team to reference the elements in the original questions associated with Q16/17 and see whether/how to revise the wording.

For AG:

• AG to change 60 days to 30 days.

For Staff:

• Staff to extend the meeting on 07 June at 1600 UTC by 30 minutes to 120 minutes total (done – calendar invite updated and sent)
• Staff to schedule a call on 08 June at 1600 UTC for 120 minutes (ongoing)
• Staff to schedule a contingency call on Monday 11 June at 1600 UTC for 120 minutes

NOTES

Registry Operator Survey

• General Comments:
  • Analysis Group (AG) made high level changes to reduce the number of open ended questions, remove leading questions, and simplify the language. Some original questions were removed. These high-level changes also apply to other surveys
  • Sub Team members Strongly favor allowing respondents to either not answer questions, or to indicate "Not Answering" in some gentle way. There is interest to aggregate the raw data, but some ROs may want to keep their survey responses confidential. Make it clear if the respondent skips certain questions, we still count their responses as valid. All questions will provide the option for the respondents to skip.
  • There should be one questionnaire going to one company, and we should avoid one RO sending back multiple responses if they manage multiple TLDs.
  • The current survey is quite long, so AG suggests dividing the survey into two parts, asking some ROs to respond one set, the other ROs to respond the other set. This approach may generate more responses in total. Sub Team feels that while this approach may be welcomed by the respondents, there may not be a big enough pool of ROs to do this. How valid the responses would be based on this approach? Sub Team don't want to send separate surveys to ROs but can parse the data once they are collected.
  • Sub Team would want to be able to quote responses that are text based.
• Q1
  • The question should ask the name of the respondent’s company/business?
  • This question can be removed; the intent is to it know the respondent. If this question remains, it should be optional.
  • Can we make the Q1 mandatory if the RO operates more than one TLD, so they can identify themselves? It may be useful to include such question.
  • The survey could ask ROs approximately how many TLDs that they operate, to provide more insight into the RO's operation. Or could provide a drop down, so large/medium/small ROs could identify themselves.
  • This question depends on how the RO selection is done. If specific ROs are invited to respond, it does not matter too much if they identify or not identify themselves. If a bulk invitation is sent to the RySG, for example, it would become more challenging, and the instruction needs to clarify that a RO only should complete one survey.
  • Staff believes that all registry operators will be asked to fill out the survey, i.e. no particular solicitation or invitation to some only
• Q3/Q4
  • This question may be reworded. Among the TLDs that you operate, what percentage approximately are community related, geo related, restricted by edibility terms? The intent of the question is to ask respondents to identify these specific TLDs that people have concerns with.
  • Perhaps lump in the IDN TLDs for the further consolidation of the questions.
• Q7-Q9
  • AG suggested to take out some of the pricing related questions to shorten the survey. Based on previous experience, such questions would be hard to answer by the ROs. Asking these questions in a survey instrument would be hard to illicit responses and may lead to higher dropout rate for the survey.
  • These are things (on premium pricing) that the Sub Team need to know. AG is willing to add questions back, with the caveat that it would raise potential issues of higher drop off rate.
• Q7
  • ROs do not know the names reserved in the TMCH. AG will revise the language.
• Q8
  • This question may potentially be removed if other pricing related questions are added in.
• Q8a
  • This question may also be removed if other pricing related questions are added in.
• Q9
  • It has the same issue like Q7, as the RO has to establish premium pricing before launches. The RO does not know what is in the TMCH and wouldn't know whether they are eligible for Sunrise. ROs would not be able to answer this question.
  • The original question in the data table is a part of a series of questions. The Sub Team want to know whether the ROs consciously try to avoid situation where they price brand names as premium names. There are some allegations that ROs 'targeted' brand names. So the series questions aim to set the stage about sunrise pricing/premium pricing, and then lead to this question.
  • AG will redraft this question based on Sub Team input.
• Q10
  • This question should be placed before Q9, or merge with Q9.
• Q11
  • As some ROs read through the survey, they might get confused this question with the one related geo/community TLDs (e.g., Q22). Make it clear that they are not the same questions.
  • AG will consider merging Q11 and Q22 as they are related.
• Q12
  • It is a technical matter and want to make sure ROs understand the language when taking the survey.
  • The word "required" -- clarify that it is an already existing policy
  • References to the Spec may not be necessary.
  • ACTION ITEM: Sub Team to discuss internally about Q12 and see whether specific additional questions/clarifications are needed.
  • ACTION ITEM: Sub Team to suggest to AG additional options associated with Q12.
• Q14
  • ROs would not know the names in TMCH. So would Q14 be unanswerable?
  • This question tries to address the gap related to the quasi perpetual claims period.
  • Not sure whether this question is clear enough for the respondent. But from RO perspective, this would not be a confusing question. If a similar question is asked from the brand owners or other parties, the wording needs to be clarified.
  • The RO does not need to know what is in the TMCH in order to comply with the spec.
• Q15
  • ACTION ITEM: AG to change 60 days to 30 days.
• Q16/Q17
  • AG suggested a few outcomes based on the intent of the original question.
  • This question needs to be clarified so the respondent knows why they are being asked. Some respondent might believe that Sunrise period can be unlimited.
  • Should AG suggest different lengths of Sunrise period, or ask respondents what ideal length would be?
  • As to the comparison of 30 day and more than 30 day charts -- how can we make this comparative? One could see answering all 30 days as GREAT and all More than 30 days as GREAT -- which tells us nothing.
  • Could we also ask the general question:  Do you believe increasing the 30 day period would improve any of these results?  How many days would you suggest?
  • Q16/17 do not seem to reflect the balance concerning all parties in the original question.
  • Should this question be deleted? There is a potential to turn the original open-ended questions into quantifiable questions. It may be helpful to come up with some alternative wordings for this question, as data is needed. Sub Team want the respondents to provide data, but also give feedback, so they would know what to do differently. AG should read deeper into the questions and see what they can pull from the responses.
  • ACTION ITEM: Sub Team to reference the elements in the original questions associated with Q16/17 and see whether/how to revise the wording.

Procedure Matters

• Sub Team would need to homework in order to provide targeted input for AG.
• We cannot sacrifice the quality of surveys just to meet the timeline. This is not the place to cut the corner, so we may need to have more calls. Taking the time now to ensure the BEST survey and metrics will dramatically reduce the time spent "discussing" reality based on narrative, perception, and preconceptions. 
• The RFP / contract includes the timeline. There is flexibility, but the contract contemplates that Analysis Group's work will be more or less completed by August.
• ACTION ITEM:
  • Staff to extend the meeting on 07 June at 1600 UTC by 30 minutes to 120 minutes total (done – calendar invite updated and sent)
  • Staff to schedule a call on 08 June at 1600 UTC for 120 minutes (ongoing)
  • Staff to schedule a contingency call on Monday 11 June at 1600 UTC for 120 minutes
  • Sub Team to provide comments/suggestions/questions on the draft surveys via Google docs to avoid the confusion that would result from multiple redlined documents.