Hello Brett! Thank you very much for all of your very insightful comments. I addressed your questions in bold font below each point. Please don’t hesitate to reach out to me for further clarification on this topic. I also would like to take this opportunity to wish everyone Happy Holidays and look forward to continuing to have these productive discussions with the group in the coming year! Regards, Marilia Hirano IANA Services Continuous Improvement PTI - Public Technical Identifiers Direct: +1 310 578 8971 Mobile: +1 310 804 3549 www.pti.icann.org From: Brett Carr <brett.carr@nominet.uk> Date: Wednesday, December 12, 2018 at 11:04 PM To: Naela Sarras <naela.sarras@iana.org>, "icann-csc@icann.org" <icann-csc@icann.org> Cc: Marilia Hirano <marilia.hirano@iana.org> Subject: Re: [ICANN-CSC] FW: [Ext] ICANN News Alert -- IANA Functions Customer Survey Results Available Naela, Congratulations on the overall excellent results in the survey, a few small comments: Slide3: Participation: 22% Contact rate and 5% completion rate. These numbers seem quite low, obviously driving up contact should drive up completion. Appreciate it’s not easy but I believe it would be beneficial to look at ways of driving this up for future surveys. MH: The response rate over the years has been dropping, and last year the CSC got involved to try to revamp interest in the annual survey, specifically since in the ccTLD segment we had 0% responses for two years. While their campaign there worked and we got more people responding in the ccTLD community, we continued to see a drop in responses from the gTLD side as well as the other segments. A lot of the feedback we received over the years was about the cadence of the survey. Basically, customers felt like they didn’t really remember their interactions with our team after a whole year had passed, and therefore couldn’t really respond. For this reason, we agreed earlier this year to change the annual survey to a post-interaction survey, where customers would get a chance to provide feedback immediately after their request or enquiry was resolved. We spent the half part of this year developing a tool for this survey and launched it in early December. We are hoping that this will address the timeliness issue, but most importantly in the survey world, allow us to tackle negative feedback as they happen, rather than hear it once a year. With that being said, we still plan on having an annual survey in 2019, however, it will be a different scope – more engagement driven rather than request driven. We are still in early talks about what this will look like, but I would welcome the CSC input if the group is interested. Slide 9: It seems like 51% awareness of the process for resolving customer service issues is low, it could be argued of course that this is because you do such a good job people don’t need to know. However this may trigger you into thinking of ramping up any “awareness” campaigns you already do. MH: Yes, this has been the average rate over the years. We can definitely discuss ways to increase that awareness if the group wants. Slide 10: Customer participation in the survey seems to be on a downward trend which is worrying I think. MH: Same comment as your participation one above. Slide 37: I’m wondering what the cause of the “Accuracy of the root zone database” only being 85% last year was? It seems all the scores here have gone up significantly over the last 12 months, assume this is because there are less GTLD new delegations/changes than there were previously. MH: Last year we had a N/A option, which drove our results down, so we removed this option this year since we felt like there weren’t any questions in the survey where a Not Applicable option would be warranted. The actual satisfaction rate for this segment last year was: 85% satisfied/very satsifed, 0% dissatisfied/ very dissatisfied and 15% chose the N/A option. If you would like to see more on how that impacted our results, please see the full 2017 report here: https://www.iana.org/reports/2018/customer-survey-20180118.pdf Slide 38: The response number of four seems very low. Where an organisation is both a ccTLD operator and a GTLD operator do they get sent two surveys because we (Nominet) only received one AFAIK. MH: Yes, we agree it is very low. Especially since in previous years we had a lot more participation from the gTLD groups. To answer your question, if the email address of the invitee is attached to more than one segment, the email would get one survey link, but with questions about all relevant segments. This is a way we avoid sending multiple links to the same person and then having them fill out the same general questions more than once. Looking through the report as a whole there are not many occurrences of “dissatisfied” but any at all are a matter of concern, I was wondering if you follow up on any of these or is the survey anonymous preventing this? MH: The survey is anonymous which prevents us from addressing dissatisfaction to the actual respondent. What we do is identify trends and use them to prioritize our improvements – examples of such are RZMS enhancements, Private Enterprise Number application improvements, and improvements to the survey methodology. Please don’t take all this to be negative though merely observations that I hope can help. The overall survey result is excellent, yourself, Kim and the rest of the team should be very proud of yourselves. MH: we really appreciate you taking the time to raise these questions! As the lead on department improvements and planning, I am always happy to hear what our stakeholders have to say. Regards Brett -- Brett Carr Manager DNS and Network Engineering Nominet UK From: ICANN-CSC <icann-csc-bounces@icann.org> on behalf of Naela Sarras <naela.sarras@iana.org> Date: Friday, 7 December 2018 at 17:04 To: "icann-csc@icann.org" <ICANN-CSC@icann.org> Cc: Marilia Hirano <marilia.hirano@iana.org> Subject: [ICANN-CSC] FW: [Ext] ICANN News Alert -- IANA Functions Customer Survey Results Available Dear Colleagues, This email is to call your attention to the posting of the results from the Annual IANA Functions Customer Survey Results. Links to the announcement and report are available below. Also related to measuring customer satisfaction, please note that this week the IANA Functions released phase 1 of the a post-interaction survey where customers can provide immediate feedback regarding the services received. Phase 1 of the release targets completed requests for the domain names function with future phases focused on other request types to follow. Best regards, Naela From: ICANN News Alert <no-reply@external.icann.org> Reply-To: "no-reply@external.icann.org" <no-reply@external.icann.org> Date: Thursday, December 6, 2018 at 4:51 PM To: Naela Sarras <naela.sarras@iana.org> Subject: [Ext] ICANN News Alert -- IANA Functions Customer Survey Results Available [icann.org] News Alert https://www.icann.org/news/announcement-2018-12-06-en [icann.org] IANA Functions Customer Survey Results Available 6 December 2018 Los Angeles – 6 December 2018 – Today, the Internet Corporation of Assigned Names and Numbers (ICANN) published the results of its annual survey which measures customer satisfaction of the Internet Assigned Numbers Authority (IANA) functions. Results of the 2018 survey, which accounts for transactions completed between September 2017 and August 2018, reported 96 percent of respondents being "satisfied" or "very satisfied." View Customer Survey Findings [PDF, 3.47 MB] The IANA Functions Customer Survey measures perception of satisfaction in relation to documentation quality, process quality, transparency, timeliness, accuracy, reporting and courtesy. In the 2018 survey, customers identified accuracy as the most important measure of performance for the sixth consecutive year, and 96 percent reported satisfaction on that metric. Notably, overall satisfaction ratings across all of the performance categories were the highest ever this year. While the results of the survey are consistently positive, ICANN continues to explore opportunities for improvement. For instance, some respondents expressed concerns regarding the resolution of customer service issues. To help address this, a post-interaction survey was recently launched where customers can provide immediate feedback regarding the services, with resources dedicated to reviewing and responding in a timely manner. "Over the years our customers have reported that an annual survey doesn't allow them to provide immediate feedback on their interaction. The post-interaction survey will provide direct, actionable feedback from our customers, and offer a holistic view of our systems and processes to drive future improvements," said Kim Davies, President of PTI, the ICANN affiliate responsible for delivering the IANA functions. Davies added, "The first few years we ran this survey resulted in a 10-11 percent participation rate. But in the last two years, this number decreased in half which was another driver to revamp the surveying approach." ICANN commissioned Echo Research, LLC, a global reputation consultancy, to administer the survey. Those who make IANA functions transactions – top-level domain operators, protocol parameter and regional Internet registry managers, RFC authors, Internet Engineering Steering Group members, DNSSEC KSK ceremony participants and .INT domain registrants were invited to take the survey. This year, ICANN issued 3,710 invitations with a five percent response rate. About ICANN ICANN's mission is to help ensure a stable, secure and unified global Internet. To reach another person on the Internet, you need to type an address – a name or a number – into your computer or other device. That address must be unique so computers know where to find each other. ICANN helps coordinate and support these unique identifiers across the world. ICANN was formed in 1998 as a not-for-profit public-benefit corporation with a community of participants from all over the world. Manage Your Subscription [app.icontact.com] This message was sent to naela.sarras@icann.org from no-reply@external.icann.org ICANN News Alert ICANN 12025 Waterfront DriveSuite 300 Los Angeles, CA 90094-2536 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [icontact.com]