Proposed Update CSC Findings Qualifications “Excellent” and “Satisfactory”

In 2016 the CSC and PTI agreed on standardized language to qualify the performance of PTI. Recently the CSC requested to clarify the qualification of PTI’s performance. The CSC and PTI recently agreed to additionally include a reference in PTI’s monthly report to the number of Thresholds/SLAs PTI has met out of total number of Thresholds/SLA.  At its April meeting the CSC agreed to include a reference in their monthly Findings of PTIs Performance and change the standard language to that effect. Going forward, starting in May 2020 following generic language is proposed to designate the findings of the CSC of PTI’s performance. The changes are marked and the Original language is included  below

• Excellent- PTI’s performance over [MONTH] was 100%. PTI met all [insert number] of the currently defined thresholds over [MONTH]. The current list of thresholds is included in PTI’s performance report.

• Satisfactory- PTI’s performance over [MONTH] was [insert %]. PTI met [insert number] of the currently defined [INSERT TOTAL NUMBER] thresholds.  Missing the threshold [INSERT NAME OF THRESHOLD] was satisfactorily explained and not an indication of a performance issue. [ADDITIONAL NOTE, Explaining]. The current list of thresholds is included in PTI’s performance report.

• Needs Improvement - Performance needs improvement due to a) severe degradation in meeting the number of thresholds from previous months, b) a trend in complaints that indicate a persistent issue to be resolved, or c) a negative trend in compliance to thresholds over several months. The current list of thresholds is included in PTI’s performance report.

Original Language

• Excellent - PTI met all service level agreements (100 %) for the month.

• Satisfactory - PTI met the service level agreement for [less than 100%] of defined metrics. Missed service level agreements were satisfactorily explained and the CSC has determined that these exceptions were no cause for concern. No persistent problems were identified and no further action is needed.
• Needs Improvement - Performance needs improvement due to a) severe degradation in meeting SLAs from previous months, b) a trend in complaints that indicate a persistent issue to be resolved, and c) a negative trend in compliance to SLAs over several months.

Applying the proposed qualification Using the proposed language in recent CSC Findings report, they would:

• Excellent- PTI’s performance over March 2020 was 100%. PTI met all [65] of the currently defined thresholds over March 2020. The current list of thresholds is included in PTI’s performance report.

• Satisfactory- PTI’s performance over January 2020 was 98.5%. PTI met 64 of the currently defined 65 thresholds. Missing the threshold Validation and Reviews (ccTLD Creation/Transfer) was satisfactory explained and not an indication of a performance issue. This threshold is subject to a CSC recommendation to introduce new measurements to create a more meaningful performace measurement for this process. The current list of thresholds is included in PTI’s performance report.

Examples of Recent qualifications in CSC Findings of PTI Performance

• Findings February 2020: Excellent- PTI met the service level agreement at 100% for the month of February 2020.

• Findings January 2020: Satisfactory- PTI met the service level agreement at 98.5% for the month of January 2020.

Validation and Reviews (ccTLD Creation/Transfer): This missed service level is subject to a CSC recommendation to introduce new measurements to create a more meaningful performace measurement for this process.

• Needs Improvement – No example. Has never been used to date.