Good to see this.

Some comments:

1. Some typos/grammar bits that need to be fixed.
2. Are you expecting to include actual names of application in the eventual report?
3. Would be good to get an assessment of over-arching deficits – issues with POP and IMAP – reference to RFC6856 – put these into the executive summary.   Perhaps an appendix on that particular RFC, it’s history, and why it may not yet have been widely adopted.
4. The amount of effort to perform a complete test still seems to be very high to me – 80 hours + admin efforts.   I would like to see a recommendation for a faster, less exhaustive, testing criteria that may find problems more quickly.
5. It will be good to share the evaluation criteria with the application developers so that they can, perhaps, self-certify.
6. Could the testing be structured for crowd sourcing or some other markedly lower cost transaction effort?   That’s a question for the measurement group but they’ll need input from Evan and team.
7. I would like to see an Executive Summary.  Something akin to ‘We tested these three applications and apart from a lack of support of the LANG feature in RFC6856 all performed well.   Or didn’t.

From: UA-discuss <ua-discuss-bounces@icann.org> On Behalf Of Sarmad Hussain
Sent: Tuesday, 6 August 2019 11:41 PM
To: Universal Acceptance <ua-discuss@icann.org>
Subject: [UA-discuss] Evaluation of Major Email Software and Services for EAI Support

Dear All,

UASG has undertaken an evaluation of email software and services for EAI support, as per the details below.  The report and associated documentation is being shared for your review and feedback, before it is finalized and published.

The first phase of this project, Discovery and Analysis, was completed September 2018 and published on the UASG’s website as UASG021A. That initial phase selected a broad set of components from the email software ecosystem to be analyzed, defined the test criteria to use when evaluating a particular piece of software, and provided an estimate of the effort required to evaluate a select few representative software and service offerings from that set. The resulting Discovery and Analysis Report and accompanying test criteria can be found on the UASG’s website.

The second and current phase of this project comprises a Proof of Concept Evaluation. A targeted set of software and service providers have been evaluated according to the criteria detailed by UASG021A. The purpose of this phase is to:

1. Confirm whether the evaluation criteria developed in the UASG021A report are fair and reasonable, amending or clarifying them where necessary,
2. Develop a template for reporting future test results, and
3. Determine the amount of effort required to evaluate additional email software and service providers.

The third and final phase of the project will provide a more complete evaluation of the email software ecosystem, including a larger number of components and informed by the experience of this pilot.

Please share your feedback at ua-discuss@icann.org by 25 August 2019.  Please visit https://uasg.tech/subscribe to self-scribe to this email list.

Regards,
Sarmad