Dear Sub Team Members, Please see below the action items and discussion notes captured by staff from the meeting on 31 August. These high-level notes are designed to help Work Track Sub Team members navigate through the content of the call and are not meant to be a substitute for the recording. Please also see the recording on the meetings page at: https://community.icann.org/display/NGSPP/Work+Track+4+Meetings. Note also that the referenced slides for today’s meeting are attached and excerpts from the chat room are included below. Best regards, Julie Julie Hedlund, Policy Director Action Items/Discussion Notes 31 August Application Evaluation (continued) Non-scored questions (slide 5) -- Is that a theme for WT4: -- Apparent consensus: Forward to Q18 to WT2 (incorporation into agreement) and WT3 (objections and advices) -- Keep Q23 with WT4 (Registry Services)

From the chat:

Trang Nguyen: Note that Q18, although not scored, is used by the Tech/Fin panel as context for those evals. Registry Services (slide 6) -- Mainly used as a means to collection information to build "Exhibit A" (Approved Services in registry contracts -- Probably less useful when technical evaluation is done in bulk or not done at all (RSP Program) -- Undergooing discussions might streamline registry service adoption Registry Services -- Straw-person (slide 7) -- Applicants allowed but not required to specify additional registry services [reference slide for details] -- Services evaluated through RSEP -- If no additional services, RSEP will be used after contract signing Discussion: -- Good middle ground.  Only concern is with the technical streamlining: how this affects the order of processing of applications.  Could it somehow affect how we evaluate applications for the same string?  Have we thought about how this middle ground would affect identical strings but with different services? -- All possible applicants to the same strings will follow the same path.  Doesn't change the order of processing even if one applicant has proposed different services for one string. -- So, the description of the additional services will be described in the application so that would be transparent.  There would an award before the services are approved? -- Two issues: First, it is not clear that an applicant will be able to answer the questions associated with RSEP at the time of application.  It may not be practical at that time.  If this is really an ancillary service that the registry can do or not depending on how the RSEP is treated that could be find.  But if this is a service that the registry has to provide because of its business model, then that is problematic.  Also, in the normal RSEP process ICANN gives a quick decision, then there is a public comment process, and then maybe a panel.  Not clear that it is feasible for the registry to wait to go through that process.  I think they need a heads up earlier. -- If you want the fast track to contractual delegation then don't propose any additional services, but some applicants might want those services evaluated before contention sets. -- We are looking for innovation.  There could be a case where if we are looking for new ideas I'm not sure this supports that direction.  That is an issue that needs to be considered. -- An applicant can describe additional services, but is not obliged to.  Maybe move to an RSEP light. -- In the 2012 round there was a separate registry services evaluation.  That evaluation had a set of criteria that was used.  What happened was at contracting time for the purpose of pulling together exhibit A of the contract we went through the process again, which was not very efficient.  The registry services were provided in question 23 of the application.  That was not written in a way that it could be used as exhibit A.  There are some improvements that could be made there.  If you waited until you signed a contact and then applied for a new service it went through the RSEP process.  Doesn't make sense for it to go through a different process before and after the contract is signed. -- Nothing wrong with doing a segmented process on new applications, but the process described above is reasonable because it is fair because if something should fail the applicant should know early.  Perhaps we should ask if this is a service that is necessary and would they withdraw the application if it is not approved.  I don't think we gave the public an opportunity to comment on these registry services.  -- Devil's advocate: The slowness of the process discouraged innovation.  Speeding up the process could encourage innovation. -- Could be the failure to identify and address the root causes of the delays.  When you are working on remedies for that you have different types of remedies.  One could be not enough qualified staff to process applications.  If you want to encourage innovation you will find a way to train and hire qualified people and you will address the issue of how contract evaluations get done without disfavoring those who provide these additional services. -- Is there a different way we could look at this.  Not just speed (fast or slow) but about efficiency.  If the process is more efficient it will benefit everyone.  Instead of doing things twice let's figure out if we can do the process once and then figure out where that would happen -- before contention resolution or after. -- In 2012 the majority were services that were already being offered so having them go through an additional evaluation is not very efficient. -- Agree: more about efficiency than speed.  One of the problems was that every time someone suggested something new or different members of the community objected.  The fact that things need to go back to an IRT that can result in incredible delays.  The fact that some brands proposed closed models the community said we hadn't thought about that.  Name collision issue also caused delays.  There is no one root cause as to why there were delays.  If we could try to address those by creating a more efficient application and evaluation process, and having a more solid predictability framework that doesn't rely on having long community discussions and processes.  But if we have and RFP program to approve them and a predictability framework these will help.  I think we need to not think about each thing in its own silo. -- Proposal that there are departments in ICANN one that processes contracts that don't have new services and one that processes contracts that do.  From the chat: [Anne Aikman-Scalese (IPC): PROPOSAL - ICANN Should have two sections for processing applications:  one would process applications that contain no new services proposals.  The other would handle applications which contain proposals for new services.  In this manner, they would proceed in parallel so that processing of applications with new services proceeds apace and is not disadvantaged by being considered later at the contracting phase.  The other problem with this is that applicants will come in with basic services in order to get approval and then raise new services later in contracting that may not be part of evaluation but maybe should have been part of evaluation.] [Move this discussion to the list.  Note that if we cannot come up with a consensus for change then the process will stay as it is.]

From the chat:

Jeff Neuman: @Rubens - I am not sure that was what I understood.  If you have contention resolution prior to evaluating the new services, then in theory if there are auctions, the  participant of the auction that proposed the new ancillary service will be assuming that the service will be approved in determining how much it wants to bid.  IF the service is refused, that could be a huge hit on the winner Anne Aikman-Scalese (IPC): @Jeff - I am very concerned about streamlining and granting early contracting to applicants that are not offering new services.  Agree with Alan we are looking for innovation and new ideas.  This proposal does not support that direction. hil Buckingham: @ Trang  I agree.. The question is whether to drop Q18 . What happens if an applicant wishes to change their mission statement ( Q18 ) post acheiving a financial / technical evaluation pass . Should the evaluations need to be redone based on the new mission statement . Jeff Neuman: Because the last process took years for applications to be evaluated, approved, contracted, etc.....those that did want to innovate either had huge staff turnover, lost budget, lost staff, lost resources, etc. between the application submission in 2012 and when they were finally approved/contracted in 2015/2016 Jeff Neuman: @Anne - you are assuming a limited 3rd party registration model Jeff Neuman: you need to think bigger than that Jeff Neuman: One of the biggest reasons I have found that brands have not launched or have not launched as big as we all throught they would, is the time lage between app submission and approval Jeff Neuman: all of the delays took its toll Anne Aikman-Scalese (IPC): Odd refference to my type of thinking Jeff - please be more specific - it doesn't help to just say my thinking is somehow not big enought for you. Cheryl Langdon-Orr (CLO): indeed Jeff Jeff Neuman: So I would argue that the slowness discouraged innovation greatly Anne Aikman-Scalese (IPC): @Jeff - so the problem is how quickly we process applications - it may be about  not having enough qualified conttractting staff that is temporary.  The root cause could be something other than you say. Cheryl Langdon-Orr (CLO): yes Alan Rubens Kuhl: Personal opinion: brands are wating for Google to use non-legacy TLDs. ;-) bens Kuhl: Personal opinion: brands are wating for Google to use non-legacy TLDs. ;-) Alan Greenberg: @Jeff, that is exactly the type of registry that may NEED a special service. Jeff Neuman: There were a lot of factors they caused the delays.  There was not just one root cause Jeff Neuman: @Anne - I disagree that that is the root cause avri doria: i would probably arrgue that there are several contenders for root cause. Rubens Kuhl: Amdahl's law: we can only increase the efficiency of the scope we are looking at. Cheryl Langdon-Orr (CLO): thanks Trang Rubens Kuhl: .frogans was also something a bit different. Anne Aikman-Scalese (IPC): I believe that proposed new services are part of evaluation.  We should have dual simultaneous tracks - not just delay all new services to later contracting. avri doria: yes, exception processing, of all sorts, was a time sink. Rubens Kuhl: TM+50 also caused a delay. Sarah L Verisign: Speaking of the new CEO (at the time) I remember when Fadi Chehade stood up at the time and said that the program wasn’t ready, in his opinion, but he was being told by the community to push ahead regardless (paraphrase) and that is what happened.  I think there is a concern that when we put expediency ahead of efficiency and general readiness there are going to be delays – things don’t go as expected for any number of reasons which is why it’s so important to get it closer to general readiness this time. Cheryl Langdon-Orr (CLO): thanks Sarah,  Jeff Neuman: @Sarah - Fadi claimed it wasnt ready because he believed that there should be a unilateral right to amend registry contracts and a way to incorporate "public interest commitments".  I do not believe he claimed it was because ICANN staff  and third party providers were not ready to accept applications and evaluate them Jeff Neuman: @Sarah - and developing a predictability framework as we discussed on Monday is key to ensuring there is a process to handle the new things when they do arise Sarah L Verisign: What he said was... . "Honestly, if it was up to me, I would delay the whole release of new gTLDs by at least a year." 2. "… a lot of the foundations that I would be comfortable with, as someone who has built businesses before, are just not yet there." 3. "We have people who took six years to write the [new gTLD Applicant] Guidebook and we're asking engineers and software people and third-party vendors and hundreds of people to get that whole program running in six months." Anne Aikman-Scalese (IPC): PROPOSAL - ICANN Should have two sections for processing applications:  one would process applications that contain no new services proposals.  The other would handle applications which contain proposals for new services.  In this manner, they would proceed in parallel so that processing of applications with new services proceeds apace and is not disadvantaged by being considered later at the contracting phase.  The other problem with this is that applicants will come in with basic services in order to get approval and then raise new services later in contracting that may not be part of evaluation but maybe should have been part of evaluation. Donna Austin, Neustar: Agree with Jeff, that its important that context is taken into account particularly when quoting Fadi, also important to understand who is audience was. Sarah L Verisign: @Donna and @Jeff, I do agree that  his comments should be considered in context are important but it isnt correct just to say they were contractual or framework related. Anne Aikman-Scalese (IPC): Thanks Sarah  - I think that if you have two ramps up front - you have a much better "middle ground".  That way extra services are part of evaluation phase (not just contracting.) and these proceed as quickly as possible. avri doria: or an option to get an RSEP-like preeval before  needing to pay for any other exception processing. (exception = objection, contentions, &c.) Sarah L Verisign: I like Anne's suggestion that there should be an "off-ramp" for applications that have exceptional requirements so as not to delay more standard applications Phil Buckingham: @ Anne , I agree the need  for two sections . Perhaps there should be an inbuilt time factor. Applicant have to wait 2 years ( say) post launch  before new proposals for  new services can be proposed . That would require a full re-evaluation  of the new business model. Jeff Neuman: @sarah and @Anne - if there is an off ramp, to the extent that we do rounds, that will be an off ramp for all applications in that contention set Technical Questions -- Security Policy (slide 8) Straw-person: -- Keep Q30a (summary of security policy) -- Remove Q30b (full security policy) Draft Language (slide 9) Discussion: -- Flag that WT4 feedback on the technical evaluation criteria.  Some of the responses did touch on this.  Touch on some time of an RSP Program.  Also suggested some additional enhancements on certification of security practices.