I will add the observation that while we have a good rapport with larger RSPs, which tend to be the driver of most peaks in requests, under the current relationship there is a direct incentive to do so. As we do not have the concept of bulk operations in our current system, an RSP submitting a change impacting a few hundred TLDs will need to submit several hundred change requests to effect the change. We make the offer to TLD operators that we will manage bulk operations on their behalf by performing the repetitive submission process for them, as well as bulk-processing authorizations out-of-band when they are all by the same party. This dynamic could change as our next generation root zone management platform will offer API access which should make the submission and approval process largely automatable by RSPs with large portfolios. I’ll also add another driver of bulk operations is policy changes so we work closely with our colleagues in the broader ICANN Org to mitigate any potential peaks when new policies are being implemented. As to how we implement these, we certainly do plan ahead to make sure our staffing can cope with the anticipating load by these activities. We have a small team, and just one person being on vacation in a given week could be problematic. With usually at least a few weeks notice before these kinds of bulk operations are anticipated it allows is to confirm staffing, defer non-essential project work, bring in extra resources and so forth to manage the surge. kim From: IFRT <ifrt-bounces@icann.org> on behalf of James Gannon <james@cyberinvasion.net> Date: Wednesday, June 3, 2020 at 6:45 AM To: Peter Koch <pk@DENIC.DE>, "Wilhelm, Richard via IFRT" <IFRT@icann.org> Subject: Re: [IFRT] peak loads by large 'routine' operations I know this has been discussed between the CSC and PTI, and the feedback we have gotten is that the relationship between PTI and the larger RSPs allow them to plan for bulk changes and updates in advance so that the teams are able to align on workloads for such events etc. On 6/3/2020 2:32:56 PM, Peter Koch <pk@denic.de> wrote: Dear fellow IFRT members, I'd like to share an observation that is rather operational and may or may not be within the IFR scope (and might also not be new). It shouldn't fall into a void, so here it is: With approx. 1500 TLDs the net has seen some concentration, so that a single entity is responsible for or providing technical services to multiple TLDs, be that 100, 200 or more. Sometimes, contact details change, sometimes technical parameters change (like nameserver names or addresses) and in some cases parameters are changed regularly (DNSSEC key rollover). I've seen this happen in larger bundles and am wondering whether there is any noticable effect on workloads or infrastructure and how that might change if the bundles grow larger. I'm pretty confident everybody involved is aware of scaling effects and given that it's routine operation, no immediate risk should exist. Best regards, Peter _______________________________________________ IFRT mailing list IFRT@icann.org https://mm.icann.org/mailman/listinfo/ifrt _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.