[rssac-caucus] FYI: CDAR Study Cannot Predict Stability of the Root Server System
Fellow Caucus Colleagues, I have a personal request for comments: After some reflection I plan to submit the following comment about the CDAR study. I would appreciate any comments or suggestions you might have. Personal e-mail welcome. Thank you for your time and consideration. Daniel ------ To whom it may concern, The proposed core methodology of the study is a "quantitative model" based on measurement of past behavior of the DNS root server system. Such models of complex systems are by nature simplifications. They are indeed very useful to identify possible instabilities in the real-world system they are modeled after. However these models cannot predict the *absence* of instabilities in the real-world system with any useful level of confidence. In other contexts this is referred to as "Past Performance is Not Necessarily Indicative of Future Results". Our main comment and advice to the researchers is to carefully avoid any perception that their results predict the absence of instabilities unless the results solidly support such claims. Our main comment and advice to ICANN is not to expect the study to predict the absence of instabilities in the DNS root name server system that are wholly or partly caused by root zone expansion. ICANN therefore should make proper contingency plans for the unpredictable cases where root zone expansion causes or contributes to instabilities in the DNS root sever system. Research is most useful and should be called scientific only when it can be verified and possibly reproduced by other researchers and scientists. Therefore we support the researchers intention to base their study on openly available measurement data and meta-data wherever possible. Where this is not feasible the observations and meta-data should at least be available to other researchers under reasonable conditions. Daniel Karrenberg Chief Scientist RIPE NCC, not speaking on behalf of the RIPE NCC Relevant experience: 30+ years of operational experience in DNS; 15+ years of operational experience and responsibility for k.root-servers.net; contributor to DNS software diversity by instigating and helping with design and implementation of NSD 1.0, an authoritative name server; inventor and principal implementor of 'dnsmon', the first comprehensive and public monitoring system for high level DNS servers; co-inventor and responsible for the initial deployment of RIPE Atlas, the active measurement system proposed to be used in the study; ... . ICANN entanglements: member of ICANN RSSAC since its inception and not speaking on behalf of RSSAC either.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Fellow Caucus Colleagues, thank you for your consideration and comments. For your information I attach the comment I ended up submitting. In case you are interested in the comments process and other comments submitted, go to https://www.icann.org/public-comments/cdar-study-plan-2015-12-02-en Daniel - ------ - -------- Forwarded Message -------- From: Daniel Karrenberg <daniel.karrenberg@ripe.net> To: comments-cdar-study-plan-02dec15@icann.org Subject: CDAR Study Cannot Predict Stability of the Root Server System Message-ID: <56A75BBF.90107@ripe.net> Date: Tue, 26 Jan 2016 12:42:55 +0100 To whom it may concern, Research is most useful and should be called scientific research only when it can be verified and possibly reproduced by other researchers and scientists. Therefore we support the researchers' intention to base their study on openly available measurement data and meta-data wherever possible. Where this is not feasible the observations and meta-data should at least be available to other researchers under reasonable conditions. The DNS root name server system is a complex system with hundreds of servers, tens of thousands of clients and millions of users all connected by the open Internet and subject to unpredictable use and abuse. The proposed core methodology of the study is a "quantitative model" based on measurement of past behavior of the DNS root server system. Models of complex systems are by their nature simplifications. Well constructed models may indeed be very useful to predict possible instabilities in the real-world systems. However these models cannot predict the *absence* of instabilities in complex real-world system with any useful level of confidence. In other contexts this is referred to as "past performance is not necessarily indicative of future results". Our main comment and advice to the researchers is to carefully avoid any perception that their results predict the absence of instabilities in the DNS root name server system unless the results solidly support such claims. Our main comment and advice to ICANN is: do not expect the study to predict the absence of instabilities in the DNS root name server system including absence of instabilities that may be wholly or partly caused by root zone expansion. ICANN therefore must make proper contingency plans for the unpredictable cases where root zone expansion causes or contributes to instabilities in the DNS root sever system. Daniel Karrenberg Chief Scientist RIPE NCC, not speaking on behalf of the RIPE NCC - ---- Some relevant experience: 30+ years of operational experience in DNS; 15+ years of operational experience and responsibility for k.root-servers.net; contributor to DNS software diversity by instigating and helping with design and implementation of NSD 1.0, an authoritative name server; inventor and principal implementor of 'dnsmon', the first comprehensive and public monitoring system for high level DNS servers; co-inventor and responsible for the initial deployment of RIPE Atlas, the active measurement system proposed to be used in the study. ICANN entanglements: member of ICANN RSSAC since its inception and not speaking on behalf of RSSAC either. On 21.01.16 16:54 , Daniel Karrenberg wrote:
Fellow Caucus Colleagues,
I have a personal request for comments: After some reflection I plan to submit the following comment about the CDAR study. I would appreciate any comments or suggestions you might have. Personal e-mail welcome.
Thank you for your time and consideration.
Daniel -----BEGIN PGP SIGNATURE-----
iQIcBAEBCAAGBQJWp2StAAoJENDY5Emqa716jlYP/3cHJWn8P/bAISn72q6sVxWt NC4mUAoeyRVtyZDvlRTx0sxMccvtmRUyq5rifSL4ox77iiasqHhgpsfwvXP8s887 J9j/4PE7dRaAdh7f2nORWf5TptoOLEq0+p1EeU+4WWRT+wBvx8KYWob7ObdjndZp 7f3BZzJAGMMhkIZfVwX+TsriT0OQaBJqvdDsgCIbV91tVNcmk+TC0cGCIYmyMIl9 h4IUDnxKM5hhkSx963loXBUvfkL8BrS7ywF3IrE4wNWEP0m135IiUX+Y4562KTBk DsliMLxSgRLOv+tzv1c14yT2+xefrBei/eZwOuqR67cbACCqTNUtOxeR26J0NdF4 NPKllJU8CZjUXYb7Cxj3Nf5SIlWrLNkbcA0Ib7ET98fQverGd+aG+cpqAKPSX1cQ eNXVXa6XrvBKuEa1y3sbONjVFOLk/WXD4eC1OF5JJvSnJ02VIvq+mS3pulB0+Riv KDOCAwDZWrxcIQpJfxqhEPlHuaJJpfDrm6ZRsXA2xZ6Jh/mKEqbW7+jaBJYL6n0y m7osgwJjNNEmKCFEkgwvAFHrGEyGxuQSHidQIGCrZYK0zmpiCX/rhMvtIswhN4Ky 7MTZ9qde620Ja1WBJgDnJjEayv4LSTpTvo0vugkZBoOPDjElVasA9YFjfsDVE9Ry AGq0GvxsY2k7QuL5WUi5 =85gN -----END PGP SIGNATURE-----
participants (1)
-
Daniel Karrenberg