[rssac-caucus] FOR REVIEW: DRAFT RSSAC002 Version 2
Dear RSSAC Caucus, Attached please find RSSAC002 version 2 for your review. In this version, we made the following changes based on the RSSAC caucus discussions led by Duane: * Section 2.2 (The size of the overall root zone) was amended to clarify that TCP size prefix octets are not included in the metric. * Section 2.4 (The query and response size distribution) was amended to clarify that TCP size prefix octets are not included in these metrics. * Section 2.4 was amended to include 0-15 in size ranges to be tabulated. * Superfluous quotes around YAML keys were removed from example YAML in sections 4.1 (The ‘load-time’ Metric) and 4.2 (The ‘zone-size’ Metric). * Indentation was fixed for example YAML in sections 4.3 (The ‘traffic-volume’ Metric) and 4.6 (The ‘unique-sources’ Metric). * Section 4.5 (The ‘rcode-volume’ Metric) was amended to clarify that nonzero counts should be omitted. In addition, we made the following organizational changes: * Make this document RSSAC002 version 2 * Added a Revision History section (see section 7). Attached please find the redline, clean word and PDF version. Since the proposed revisions have already been discussed and agreed on the caucus list, we will give this update a 7 day additional review period. Please provide your feedback, if any, no later than 23 November 2015. After that, this document will be sent to RSSAC for formal action. Best, Steve
Do you need a +1 or silence is enough to show that we agree on the changes. Regards as On Tue, 17 Nov 2015 at 12:05 Steve Sheng <steve.sheng@icann.org> wrote:
Dear RSSAC Caucus,
Attached please find RSSAC002 version 2 for your review. In this version, we made the following changes based on the RSSAC caucus discussions led by Duane:
- Section 2.2 (The size of the overall root zone) was amended to clarify that TCP size prefix octets are not included in the metric. - Section 2.4 (The query and response size distribution) was amended to clarify that TCP size prefix octets are not included in these metrics. - Section 2.4 was amended to include 0-15 in size ranges to be tabulated. - Superfluous quotes around YAML keys were removed from example YAML in sections 4.1 (The ‘load-time’ Metric) and 4.2 (The ‘zone-size’ Metric). - Indentation was fixed for example YAML in sections 4.3 (The ‘traffic-volume’ Metric) and 4.6 (The ‘unique-sources’ Metric). - Section 4.5 (The ‘rcode-volume’ Metric) was amended to clarify that nonzero counts should be omitted.
In addition, we made the following organizational changes:
- Make this document RSSAC002 version 2 - Added a Revision History section (see section 7).
Attached please find the redline, clean word and PDF version. Since the proposed revisions have already been discussed and agreed on the caucus list, we will give this update a 7 day additional review period. Please provide your feedback, if any, no later than 23 November 2015. After that, this document will be sent to RSSAC for formal action.
Best, Steve _______________________________________________ rssac-caucus mailing list rssac-caucus@icann.org https://mm.icann.org/mailman/listinfo/rssac-caucus
Hi Jaap, Monday, 23 November 2015 is the deadline. Best Steve On 11/18/15, 1:42 AM, "Jaap Akkerhuis" <jaap@NLnetLabs.nl> wrote:
Arturo Servin writes:
Do you need a +1 or silence is enough to show that we agree on the changes.
I first want to have some time to read it properly. Is tehere a drop death date for reactions?
jaap
Thanks Arturo, +1 would be appreciated! Best Steve From: Arturo Servin <arturo.servin@gmail.com<mailto:arturo.servin@gmail.com>> Date: Tuesday, November 17, 2015 at 7:46 PM To: Steve Sheng <steve.sheng@icann.org<mailto:steve.sheng@icann.org>>, "rssac-caucus@icann.org<mailto:rssac-caucus@icann.org>" <rssac-caucus@icann.org<mailto:rssac-caucus@icann.org>> Subject: Re: [rssac-caucus] FOR REVIEW: DRAFT RSSAC002 Version 2 Do you need a +1 or silence is enough to show that we agree on the changes. Regards as On Tue, 17 Nov 2015 at 12:05 Steve Sheng <steve.sheng@icann.org<mailto:steve.sheng@icann.org>> wrote: Dear RSSAC Caucus, Attached please find RSSAC002 version 2 for your review. In this version, we made the following changes based on the RSSAC caucus discussions led by Duane: * Section 2.2 (The size of the overall root zone) was amended to clarify that TCP size prefix octets are not included in the metric. * Section 2.4 (The query and response size distribution) was amended to clarify that TCP size prefix octets are not included in these metrics. * Section 2.4 was amended to include 0-15 in size ranges to be tabulated. * Superfluous quotes around YAML keys were removed from example YAML in sections 4.1 (The ‘load-time’ Metric) and 4.2 (The ‘zone-size’ Metric). * Indentation was fixed for example YAML in sections 4.3 (The ‘traffic-volume’ Metric) and 4.6 (The ‘unique-sources’ Metric). * Section 4.5 (The ‘rcode-volume’ Metric) was amended to clarify that nonzero counts should be omitted. In addition, we made the following organizational changes: * Make this document RSSAC002 version 2 * Added a Revision History section (see section 7). Attached please find the redline, clean word and PDF version. Since the proposed revisions have already been discussed and agreed on the caucus list, we will give this update a 7 day additional review period. Please provide your feedback, if any, no later than 23 November 2015. After that, this document will be sent to RSSAC for formal action. Best, Steve _______________________________________________ rssac-caucus mailing list rssac-caucus@icann.org<mailto:rssac-caucus@icann.org> https://mm.icann.org/mailman/listinfo/rssac-caucus
Section 4.1, second paragraph, "The latency in the distribution system" needs to be changed to "Latency in publishing available data". Section 3, third bullet - is this still true? If this was for NSD, it has been corrected. Section 2.1 - For v3 of the document I would like to see more details on how this value should be computed, for the sake of consistency. For example, I have six servers that receive notifies directly from the distribution masters. Should the load-time value be computed on each server separately and then averaged (this is what I am currently doing)? Or should the notify and load times from all servers be collected centrally and the value computed by subtracting the earliest (first received) notify time from the latest/last load time? What if a notify is never received (by an individual server in the first case or by any of the servers in the second case)? Section 2.1, second paragraph - "may present as anomalies in measurements", perhaps change to "may introduce anomalies"? Section 4.1, "If not load-time metric is available...", change "not" to "no". Section 5, make title plural (Recommendations) Thanks, Howard
Hi Howard, On 19/11/2015 12:24, "rssac-caucus-bounces@icann.org on behalf of Kash, Howard M CIV USARMY RDECOM ARL (US)" <rssac-caucus-bounces@icann.org on behalf of howard.m.kash.civ@mail.mil> wrote:
Section 2.1 - For v3 of the document I would like to see more details on how this value should be computed, for the sake of consistency. For example, I have six servers that receive notifies directly from the distribution masters. Should the load-time value be computed on each server separately and then averaged (this is what I am currently doing)? Or should the notify and load times from all servers be collected centrally and the value computed by subtracting the earliest (first received) notify time from the latest/last load time? What if a notify is never received (by an individual server in the first case or by any of the servers in the second case)? This is documented as
"...Therefore measurements may only represent the average for 95% of the operationally-active instances of a root server per root zone serial." So what you are currently doing is correct. Thanks John
On 11/19/15, 4:24 AM, "rssac-caucus-bounces@icann.org on behalf of Kash, Howard M CIV USARMY RDECOM ARL (US)" <rssac-caucus-bounces@icann.org on behalf of howard.m.kash.civ@mail.mil> wrote:
Section 4.1, second paragraph, "The latency in the distribution system" needs to be changed to "Latency in publishing available data".
Agre.
Section 3, third bullet - is this still true? If this was for NSD, it has been corrected.
It might still be true for operators who have not updated to the latest NSD.
Section 2.1 - For v3 of the document I would like to see more details on how this value should be computed, for the sake of consistency. For example, I have six servers that receive notifies directly from the distribution masters. Should the load-time value be computed on each server separately and then averaged (this is what I am currently doing)? Or should the notify and load times from all servers be collected centrally and the value computed by subtracting the earliest (first received) notify time from the latest/last load time? What if a notify is never received (by an individual server in the first case or by any of the servers in the second case)?
Agree that this would be a useful expansion in version 3.
Section 2.1, second paragraph - "may present as anomalies in measurements", perhaps change to "may introduce anomalies"?
Both are true, so maybe let's keep it as-is.
Section 4.1, "If not load-time metric is available...", change "not" to "no".
Good catch!
Section 5, make title plural (Recommendations)
Sure. --Paul Hoffman
On Nov 19, 2015, at 4:24 AM, Kash, Howard M CIV USARMY RDECOM ARL (US) <howard.m.kash.civ@mail.mil> wrote:
Section 4.1, second paragraph, "The latency in the distribution system" needs to be changed to "Latency in publishing available data".
Yes.
Section 3, third bullet - is this still true? If this was for NSD, it has been corrected.
Not sure if its still true. We could poll the operators. I see little harm in leaving it as-is for now.
Section 2.1 - For v3 of the document I would like to see more details on how this value should be computed, for the sake of consistency. For example, I have six servers that receive notifies directly from the distribution masters. Should the load-time value be computed on each server separately and then averaged (this is what I am currently doing)? Or should the notify and load times from all servers be collected centrally and the value computed by subtracting the earliest (first received) notify time from the latest/last load time? What if a notify is never received (by an individual server in the first case or by any of the servers in the second case)?
This sounds to me like more than a simple clarification and probably worthy of a work party. So as you say, a good item for "v3".
Section 2.1, second paragraph - "may present as anomalies in measurements", perhaps change to "may introduce anomalies"?
How would you feel about "may appear as anomalies"? To me "introduce" sounds just a little too much like blaming anycast for causing a problem. Or maybe since this is the same section as the above item, it will likely all get rewritten in v3 so we save it until then?
Section 4.1, "If not load-time metric is available...", change "not" to "no"
Yes.
Section 5, make title plural (Recommendations)
Yes. Thanks!
Hi Howard, Thank you for these comments. Based on the ensuing mailing list discussion. The following changes were made to the document: Section 4.1, second paragraph, "The latency in the distribution system" changed to "Latency in publishing available data". Section 2.1, second paragraph - "may present as anomalies in measurements", changed to "may be present as anomalies in measurements”. Section 4.1, "If not load-time metric is available...", changed "not" to "no". Section 5, made title plural (Recommendations) The following feedback did not result in changes to the document, as these are either due for the next iteration of RSSAC002, or the current text is still valid. Section 3, third bullet - is this still true? If this was for NSD, it has been corrected. Section 2.1 - For v3 of the document I would like to see more details on how this value should be computed, for the sake of consistency. For example, I have six servers that receive notifies directly from the distribution masters. Should the load-time value be computed on each server separately and then averaged (this is what I am currently doing)? Or should the notify and load times from all servers be collected centrally and the value computed by subtracting the earliest (first received) notify time from the latest/last load time? What if a notify is never received (by an individual server in the first case or by any of the servers in the second case)? Attached please find the latest document incorporating these changes. The deadline to review is still close of business Monday 23 November. Best Steve On 11/19/15, 7:24 AM, "Kash, Howard M CIV USARMY RDECOM ARL (US)" <howard.m.kash.civ@mail.mil> wrote:
Section 4.1, second paragraph, "The latency in the distribution system" needs to be changed to "Latency in publishing available data".
Section 3, third bullet - is this still true? If this was for NSD, it has been corrected.
Section 2.1 - For v3 of the document I would like to see more details on how this value should be computed, for the sake of consistency. For example, I have six servers that receive notifies directly from the distribution masters. Should the load-time value be computed on each server separately and then averaged (this is what I am currently doing)? Or should the notify and load times from all servers be collected centrally and the value computed by subtracting the earliest (first received) notify time from the latest/last load time? What if a notify is never received (by an individual server in the first case or by any of the servers in the second case)?
Section 2.1, second paragraph - "may present as anomalies in measurements", perhaps change to "may introduce anomalies"?
Section 4.1, "If not load-time metric is available...", change "not" to "no".
Section 5, make title plural (Recommendations)
Thanks, Howard
Steve Sheng writes:
Among other things
Attached please find the latest document incorporating these changes. The deadline to review is still close of business Monday 23 November.
Triggered by questions from some people i had good look at the examples given. These seem to be made up and it shows. Example 4.3 "The `traffic volume' Metric" shows that there a total of 42447 requests while there where 148013 responses. That seems odd to me. Especially since the example 4.6 "The `unique sources' Metric" says that the responses should come from 2086124 ipv4 and 42941 ipv6 adresses. That seems very odd to me. Maybe we should change the examples to some actual (public available) data? BTW, the questions I got was whether there was any idea how accurate the RSSAC 002 numbers are. QUite often one sees that there are less responses then requests. This can of course come from dropped requests (rate limiting stuff etc.) but the reverse, more responses then request have also been seen. For UDP at a.root: <http://a.root-servers.org/rssac-metrics/raw/2015/11/traffic-volume/a-root-20...> (and days after that). For TCP, see <https://www-static.ripe.net/dynamic/rssac002-metrics/2015/04/traffic-volume/...>. One wonders how this can happen. Any idea? A suggestion was made that the actual answer/response packets where counted instead of DNS query/responses and fragmentation happened but that seems odd to me. Anyway, I woud still advice that for the new version of the dpocument some real data for the examples will be used. And of course, some answer to the questions asked will be appreciated. Regards, jaap
Hi I would agree, there are a couple of observations: 1) In a previous presentation at a DNS-OARC workshop I had discussed some issues in the RSSAC-002 data collated so far, but I thin this has to do more with implementation than flaw in the document. 2) Yes, some internally consistent - perhaps real-world - examples would be helpfull. On Mon, 23 Nov 2015, Jaap Akkerhuis wrote:
Steve Sheng writes:
Among other things
Attached please find the latest document incorporating these changes. The deadline to review is still close of business Monday 23 November.
Triggered by questions from some people i had good look at the examples given. These seem to be made up and it shows. Example 4.3 "The `traffic volume' Metric" shows that there a total of 42447 requests while there where 148013 responses. That seems odd to me. Especially since the example 4.6 "The `unique sources' Metric" says that the responses should come from 2086124 ipv4 and 42941 ipv6 adresses. That seems very odd to me. Maybe we should change the examples to some actual (public available) data?
BTW, the questions I got was whether there was any idea how accurate the RSSAC 002 numbers are. QUite often one sees that there are less responses then requests. This can of course come from dropped requests (rate limiting stuff etc.) but the reverse, more responses then request have also been seen. For UDP at a.root: <http://a.root-servers.org/rssac-metrics/raw/2015/11/traffic-volume/a-root-20...> (and days after that). For TCP, see <https://www-static.ripe.net/dynamic/rssac002-metrics/2015/04/traffic-volume/...>. One wonders how this can happen. Any idea? A suggestion was made that the actual answer/response packets where counted instead of DNS query/responses and fragmentation happened but that seems odd to me.
Anyway, I woud still advice that for the new version of the dpocument some real data for the examples will be used. And of course, some answer to the questions asked will be appreciated.
Regards,
jaap _______________________________________________ rssac-caucus mailing list rssac-caucus@icann.org https://mm.icann.org/mailman/listinfo/rssac-caucus
On Nov 23, 2015, at 4:51 AM, Jaap Akkerhuis <jaap@NLnetLabs.nl> wrote:
Triggered by questions from some people i had good look at the examples given. These seem to be made up and it shows. Example 4.3 "The `traffic volume' Metric" shows that there a total of 42447 requests while there where 148013 responses. That seems odd to me. Especially since the example 4.6 "The `unique sources' Metric" says that the responses should come from 2086124 ipv4 and 42941 ipv6 adresses. That seems very odd to me. Maybe we should change the examples to some actual (public available) data?
Jaap, I'm fine with having real data in the examle YAML documents. Would you be alright with waiting on this for v3 so that we can get v2 approved and published soon? DW
"Wessels, Duane" writes:
Jaap,
I'm fine with having real data in the examle YAML documents. Would you be alright with waiting on this for v3 so that we can get v2 approved and published soon?
I had assumed that putting real data won't be a lot of work, but I'm naive on these things. So yes, it is is going to delay the publication let's put it on the TODO list for the next version. jaap
"Kash, Howard M CIV USARMY RDECOM ARL (US)" writes:
Section 3, third bullet - is this still true? If this was for NSD, it has been corrected.
For NSD this is fixed in 4.12, in 4.1.4 it will now also be logged at a less verose loggiong level (on request of Anand, K-root). jaap
[One hand typing (broke my pinky) is hard, here it is again] Jaap Akkerhuis writes:
"Kash, Howard M CIV USARMY RDECOM ARL (US)" writes:
Section 3, third bullet - is this still true? If this was for NSD, it has been corrected.
For NSD this is fixed in 4.1.2, in 4.1.4 it will now also be logged at a less verose loggiong level (on request of Anand, K-root).
For NSD this is fixed starting with version 4.12, since 4.1.4 it will now also be logged at a less verbose log level (on request of Anand Buddhdev, Ripe-NCC). jaap
Hi, I've made some (purely editorial / readability) edits / comments. Feel free to incorporate or ignore. W On Tue, Nov 17, 2015 at 3:06 PM Steve Sheng <steve.sheng@icann.org> wrote:
Dear RSSAC Caucus,
Attached please find RSSAC002 version 2 for your review. In this version, we made the following changes based on the RSSAC caucus discussions led by Duane:
- Section 2.2 (The size of the overall root zone) was amended to clarify that TCP size prefix octets are not included in the metric. - Section 2.4 (The query and response size distribution) was amended to clarify that TCP size prefix octets are not included in these metrics. - Section 2.4 was amended to include 0-15 in size ranges to be tabulated. - Superfluous quotes around YAML keys were removed from example YAML in sections 4.1 (The ‘load-time’ Metric) and 4.2 (The ‘zone-size’ Metric). - Indentation was fixed for example YAML in sections 4.3 (The ‘traffic-volume’ Metric) and 4.6 (The ‘unique-sources’ Metric). - Section 4.5 (The ‘rcode-volume’ Metric) was amended to clarify that nonzero counts should be omitted.
In addition, we made the following organizational changes:
- Make this document RSSAC002 version 2 - Added a Revision History section (see section 7).
Attached please find the redline, clean word and PDF version. Since the proposed revisions have already been discussed and agreed on the caucus list, we will give this update a 7 day additional review period. Please provide your feedback, if any, no later than 23 November 2015. After that, this document will be sent to RSSAC for formal action.
Best, Steve _______________________________________________ rssac-caucus mailing list rssac-caucus@icann.org https://mm.icann.org/mailman/listinfo/rssac-caucus
Hi all, I probably missed the deadline, but I read the document and the only thing caught my attention was section 2.5 -> I would move the text from 4.5 (or add short note) about extended RCODEs to 2.5. Cheers, Ondrej -- Ondřej Surý -- Technical Fellow -------------------------------------------- CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC Milesovska 5, 130 00 Praha 3, Czech Republic mailto:ondrej.sury@nic.cz https://nic.cz/ -------------------------------------------- ----- Original Message -----
From: "Steve Sheng" <steve.sheng@icann.org> To: rssac-caucus@icann.org Sent: Tuesday, November 17, 2015 9:04:30 PM Subject: [rssac-caucus] FOR REVIEW: DRAFT RSSAC002 Version 2
Dear RSSAC Caucus,
Attached please find RSSAC002 version 2 for your review. In this version, we made the following changes based on the RSSAC caucus discussions led by Duane:
* Section 2.2 (The size of the overall root zone) was amended to clarify that TCP size prefix octets are not included in the metric. * Section 2.4 (The query and response size distribution) was amended to clarify that TCP size prefix octets are not included in these metrics. * Section 2.4 was amended to include 0-15 in size ranges to be tabulated. * Superfluous quotes around YAML keys were removed from example YAML in sections 4.1 (The ‘load-time’ Metric) and 4.2 (The ‘zone-size’ Metric). * Indentation was fixed for example YAML in sections 4.3 (The ‘traffic-volume’ Metric) and 4.6 (The ‘unique-sources’ Metric). * Section 4.5 (The ‘rcode-volume’ Metric) was amended to clarify that nonzero counts should be omitted.
In addition, we made the following organizational changes:
* Make this document RSSAC002 version 2 * Added a Revision History section (see section 7).
Attached please find the redline, clean word and PDF version. Since the proposed revisions have already been discussed and agreed on the caucus list, we will give this update a 7 day additional review period. Please provide your feedback, if any, no later than 23 November 2015. After that, this document will be sent to RSSAC for formal action.
Best, Steve
_______________________________________________ rssac-caucus mailing list rssac-caucus@icann.org https://mm.icann.org/mailman/listinfo/rssac-caucus
participants (10)
-
Arturo Servin -
Jaap Akkerhuis -
John Bond -
Kash, Howard M CIV USARMY RDECOM ARL (US) -
Ondřej Surý -
Paul Hoffman -
Steve Sheng -
Warren Kumari -
Wessels, Duane -
William Sotomayor