Arturo Servin writes:

Do you need a +1 or silence is enough to show that we agree on the changes.

I first want to have some time to read it properly. Is tehere a drop death date for reactions? jaap

Thanks Arturo, +1 would be appreciated! Best Steve From: Arturo Servin <arturo.servin@gmail.com<mailto:arturo.servin@gmail.com>> Date: Tuesday, November 17, 2015 at 7:46 PM To: Steve Sheng <steve.sheng@icann.org<mailto:steve.sheng@icann.org>>, "rssac-caucus@icann.org<mailto:rssac-caucus@icann.org>" <rssac-caucus@icann.org<mailto:rssac-caucus@icann.org>> Subject: Re: [rssac-caucus] FOR REVIEW: DRAFT RSSAC002 Version 2 Do you need a +1 or silence is enough to show that we agree on the changes. Regards as On Tue, 17 Nov 2015 at 12:05 Steve Sheng <steve.sheng@icann.org<mailto:steve.sheng@icann.org>> wrote: Dear RSSAC Caucus, Attached please find RSSAC002 version 2 for your review. In this version, we made the following changes based on the RSSAC caucus discussions led by Duane: * Section 2.2 (The size of the overall root zone) was amended to clarify that TCP size prefix octets are not included in the metric. * Section 2.4 (The query and response size distribution) was amended to clarify that TCP size prefix octets are not included in these metrics. * Section 2.4 was amended to include 0-15 in size ranges to be tabulated. * Superfluous quotes around YAML keys were removed from example YAML in sections 4.1 (The ‘load-time’ Metric) and 4.2 (The ‘zone-size’ Metric). * Indentation was fixed for example YAML in sections 4.3 (The ‘traffic-volume’ Metric) and 4.6 (The ‘unique-sources’ Metric). * Section 4.5 (The ‘rcode-volume’ Metric) was amended to clarify that nonzero counts should be omitted. In addition, we made the following organizational changes: * Make this document RSSAC002 version 2 * Added a Revision History section (see section 7). Attached please find the redline, clean word and PDF version. Since the proposed revisions have already been discussed and agreed on the caucus list, we will give this update a 7 day additional review period. Please provide your feedback, if any, no later than 23 November 2015. After that, this document will be sent to RSSAC for formal action. Best, Steve _______________________________________________ rssac-caucus mailing list rssac-caucus@icann.org<mailto:rssac-caucus@icann.org> https://mm.icann.org/mailman/listinfo/rssac-caucus

Hi Howard, On 19/11/2015 12:24, "rssac-caucus-bounces@icann.org on behalf of Kash, Howard M CIV USARMY RDECOM ARL (US)" <rssac-caucus-bounces@icann.org on behalf of howard.m.kash.civ@mail.mil> wrote:

Section 2.1 - For v3 of the document I would like to see more details on how this value should be computed, for the sake of consistency. For example, I have six servers that receive notifies directly from the distribution masters. Should the load-time value be computed on each server separately and then averaged (this is what I am currently doing)? Or should the notify and load times from all servers be collected centrally and the value computed by subtracting the earliest (first received) notify time from the latest/last load time? What if a notify is never received (by an individual server in the first case or by any of the servers in the second case)? This is documented as

"...Therefore measurements may only represent the average for 95% of the operationally-active instances of a root server per root zone serial." So what you are currently doing is correct. Thanks John

On Nov 19, 2015, at 4:24 AM, Kash, Howard M CIV USARMY RDECOM ARL (US) <howard.m.kash.civ@mail.mil> wrote:

Section 4.1, second paragraph, "The latency in the distribution system" needs to be changed to "Latency in publishing available data".

Yes.

Section 3, third bullet - is this still true? If this was for NSD, it has been corrected.

Not sure if its still true. We could poll the operators. I see little harm in leaving it as-is for now.

Section 2.1 - For v3 of the document I would like to see more details on how this value should be computed, for the sake of consistency. For example, I have six servers that receive notifies directly from the distribution masters. Should the load-time value be computed on each server separately and then averaged (this is what I am currently doing)? Or should the notify and load times from all servers be collected centrally and the value computed by subtracting the earliest (first received) notify time from the latest/last load time? What if a notify is never received (by an individual server in the first case or by any of the servers in the second case)?

This sounds to me like more than a simple clarification and probably worthy of a work party. So as you say, a good item for "v3".

Section 2.1, second paragraph - "may present as anomalies in measurements", perhaps change to "may introduce anomalies"?

How would you feel about "may appear as anomalies"? To me "introduce" sounds just a little too much like blaming anycast for causing a problem. Or maybe since this is the same section as the above item, it will likely all get rewritten in v3 so we save it until then?

Section 4.1, "If not load-time metric is available...", change "not" to "no"

Yes.

Section 5, make title plural (Recommendations)

Yes. Thanks!

Steve Sheng writes:

Among other things

Attached please find the latest document incorporating these changes. The deadline to review is still close of business Monday 23 November.

Triggered by questions from some people i had good look at the examples given. These seem to be made up and it shows. Example 4.3 "The `traffic volume' Metric" shows that there a total of 42447 requests while there where 148013 responses. That seems odd to me. Especially since the example 4.6 "The `unique sources' Metric" says that the responses should come from 2086124 ipv4 and 42941 ipv6 adresses. That seems very odd to me. Maybe we should change the examples to some actual (public available) data? BTW, the questions I got was whether there was any idea how accurate the RSSAC 002 numbers are. QUite often one sees that there are less responses then requests. This can of course come from dropped requests (rate limiting stuff etc.) but the reverse, more responses then request have also been seen. For UDP at a.root: <http://a.root-servers.org/rssac-metrics/raw/2015/11/traffic-volume/a-root-20...> (and days after that). For TCP, see <https://www-static.ripe.net/dynamic/rssac002-metrics/2015/04/traffic-volume/...>. One wonders how this can happen. Any idea? A suggestion was made that the actual answer/response packets where counted instead of DNS query/responses and fragmentation happened but that seems odd to me. Anyway, I woud still advice that for the new version of the dpocument some real data for the examples will be used. And of course, some answer to the questions asked will be appreciated. Regards, jaap

On Nov 23, 2015, at 4:51 AM, Jaap Akkerhuis <jaap@NLnetLabs.nl> wrote:

Triggered by questions from some people i had good look at the examples given. These seem to be made up and it shows. Example 4.3 "The `traffic volume' Metric" shows that there a total of 42447 requests while there where 148013 responses. That seems odd to me. Especially since the example 4.6 "The `unique sources' Metric" says that the responses should come from 2086124 ipv4 and 42941 ipv6 adresses. That seems very odd to me. Maybe we should change the examples to some actual (public available) data?

Jaap, I'm fine with having real data in the examle YAML documents. Would you be alright with waiting on this for v3 so that we can get v2 approved and published soon? DW

"Kash, Howard M CIV USARMY RDECOM ARL (US)" writes:

Section 3, third bullet - is this still true? If this was for NSD, it has been corrected.

For NSD this is fixed in 4.12, in 4.1.4 it will now also be logged at a less verose loggiong level (on request of Anand, K-root). jaap