- dns-stats-users - lists.icann.org

Compactor Compile Problem
by Peter DeVries Nov. 22, 2017

Nov. 22, 2017

Hi all, trying to compile compactor on a linux system. I've got all the dependencies met and configure completes successfully but make is failing with the following errors. Any advice is appreciated. Peter make all-am make[1]: Entering directory '/home/peter/Downloads/dns-stats-compactor-0.10.1' CXX src/compactor-baseoutputwriter.o In file included from src/dnsmessage.hpp:22:0, from src/baseoutputwriter.hpp:20, from src/baseoutputwriter.cpp:15: src/capturedns.hpp: In member function ‘virtual CaptureDNS* CaptureDNS::clone() const’: src/capturedns.hpp:649:36: error: invalid new-expression of abstract class type ‘CaptureDNS’ return new CaptureDNS(*this); ^ src/capturedns.hpp:72:7: note: because the following virtual functions are pure within ‘CaptureDNS’: class CaptureDNS : public Tins::PDU ^~~~~~~~~~ In file included from /usr/local/include/tins/dns.h:39:0, from /usr/local/include/tins/tins.h:33, from src/ipaddress.hpp:22, from src/configuration.hpp:21, from src/baseoutputwriter.cpp:13: /usr/local/include/tins/pdu.h:516:18: note: virtual void Tins::PDU::write_serialization(uint8_t*, uint32_t) virtual void write_serialization(uint8_t* buffer, uint32_t total_sz) = 0; ^~~~~~~~~~~~~~~~~~~ In file included from src/baseoutputwriter.hpp:20:0, from src/baseoutputwriter.cpp:15: src/dnsmessage.hpp: At global scope: src/dnsmessage.hpp:228:16: error: cannot declare field ‘DNSMessage::dns’ to be of abstract type ‘CaptureDNS’ CaptureDNS dns; ^~~ Makefile:1261: recipe for target 'src/compactor-baseoutputwriter.o' failed make[1]: *** [src/compactor-baseoutputwriter.o] Error 1

2 1

Hedgehog 2.3.0 release
by Sara Dickinson Aug. 22, 2017

Aug. 22, 2017

Hi All, We are pleased to announce the 2.3.0 release is now available! Packages available from: ppa:dns-stats/presenter Source code available at: https://github.com/dns-stats/hedgehog <https://github.com/dns-stats/hedgehog> Documentation: https://github.com/dns-stats/hedgehog/wiki/Documentation <https://github.com/dns-stats/hedgehog/wiki/Documentation> Upgrade: https://github.com/dns-stats/hedgehog/wiki/Hedgehog_Upgrade_Guide_(to_Packa… Updates: * Add ability to filter the server addresses displayed in the 'By server IP address' plot. Controlled by the database_update_service_addr script. * Add new 'By region' graph which is now the default. This graph always groups the results by Region. The 'By node' graph now always displays a line per node regardless of the node selection (previously it grouped by Region when all nodes were selected). * Default plot is now a configuration option. To specify a plot use the 'Hedgehog internal plot name' as listed in the User Guide. * Update RSSAC report header format to match RSSC002v3 report. * Enhancements to datafiles_create_summary script. Regards dns-stats team

1 1

DNS-STATS Compactor 0.10.1 release
by Sara Dickinson June 21, 2017

June 21, 2017

All, We are pleased to announce the first open source release of the DNS-STATS Compactor - version 0.10.1. The Compactor is a suite of tools which can capture DNS traffic (from either a network interface or a PCAP file) and store it in the Compacted-DNS (C-DNS) file format: https://datatracker.ietf.org/doc/draft-ietf-dnsop-dns-capture-format/ <https://datatracker.ietf.org/doc/draft-ietf-dnsop-dns-capture-format/> PCAP files for the captured traffic can also be reconstructed For more information please see the project wiki: https://github.com/dns-stats/compactor/wiki <https://github.com/dns-stats/compactor/wiki> Packages available from: ppa:dns-stats/compactor Source code available at: https://github.com/dns-stats/compactor <https://github.com/dns-stats/compactor> Documentation: https://github.com/dns-stats/compactor/wiki/user-guide.pdf <https://github.com/dns-stats/compactor/wiki/user-guide.pdf>, Sara.

1 0

Hedgehog 2.2.0 release
by Sara Dickinson Sept. 2, 2016

Sept. 2, 2016

Hi All, We are pleased to announce the 2.2.0 release is now available! Packages available from: ppa:dns-stats/presenter Source code available at: https://github.com/dns-stats/hedgehog <https://github.com/dns-stats/hedgehog> Documentation: https://github.com/dns-stats/hedgehog/wiki/Documentation <https://github.com/dns-stats/hedgehog/wiki/Documentation> Updates: * Add a new plot for DNSKEY queries vs ASN Regards dns-stats-team _______________________________________________ dns-stats-users mailing list dns-stats-users(a)dns-stats.org <mailto:dns-stats-users@dns-stats.org> https://mm.dns-stats.org/mailman/listinfo/dns-stats-users <https://mm.dns-stats.org/mailman/listinfo/dns-stats-users>

1 0

Hedgehog 2.1.1 - URGENT BUG FIX RELEASE - ALL USERS SHOULD UPGRADE NOW
by Sara Dickinson Aug. 17, 2016

Aug. 17, 2016

Hi All, As mentioned we are making a bug fix release to address an urgent issue effecting the generation of next months DB partitions. Without this fix data imports will fail on 1st Oct 2016. Latest package available from: ppa:dns-stats/presenter Source code available at: https://github.com/dns-stats/hedgehog <https://github.com/dns-stats/hedgehog> Documentation: https://github.com/dns-stats/hedgehog/wiki/Documentation <https://github.com/dns-stats/hedgehog/wiki/Documentation> Updates: * Fix issue with database_manage_partitions script where generating tables for -09 and -10 months didn't work * Add additional check on input parameters * Modify instructions due to issue with latest build of rcpp * Migrate to install from packages only from 2.1.1 *IMPORTANT*: After upgrading all users should manually run the following command to ensure the DB partitions for next month are successfully created: sudo -u hedgehog hedgehogctl database_manage_partitions -m 2016-09 We apologies for this inconvenience. Regards dns-stats team

1 0

Re: [dns-stats-users] Upcoming bug fix release this week
by Sara Dickinson Aug. 15, 2016

Aug. 15, 2016

Hi All, We have uncovered a bug in the database_manage_partitions script that effects the creation of new data table partitions for September and October (you may observe an error from the relevant cron job). We have a fix and there will be a 2.1.1 bug fix release coming out in the next couple of days. Best regards Sara. ------------------------- Sara Dickinson http://sinodun.com <http://sinodun.com/> Sinodun Internet Technologies Ltd. Magdalen Centre The Oxford Science Park Oxford OX4 4GA U.K.

1 0

Hedgehog 2.1.0 - Production release of 'Daurian' dns-stats presenter
by Sara Dickinson June 20, 2016

June 20, 2016

Hi All, We are pleased to announce the 2.1.0 production release is now available! Packages available from: ppa:dns-stats/presenter Download source: https://github.com/dns-stats/hedgehog/archive/2.1.0.tar.gz <https://github.com/dns-stats/hedgehog/archive/2.1.0.tar.gz> SHA-256 checksum: 7434ec01d068f63e09a2539c3f11fae190c5cfb2d2f6b581586dad4908127377 Documentation: https://github.com/dns-stats/hedgehog/wiki/Documentation <https://github.com/dns-stats/hedgehog/wiki/Documentation> Regards dns-stats-team

1 0

Hedgehog 2.1.0rc3 - Daurian release candidate of dns-stats presenter
by Sara Dickinson June 3, 2016

June 3, 2016

Hi All, A 2.1.0rc3 release is now available Download: https://github.com/dns-stats/hedgehog/archive/2.1.0rc3.tar.gz <https://github.com/dns-stats/hedgehog/archive/2.1.0rc3.tar.gz> SHA-256 checksum: afe4b86349db7930e31d45ccf3d69a2965e15a113ca5dd5522ce078e66160647 Documentation: https://github.com/dns-stats/hedgehog/wiki/Documentation <https://github.com/dns-stats/hedgehog/wiki/Documentation> Updates: * Add option to specify number of processes for refile_and_grok to use * Fix connection issue in database_update_tlds_from_zone script * Make hedgehogctl produce correct tools list for split installs * Minor documentation fixes We now plan a production release next week. Regards dns-stats team

1 0

Re: [dns-stats-users] Hedgehog 2.1.0b2 - Daurian release of dns-stats presenter
by Sara Dickinson May 20, 2016

May 20, 2016

Hi All, After initial beta testing with ICANN we are please to announce the 2.1.0b2 release of Hedgehog, the dns-stats presenter, is now available for testing. Some things to note: * We recommend this beta is used for testing only, not production. * The 2.1 release of Hedgehog is codenamed ‘Daurian’, a subspecies of hedgehog. 2.2 will be the ‘Bare-bellied’ release. * As of 2.1 debian packages are now available from ppa:dns-stats/presenter. Please see the documentation at https://github.com/dns-stats/hedgehog/wiki/Hedgehog_Upgrade_Guide_(to_Packa… for how to upgrade from an existing 2.0 install using packages. Testing of this upgrade path is encouraged. * Installation from source is still supported in this release but users are encouraged to migrate to using packages. Download: https://github.com/dns-stats/hedgehog/archive/2.1.0b2.tar.gz <https://github.com/dns-stats/hedgehog/archive/2.1.0b2.tar.gz> SHA-256 checksum: 27fa22ec57ca3f05e1793baed1844b8d494d6514ab21031f322a3dd964a56313 Documentation: https://github.com/dns-stats/hedgehog/wiki/Documentation <https://github.com/dns-stats/hedgehog/wiki/Documentation> Features: * Packages available from ppa:dns-stats/presenter * Added new graphs for different types of TLD (QTYPE and RCODE for most popular TLD). Some support for displaying IDN in their unicode glyphs. * Added new graph to display Geo-locations of querying subnets and nodes * Added new graphs to display query sources as BGP prefixes or ASN * Added new graphs to group nodes by Country/City/Instance * Added new graph to display Destination Address vs Query rate * Added option to group nodes in selection tab by Country/City/Instance (specified in nodes.csv) and config option for default choice. * Added rssacd demon to collect rssac load-time and zone-size data. Requires management IP Address of node to be specified in nodes.csv * Added GeoLite2 data created by MaxMind, available from http://www.maxmind.com <http://www.maxmind.com/> * Added static link URI to provide access to raw plot files * Updated database_update_nodes script to support new fields. * Multiple remote Web front ends now supported * New config option to customise 'Support' link on Web page * Ensure output format consistent with latest RSSAC02 specs Maintenance/Bug fixes: * Improve DB performance by adding further indexes * Change to rssac scripts. Processing of unique_sources data now done by 'database_process_rssac_data' script which must be explicitly called. * check_user_perms script now does all user checks, check db access and data dir. access. Used in new database_update_nodes will gradually be used by all tools. * Improved chart sizes, font sizes and data ordering in some graphs * Significant enhancements to installation and management scripts and configuration files. * Fixed appearance of gvis column chart vAxis * Fixed error in rssac reports where ".root-servers.org <http://root-servers.org/>" was used instead of ".root-servers.net <http://root-servers.net/>”. We plan a release candidate in early May. Regards dns-stats team

1 2

Start of month XML import error
by matt singh June 8, 2015

June 8, 2015

Hello, I have noticed that import of xml data for my node has not been working since the beginning of this month. If I try and re-import old data using: sudo -u hedgehog /usr/local/bin/refile_and_grok.sh -s 2015-06-01 dsc-extractor-xml-db.out ----------------------- 2015-06-08 11:08:25 BST ---------------------- *** Database connected *** Processing server: ns_5 *** Processing node: Node-1 *** Found: 10684 .xml files to process. 2015-06-08 11:08:25 BST: Starting: ./incoming/1433116920.dscdata.xml Format 3b write_db runtime error: ERROR: new row for relation "data_ns_5_idn_vs_tld_2015_06" violates check constraint "data_ns_5_idn_vs_tld_2015_06_starttime_check" DETAIL: Failing row contains (2015-06-01 00:00:00+01, 2, 2, 16, ALL, abc, 5). The psql table is indeed empty. select * from data_ns_5_idn_vs_tld_2015_06; starttime | server_id | node_id | plot_id | key1 | key2 | value -----------+-----------+---------+---------+------+------+------- (0 rows) Can I only import historical data if the file format is DAT? I've converted some of the files to dat format using the dsc-extractor utility. But got a similar error when running sudo -u hedgehog /usr/local/bin/refile_and_grok.sh -i DAT ----------------------- 2015-06-08 11:41:46 BST ---------------------- *** Database connected *** Processing server: ns_5 *** Processing node: Node-1 *** Found: 22 .dat files to process. 2015-06-08 11:41:46 BST: Starting: ./20150601/certain_qnames_vs_qtype.dat Format 2 unique_violation error: ERROR: duplicate key value violates unique constraint "pk_data_ns_5_certain_qnames_vs_qtype_2015_06" DETAIL: Key (server_id, node_id, plot_id, starttime, key1, key2)=(2, 2, 1, 2015-06-01 02:55:00+01, else, 15) already exists. 2015-06-08 11:41:47 BST: Error: failed to process: ./20150601/certain_qnames_vs_qtype.datReason: Failed to process data 2015-06-08 11:41:47 BST: File renamed to: ./20150601/certain_qnames_vs_qtype.dat.failed.7083-7868-9f17-1423 2015-06-08 11:41:47 BST: Completed: ./20150601/certain_qnames_vs_qtype.dat 21 remaining (1 failures). 2015-06-08 11:41:47 BST: Starting: ./20150601/client_addr_vs_rcode_accum.dat Format 4 write_db Runtime error: ERROR: new row for relation "data_ns_5_client_addr_vs_rcode_accum_2015_06" violates check constraint "data_ns_5_client_addr_vs_rcode_accum_2015_06_starttime_check" DETAIL: Failing row contains (2015-06-01 00:00:00+01, 2, 2, 3, 201.150.62.201, 0, 28). Any ideas what I'm overlooking here? thanks, matt

2 1