What happens if you press the Today button and regenerate the plot? John
On 19 May 2015, at 15:04, matt singh <matt.singh@centralnic.com> wrote:
/ usr/local/var/hedgehog/data/refile_and_grok.stdout
Tue May 19 14:50:01 BST 2015 SCRIPT COMMENCED Tue May 19 14:50:02 BST 2015 START: Server-A/Node-1 Tue May 19 14:50:02 BST 2015 DONE: Server-A/Node-1 Tue May 19 14:50:02 BST 2015 SCRIPT COMPLETE
/usr/local/var/hedgehog/data/<Server>/<Node>/dsc-extractor-xml-db.out
----------------------- 2015-05-19 14:50:02 BST ---------------------- *** Database connected *** Processing server: server_a *** Processing node: Node-1 *** Found: 5 .xml files to process.
2015-05-19 14:50:02 BST: Starting: ./incoming/1432043160.dscdata.xml 2015-05-19 14:50:02 BST: Moved: ./incoming/1432043160.dscdata.xml to ./done/incoming/dscdata/1432043160.dscdata.xml 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043160.dscdata.xml 4 remaining (0 failures).
2015-05-19 14:50:02 BST: Starting: ./incoming/1432043220.dscdata.xml 2015-05-19 14:50:02 BST: Moved: ./incoming/1432043220.dscdata.xml to ./done/incoming/dscdata/1432043220.dscdata.xml 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043220.dscdata.xml 3 remaining (0 failures).
2015-05-19 14:50:02 BST: Starting: ./incoming/1432043280.dscdata.xml 2015-05-19 14:50:02 BST: Moved: ./incoming/1432043280.dscdata.xml to ./done/incoming/dscdata/1432043280.dscdata.xml 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043280.dscdata.xml 2 remaining (0 failures).
2015-05-19 14:50:02 BST: Starting: ./incoming/1432043340.dscdata.xml 2015-05-19 14:50:02 BST: Moved: ./incoming/1432043340.dscdata.xml to ./done/incoming/dscdata/1432043340.dscdata.xml 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043340.dscdata.xml 1 remaining (0 failures).
2015-05-19 14:50:02 BST: Starting: ./incoming/1432043400.dscdata.xml 2015-05-19 14:50:02 BST: Moved: ./incoming/1432043400.dscdata.xml to ./done/incoming/dscdata/1432043400.dscdata.xml 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043400.dscdata.xml 0 remaining (0 failures).
*** Database disconnected ----------------------- 2015-05-19 14:50:02 BST -----------------------
I have changed log_statement to 'all'. Its now logging all the queries (for the last 12 hours window)
May 19 15:02:51 localhost postgres[6279]: [2-1] LOG: 00000: connection received: host=[local] May 19 15:02:51 localhost postgres[6279]: [2-2] LOCATION: BackendInitialize, postmaster.c:3544 May 19 15:02:51 localhost postgres[6279]: [3-1] LOG: 00000: connection authorized: user=www-data database=hedgehog May 19 15:02:51 localhost postgres[6279]: [3-2] LOCATION: PerformAuthentication, postinit.c:230 May 19 15:02:51 localhost postgres[6279]: [4-1] LOG: 00000: statement: set datestyle to ISO May 19 15:02:51 localhost postgres[6279]: [4-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [5-1] LOG: 00000: statement: set statement_timeout to 300000; May 19 15:02:51 localhost postgres[6279]: [5-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [6-1] LOG: 00000: statement: PREPARE getpltdetails (INTEGER) AS SELECT name, title, description, plot_id FROM dsc.plot WHERE id=$1; May 19 15:02:51 localhost postgres[6279]: [6-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [7-1] LOG: 00000: statement: EXECUTE getpltdetails(1); May 19 15:02:51 localhost postgres[6279]: [7-2] DETAIL: prepare: PREPARE getpltdetails (INTEGER) AS SELECT name, title, description, plot_id FROM dsc.plot WHERE id=$1; May 19 15:02:51 localhost postgres[6279]: [7-3] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [8-1] LOG: 00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='2' May 19 15:02:51 localhost postgres[6279]: [8-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [9-1] LOG: 00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='5' May 19 15:02:51 localhost postgres[6279]: [9-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [10-1] LOG: 00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='6' May 19 15:02:51 localhost postgres[6279]: [10-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [11-1] LOG: 00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='7' May 19 15:02:51 localhost postgres[6279]: [11-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [12-1] LOG: 00000: statement: PREPARE by_node_all_nodes (INTEGER, INTEGER, TIMESTAMPTZ, TIMESTAMPTZ) AS select to_timestamp((extract(epoch from sq.sx)::int / (extract(epoch from $4::timestamp - $3::timestamp + interval '1 minute')/1440)::int )*(extract(epoch from $4::timestamp - $3::timestamp + interval '1 minute')/1440)::int) AS x, sq.skey as key, avg(sq.sy) as y from (SELECT d.starttime AS sx, n.region AS skey, sum(d.value)/60.0 AS sy FROM dsc.data d, dsc.node n WHERE d.node_id = n.id AND d.server_id=$1 AND d.plot_id=$2 AND d.starttime>=$3 AND d.starttime<=$4 GROUP BY sx, skey) as sq GROUP BY x, key; May 19 15:02:51 localhost postgres[6279]: [12-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [13-1] LOG: 00000: statement: PREPARE getdatasetids (INTEGER) AS SELECT dataset_id FROM dsc.plot WHERE id=$1; May 19 15:02:51 localhost postgres[6279]: [13-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [14-1] LOG: 00000: statement: EXECUTE getdatasetids('1'); May 19 15:02:51 localhost postgres[6279]: [14-2] DETAIL: prepare: PREPARE getdatasetids (INTEGER) AS SELECT dataset_id FROM dsc.plot WHERE id=$1; May 19 15:02:51 localhost postgres[6279]: [14-3] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [15-1] LOG: 00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='8' May 19 15:02:51 localhost postgres[6279]: [15-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [16-1] LOG: 00000: statement: select typname, typcategory from pg_type where oid = 1007 May 19 15:02:51 localhost postgres[6279]: [16-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [17-1] LOG: 00000: statement: EXECUTE by_node_all_nodes(1, '21', timestamptz '2015-05-18 03:03', timestamptz '2015-05-18 15:02'); May 19 15:02:51 localhost postgres[6279]: [17-2] DETAIL: prepare: PREPARE by_node_all_nodes (INTEGER, INTEGER, TIMESTAMPTZ, TIMESTAMPTZ) AS select to_timestamp((extract(epoch from sq.sx)::int / (extract(epoch from $4::timestamp - $3::timestamp + interval '1 minute')/1440)::int )*(extract(epoch from $4::timestamp - $3::timestamp + interval '1 minute')/1440)::int) AS x, sq.skey as key, avg(sq.sy) as y from (SELECT d.starttime AS sx, n.region AS skey, sum(d.value)/60.0 AS sy FROM dsc.data d, dsc.node n WHERE d.node_id = n.id AND d.server_id=$1 AND d.plot_id=$2 AND d.starttime>=$3 AND d.starttime<=$4 GROUP BY sx, skey) as sq GROUP BY x, key; May 19 15:02:51 localhost postgres[6279]: [17-3] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [18-1] LOG: 00000: statement: select attnotnull from pg_attribute where attrelid=18872 and attnum='7' May 19 15:02:51 localhost postgres[6279]: [18-2] LOCATION: exec_simple_query, postgres.c:915 May 19 15:02:51 localhost postgres[6279]: [19-1] LOG: 00000: disconnection: session time: 0:00:00.212 user=www-data database=hedgehog host=[local] May 19 15:02:51 localhost postgres[6279]: [19-2] LOCATION: log_disconnections, postgres.c:4434
On 19/05/15 14:50, John Dickinson wrote:
On 19 May 2015, at 14:38, matt singh <matt.singh@centralnic.com> wrote:
Hello, I have installed Hedgehog 2.0.0b2 successfully, however I am now capturing dsc XML data from one dns node using dsc. Then importing it every 5 mins into the database using;
sudo -u hedgehog /usr/local/bin/refile_and_grok.sh -c
My node is showing up ok in the web interface. But I am not seeing any results in the web interface. All I see is "Sorry... your query didn't return any results. Please try re-generating the graph with new selection criteria". I am looking back over the last week for results, but nothing is showing up. I can see the "data" table in the postgresql database is growing everytime there is an import. I have verbose logging on for postgresql and its not showing any errors. Apache is not showing any errors either. Any ideas what I'm overlooking?
Hi Matt,
Please could you check the contents of /usr/local/var/hedgehog/data/refile_and_grok.stdout and /usr/local/var/hedgehog/data/<Server>/<Node>/dsc-extractor-xml-db.out?
Also you can tell postgresql to log all queries with the /etc/postgresql/9.3/main/postgresql.conf and changing the log_statement line.
HTH John
-- Matt Singh System Administrator CentralNic Group plc Innovative, Reliable and Flexible Registry Services for ccTLD, gTLD and private domain name registries
London Stock Exchange Symbol: CNIC
CentralNic Group plc is a company registered in England and Wales with company number 8576358. Registered Offices: 35-39 Moorgate, London EC2R 6AR.