rrd_reports.cgi
rrd_reports.cgi
Background
HTML and JSON reports are generated from various applications, including process_rrd_3 and FIDO. Historically, these reports were dumped into a directory and viewed by examining the contents of a directory, "ls" style.
In 2014, I began archiving these reports so you could go back in time to see reports generated on a given day. This was great, but left us with a mountain of data to pour through. In 2015, an sql backend and web front end were introduced to ease the burden of finding interesting things to look at.
What/where
as59: https://stats.net.wisc.edu/cgi-bin/rrd_reports.cgi
as59: https://stats.net.wisc.edu/cgi-bin/rrd_reports.cgi
as3128: https://stats.uwsys.net/cgi-bin/rrd_reports.cgi
rrd_reports.pl is a front end to process_rrd_3 for generating reports. Daily, weekly and monthly reports are generated via net cron, but the script can also be ran by hand.
Both of the above read a config file in /usr/local/rrd_reports/etc/.
The SQL table is searchable via the following example URL, which is linked off the main stats page.
The rotation of files in /var/www/html/reports/archived is handled by the ns-rrd-reports project, specifically /etc/cron.d/ns-rrd-reports-purge-old-files.cron. Configuration for how many files to keep on rotation is kept in rrd_reports.pl.yaml sitelocal_config. If you update this file, you must update_config.pl and commit.
More info about rrd_reports.cgi
* 'items' is supposed to be a simple count of how many things are in the report.
* 'errors' is an indicator if the generating script had errors during generation, not to be confused with 'items'
* Type includes syslog severity link levels: [error, warning, info]. type=unknown is reserved for missing or non relevant types.
Table cell has a colored border legend:
rrd_reports_update.pl is daemonized and looks for incoming html reports. It's role is to archive incoming files appropriately and update the SQL backend. Incoming reports don't need to come from process_rrd_3 but need to have magic comment lines in order to take full advantage of the SQL/web frontend. Script can be ran non interactively via CLI to process files in batch or to scrub old/dead/broken files from disk.
Both of the above read a config file in /usr/local/rrd_reports/etc/.
The SQL table is searchable via the following example URL, which is linked off the main stats page.
The rotation of files in /var/www/html/reports/archived is handled by the ns-rrd-reports project, specifically /etc/cron.d/ns-rrd-reports-purge-old-files.cron. Configuration for how many files to keep on rotation is kept in rrd_reports.pl.yaml sitelocal_config. If you update this file, you must update_config.pl and commit.
Example:
[@stats1-cssc sitelocal_config]$ cat /usr/local/ns-rrd-reports/etc/sitelocal_config/rrd_reports.pl.yaml
---
paths:
incoming_dir: /var/local/ns-rrd-reports/incoming
rotate:
daily: '45'
monthly: '13'
weekly: '28'
More info about rrd_reports.cgi
* 'items' is supposed to be a simple count of how many things are in the report.
* 'errors' is an indicator if the generating script had errors during generation, not to be confused with 'items'
* Type includes syslog severity link levels: [error, warning, info]. type=unknown is reserved for missing or non relevant types.
Table cell has a colored border legend:
- green = no errors, no matches to report. Everything is great
- yellow = report has items to investigate
- red = the report generator itself had an error. Investigation recommended
- brown = linked html report is stale; either the test has been disabled or there is a problem with the report generation
- gray = the report didn't provide information about how many items were encountered. Recommended to update the report generator to include this information