Reports
Reports
Background
HTML and JSON reports are generated from various applications, including process_rrd_3 and FIDO. Historically, these reports were dumped into a directory and viewed by examining the contents of a directory, "ls" style.
In 2014, I began archiving these reports so you could go back in time to see reports generated on a given day. This was great, but left us with a mountain of data to pour through. In 2015, an sql backend and web front end were introduced to ease the burden of finding interesting things to look at.
What/where
rrd_reports.pl is a front end to process_rrd_3 for generating reports. Daily, weekly and monthly reports are generated via net cron, but the script can also be ran by hand.
rrd_reports_update.pl is daemonized and looks for incoming html reports. It's role is to archive incoming files appropriately and update the SQL backend. Incoming reports don't need to come from process_rrd_3 but need to have magic comment lines in order to take full advantage of the SQL/web frontend. Script can be ran non interactively via CLI to process files in batch or to scrub old/dead/broken files from disk.
Both of the above read a config file in /usr/local/rrd_reports/etc/.
The SQL table is searchable via the following example URL, which is linked off the main stats page.
https://stats.uwsys.net/cgi-bin/rrd_reports.cgi
https://stats.net.wisc.edu/cgi-bin/rrd_reports.cgi
More info about rrd_reports.cgi
Magic variables to include in the source file are as follows [as of 2015/04/27]
if (m/^#matches=(\d+)$/) {
$attributes{'items'} = $1;
} elsif (m/^#errors_encountered=(\d+)$/) {
$attributes{'errors'} = $1;
} elsif (m/^#Files Examined=(\d+)$/) {
$attributes{'considered_items'} = $1;
} elsif (m/^#Start='(.+) -/) {
$attributes{'start'} = $1;
} elsif (m/^#End='(.+) - /) {
$attributes{'end'} = $1;
} elsif (m/^#Description=(.+)/) {
$attributes{'description'} = $1;
} elsif (m/^#report_type=(.+)/) {
$attributes{'type'} = $1;
} elsif (m/^#processing_took=(\d+) /) {
$attributes{'runtime'} = $1;
}
* 'items' is supposed to be a simple count of how many things are in the report.
* 'errors' is an indicator if the generating script had errors during generation, not to be confused with 'items'
* Type includes syslog severity link levels: [error, warning, info]. type=unknown is reserved for missing or non relevant types.
Table cell has a colored border legend:
- green = no errors, no matches to report. Everything is great
- yellow = report has items to investigate
- red = the report generator itself had an error. Investigation recommended
- brown = linked html report is stale; either the test has been disabled or there is a problem with the report generation
- gray = the report didn't provide information about how many items were encountered. Recommended to update the report generator to include this information