HTML and JSON reports are generated from various applications, including process_rrd_3 and FIDO.  Historically, these reports were dumped into a directory and viewed by examining the contents of a directory, "ls" style.

In 2014, I began archiving these reports so you could go back in time to see reports generated on a given day.  This was great, but left us with a mountain of data to pour through.  In 2015, an sql backend and web front end were introduced to ease the burden of finding interesting things to look at.


as59: https://stats.net.wisc.edu/cgi-bin/rrd_reports.cgi
as3128: https://stats.uwsys.net/cgi-bin/rrd_reports.cgi

rrd_reports.pl is a front end to process_rrd_3 for generating reports.  Daily, weekly and monthly reports are generated via net cron, but the script can also be ran by hand.

rrd_reports_update.pl is daemonized and looks for incoming html reports.  It's role is to archive incoming files appropriately and update the SQL backend.  Incoming reports don't need to come from process_rrd_3 but need to have magic comment lines in order to take full advantage of the SQL/web frontend.  Script can be ran non interactively via CLI to process files in batch or to scrub old/dead/broken files from disk.  

Both of the above read a config file in /usr/local/rrd_reports/etc/. 

The SQL table is searchable via the following example URL, which is linked off the main stats page.

The rotation of files in /var/www/html/reports/archived is handled by the ns-rrd-reports project, specifically /etc/cron.d/ns-rrd-reports-purge-old-files.cron.  Configuration for how many files to keep on rotation is kept in rrd_reports.pl.yaml sitelocal_config. If you update this file, you must update_config.pl and commit.


[@stats1-cssc sitelocal_config]$ cat /usr/local/ns-rrd-reports/etc/sitelocal_config/rrd_reports.pl.yaml 
  incoming_dir: /var/local/ns-rrd-reports/incoming
  daily: '45'
  monthly: '13'
  weekly: '28'

More info about rrd_reports.cgi

* 'items' is supposed to be a simple count of how many things are in the report.
* 'errors' is an indicator if the generating script had errors during generation, not to be confused with 'items'
* Type includes syslog severity link levels: [error, warning, info].  type=unknown is reserved for missing or non relevant types.

Table cell has a colored border legend:
  • green = no errors, no matches to report.  Everything is great
  • yellow = report has items to investigate
  • red = the report generator itself had an error.  Investigation recommended
  • brown = linked html report is stale; either the test has been disabled or there is a problem with the report generation
  • gray = the report didn't provide information about how many items were encountered.  Recommended to update the report generator to include this information

KeywordsReports rrd_reports.cgi rrd reports   Doc ID50855
OwnerMichael H.GroupUW System Network
Created2015-04-27 08:26:38Updated2022-05-23 15:56:31
SitesNetwork Services, Systems & Network Control Center, University of Wisconsin System Network
Feedback  0   0