FIDO: File Formats

FIDO: File Formats

Description of the Status Files written by FIDO tests

Want to make your own test? Here is the format of the status files.
All tests, including FIDO proper itself, will write out a status file that contains information about the test itself and objects related to the test.
<item data> 
   <failed-object [object name]> # appended with the test type to avoid collisions

	       # CALCUATED BY FIDO: Only relevant for fido_node alarms.  List of contributing alarms
	       item_data s-hotspare-b360-17-hotspare-fido_node candidate []
	       # OPTIONAL
               item = [string]    # this object is related to an upstream object
               reason = [string]   # text description of why this correlation was made
		# OPTIONAL: [boolean]
                # delete this from the FIDO report file even if I didn't repoll a status update for everything this time around
		# OPTIONAL: [interger]
                # delete this from the FIDO report file even if I didn't repoll a status update for everything this time around, but only this instance
		# CALCULATED BY FIDO: [string]: FIDO's calculated description of an item
		# OPTIONAL string.  Used by the correlation engine

		# CALCULATED BY FIDO: [number]: unique identifier for this event

		# CALCULATED BY FIDO: [integer]: number of times the update was determined to have failed

		# CALCULATED BY FIDO: [string]: ingress filename for given alarm
help # OPTIONAL: will be displayed in the gui. If you don't supply, the test filename will be used key1 = help file name [string name] key2 = help file number [integer] holddown # CALCULATED BY FIDO: matches = [hash] # describes why the item in on holddown reason = [string] time = [integer] # time in minutes to hold down alarm based on alarm start time absolute_time = [string] # time in text to hold down alarm.
		# OPTIONAL: assumed 2 if not present
		value = integer
		reason = OPTIONAL [string]
		# OPTIONAL [string]. gui mouseover or 'wideinfo' shows this information
		# OPTIONAL:  successor of 'info' key

		infohash additional_info [122 accepted, 25 dropped, 1000 threshold]
		infohash icmp_source [fido_icmpv4_manual]
		infohash interface [port-channel109]
		infohash status [No response from ''
		infohash timestamp [2020/04/01 09:20:00]
		infohash AS [65011]
		infohash Descr [Goodnight Hall/Communicative Disorders]
		infohash IP []
		# OPTIONAL [string] used by the correlation engine
		# OPTIONAL [ip address] used by the correlation engine and also used for DNS lookups

		ipam circuit_id [value undefined]
		ipam connector [value undefined]
		ipam subnet []
		# [integer] used for correlation.  The lower the value, the more important to the correlation engine.
		example, ip=3, service=4, so service is correlated to IP, not vice versa.
		# OPTIONAL: [boolean]
                # keep this in the FIDO report file even if it isn't in the current status file
                # this is to support objects that aren't polled every cycle

		# CALCULATED BY FIDO: [boolean]: whether the item has been logged
		# OPTIONAL [integer]: defaults to one.  number of consecutive poll failures before the object shows in the gui

		# OPTIONAL [number] supplied by some tests to describe how long it took to verify this particular alarm. 

		# OPTIONAL [integer]: for debugging

		# CALCULATED BY FIDO.PL: federation information 
		source [1]
		source [1]
start # CALCULATED BY FIDO.PL: time in ticks alarm was first seen start_text # CALCULATED BY FIDO.PL: time in string alarm was first seen
		# REQUIRED: [string]: state of the tested object
		# CALCULATED BY FIDO.PL: whether or not the item is stale, ok, etc

		# OPTIONAL: [integer]: when suppression is being used, how many items are not shown because of this parent item.  Supplied by

		# REQUIRED: [string], the type of failure, displayed in the 'type' column, often used for correlation

		# REQUIRED: [interger]: time in ticks that the object failed during the current test period
               matches = [hash]
                   # describes why the item in on holddown
               actual_match = [string]
               reason = [string] 
               time = [string]
                   # time in text to hold down alarm.
		# OPTIONAL string.  Used by the correlation engine
		# OPTIONAL: key/value of url_key and destination
		ts_urls gnmis [^]
		# CALCULATED BY FIDO: [boolean]: 
		This is used to determine if an object should be removed from the report file.  
		The item is removed if the item itself isn't updated but the file it came from has been. 
urls # CALCULATED BY FIDO: key/value of url_key and destination urls __alarm_info [;test=icmpv6] urls rrd [<a href=> cgi: rrd <IMG SRC=""></a>]
		# CALCULATED BY FIDO: correlation
		item_data 2607:f388:e:101::10-icmpv6 via_connected subnet [2607:f388:e:101::1/64]

< /item data > 

grep "added: item_data" output.txt | egrep -v " (correlation|device|help|impact|info|infohash|interface|ip|iso|needed_failures|state|suppressed|test|time|url|start|start_text|status|source|event_id|failures|file|logged|urls|updated|descr|ipam|ts_device|ts_urls|candidate|via_connected|snmp_instance|deleteme|deleteme_instance) " 

< status >
		# OPTIONAL: [integer]: false if everything isn't down, otherwise it will be the number of down items

		# OPTIONAL: unique to updateType [scraper], lists health of downstream contributors
		contributors ban-arp state [OK]
		contributors fp-arp state [OK]
		contributors fp-systemEnvironmentals state [OK]

		# OPTIONAL, but really good idea: source data to show in fido_info.cgi
		data_files gMegaFlow_fido_icmp_ips.bin []
		data_files gMegaFlow_fido_icmp_ips_verbose.bin []

		# REQUIRED: [string] : total monitored items, not a count of what's down
		dhcp_leases items [820 rrd files [731 file(s) ignored]]

		# REQUIRED: [integer] (in seconds)

		# OPTIONAL: unique to updateType [scraper], informational/debug
		# OPTIONAL: [integer] (in seconds), last time a data source was updated
polled # OPTIONAL [array of polled items]. A list of 'test' types that had been updated during this most recent cycle. For tests that may not be testing every test type all of the time fido.status: added: snmprrd polled ADVAopticalIfDiagInputPower [array_string] fido.status: added: snmprrd polled ADVAopticalIfDiagOutputPower [array_string] fido.status: added: snmprrd polled AlphadataNumberValue [array_string] ... report # REQUIRED: [integer]: always increases by one, make sure we don't miss a report
		# REQUIRED: [integer]: time this report was written
		# OPTIONAL: [string]: used in conjunction with 'user' for fido_watchdog
		# REQUIRED: [integer]: seconds after last update before this data is stale 
start # OPTIONAL: [integer]: ticks when this daemon started status # CALCULATED BY FIDO: [string]: OK, stale, etc threads # OPTIONAL: [integer]: how many thread the test is configured to use updateType # OPTIONAL: [string]: scraper_alarm has a unique update methodology. This is a trigger to to behave differently url # OPTIONAL: [string]: points to file that contains a list of items, display in the FIDO status page uc_esx url [/uc_esx.txt] user # OPTIONAL: [string]: to be used by fido_watchdog < /status > status_output.txt | egrep -v " (items|last_cycle_time|polled|report|stale|threads|url|status|report_time|last_update|data_files|user|start|restart|contributors|updateType|last_poll) "

Description of

The main report file that is read by the CGI
   'comment_correlation' =>
        'comment' => # a comment
             'best' => # the best item for this comment, as determined by FIDO.
                       # best is defined by being the item with the most things
                       # correlated to it.  Other tiebreakers are also used.
             'items' =>  # a list of items with the above comment
                 'item with the comment mentioned above'

	comment_correlation 'cr 90035' best [ ifOperStatus Gi1/0/1-ifOperStatus]
	comment_correlation 'cr 90035' items ' ifOperStatus Gi1/0/1-ifOperStatus' [1]
	comment_correlation 'cr 90035' items [1]   

   'comments' => # where the comments are kept
                 # they are kept separate from the items themselves as we keep comment information for items that get deleted.
	# comment for this item
	comments '$itemName' comment [Problem 15824]

	# old comment data gets turned into a hash array and appended when the comment changes
	comments '$itemName' comment_history [{"discovery":"pending","first_seen":1578300837,"occurrences":1,"related_event_id":"20200106.3575"},{"discovery":"1575917448","first_seen":1578300837,"occurrences":1,"related_event_id":"20200106.3575"},{"comment":"Problem 15824","date":1578317204.17587,"discovery":"1575917448","first_seen":1578300837,"occurrences":1,"related_event_id":"20200106.3575","user":"aamohamed3"}] [array_jsonified]
	# [integer, ticks]; When this comment was last updated
	comments '$itemName' date [1578317204.17587]
# when the item was first monitored by FIDO comments '$itemName' discovery [1578410731]
	# [integer, ticks]; time the alarm was added
	comments '$itemName' first_seen [1578300837]
#[integer, ticks]; time the alarm was removed comments '$itemName' last_seen [1584513560]
	#[integer, ticks]; length in seconds that this alarm has been historically alarming.  Only present for items that have been removed at least once
	comments '$itemName' length [6200691]
	#[integer]; number of times the alarm has been seen
	comments '$itemName' occurrences [115]
        # can be used to link to an event in the sql log DB [if applicable]
	comments '$itemName' recurring_event_id [20200303.50091]
# can be used to link to an event in the sql log DB comments '$itemName' related_event_id [20200106.3575] # who commented the item comments '$itemName' user [aamohamed3]
	# summary of known network topology correlations as determined by
	correlated ' ifOperStatus Vl2671-ifOperStatus' [1]
	correlated ' ifOperStatus Po7-ifOperStatus' ' ifOperStatus Te1/0/7-ifOperStatus' [1]
	correlated 2607:f388:f:1c::2-icmpv6 [1]
	correlated s-hotspare-b360-17-hotspare-fido_node [1]
	correlated [1]
'event_id' [string] #next event will be assigned this ID event_id [20200401.31941]
	[integer, ticks]
	fido_start_time [1585746923]
	# hash structure defining correlation groupings based on alarm attributes
	group_correlated 2607:f388:2:6002::3-icmpv6 group [group='ipam descr=DoIT Platform Non Restricted Data']
	group_correlated 2607:f388:2:6002::3-icmpv6 members 2607:f388:2:6002::2-icmpv6 [1]
   'last_cycle_time' => [string]
       	# how long it took to create the last fido report
	last_cycle_time [<1]
'report' => [integer] # fido report number, increases by one every time a new report is written. report [220]
	[integer, ticks]
	When the report was written
	report_time [1585751633]

Keywords:FIDO: File Formats   Doc ID:9171
Owner:Michael H.Group:Network Services
Created:2009-02-17 19:00 CDTUpdated:2020-04-02 11:20 CDT
Sites:Network Services, Systems & Network Control Center, University of Wisconsin System Network
Feedback:  1   0