AANTS - Everything You Ever Wanted To Know About the NetWatch Tool

Background Information

The AANTS NetWatch tool displays MAC address and IP information for device ports on the network. The tool tries to show the IP address that was most recently seen associated with a given MAC address. The NetWatch database tables record the MAC addresses seen on a given port, and the IP-to-MAC relationship observed (at a different point in time) as learned from the router ARP tables.

NetWatch's notion of the current MAC address in use on a given port, is updated in two ways:

1. By periodic polling
2. Whenever the MAC address is learned, the switch is configured to send it to our service (as an SNMP trap).

So, when you plug a machine onto a port, and that machine then generates an ethernet frame, AANTS (and EdgeConf forms which display it) should show that new value immediately. However, if a machine is continually producing traffic, then the switch will not relearn the address (since it already knows that that MAC is on the given port) so, AANTS will only periodically re-discover that address.

If IP or MAC addresses show in the NetWatch report, then we reliably know they have been on the network at least at the dates shown, but it does not necessarily mean that they haven't been on the network if they weren't learned.

NetWatch only knows the IP to MAC relationships that it learns from the DoIT-managed routers that it monitors. Users will not be able to use NetWatch to find information on IP addresses that are not routed on the campus network routers. This would be the case, for instance, if you are a "delegated" customer.

If this is the case, you can find the MAC address on the gateway router then use NetWatch to search by MAC address to find the switchport(s) on which that MAC has been seen, if it has been seen on DoIT-managed switches.

NetWatch Q&A

Q: How do I do a query on a specific MAC address?

A: Enter the MAC address in the MAC address form field on the NetWatch search form. The form accepts any format of MAC address as it strips out any non-hexadecimal characters when processing. The form will also accept a partial MAC address, but this may return too many results to easily view depending on how many devices with that vendor prefix exist.

Q: I've entered a MAC address in the NetWatch page but I find no results

A: If no results are returned it means that the MAC was not found using the given selection criteria, if the host hasn't been online for a while try a wider time range, like 300 days rather than past 5 days (which is the default), or try querying by IP address, or device instead.

Q: I've entered a MAC address in the NetWatch page, but the results seem out of date. Why?

A: The device has not been seen on the network recently or is connected through a personal networking device which is not managed by DoIT which prevents our tools from collecting information about it.

Q: Why are multiple rows returned for a single device port?

A: Our database schema doesn't store IP addresses on Ports. Instead it stores only what the switches know (which MAC is on which port) and what the routers know (which IP is associated with which MAC). Since these are observed independently, we don't have any confidence in doing some temporal join across those two pieces of info so, we fall back to just showing each data point in its place in time so the reader can reconstruct the history. NetWatch shows the history sorted in reverse chronological order of the last time a particular MAC address was seen on a particular network switch port and the last time it was associated with a particular IP address (IPv4 and IPv6). Most devices which are currently online will have been seen within the last 15 minutes, but NetWatch keeps a history of over a year so you can see a history if the device changed wired ports or IP addresses over that time, including the history of IPv6 privacy addresses used during that timespan.

Q: I've tried querying NetWatch for IP addresses I know are connected, but no results are returned. What gives?

A: It is possible MAC addresses are not being learned on that port, NetWatch MAC learnining is an optional feature which must be enabled on a per-port basis using AANTS::IfTag flags in the port description. MAC learning is not enabled on uplinks or trunk ports and may not be enabled on ports which have been manually configured outside of EdgeConf. A network Operator or Engineer can verify that the device is being polled and the port is subscribed to NetWatch. A less common issue is using a personal router between the device and the campus network, a device which is not directly on the campus network on equipment not managed by DoIT Network Services cannot be seen by the NetWatch tool.

Q: When I input an IP I see several MAC addresses, but when I enter one of those MAC addresses I only get one IP. What gives?

A: The IP address has been dynamically allocated to different devices at different times, so when searching by IP one can get a list of MAC addresses associated with the IP address, but when searching by a MAC one only sees the IP addresses associated with that one device.

Q: Is there a way to force NetWatch to update its view of the world?

A: Not via the web interface. This would require additional polling every switch across campus, adding to the load on the CPU of every networking device. One could cause a network switch to re-learn the MAC address by unplugging and replugging the machine's ethernet connection, but every managed network device is already polled every 15 minutes or so. If you want to check the immedate status on any individual switch you can use LookingGlass to run a rate-limited selection of read-only commands on manged devices.

Q: I think some information in the NetWatch database is outdated. How can I confirm what's really going on?

A: If you suspect information in the NetWatch database does not reflect the current state of the world, you can query the switch directly yourself using the LookingGlass tool. Choose the Device then the Query named show mac-address-table or show mac-address-table interface (and specify the interface) and it will query the switch on demand.

Q: Is there a way I may find which enabled ports have been "inactive for N days" on a switch?

A: The PortUseAuditor tool can query the NetWatch data over long periods of time and show which ports have had activity and which are unused, so that unused ports can be identified and reclaimed in a self-service fashion by the IT staff in Schools/Colleges/Divisions.

Q: Is it possible to query all managed switch ports that have multiple mac addresses on them?

A: The FindSecondaryNetwork tool can query the NetWatch data and show ports which have multiple MAC addresses learned which is ususally an indication there is an ad-hoc network switch or bridged virtual machines present on that port.

Q: Why do I get a 'Target Unknown' error when I click on 'Bits', 'Pkts' or 'Errs' for a particular port?

A: It is possible that the port has not been in use long enough for the measurement system to start collecting data for that port. Usually our measurement system will start measuring the port the first morning after a managed network device is added to the network. This is also possible if there is a change in how time-series metrics are named/stored such that the templated URL is no longer valid.

Q: Does the NetWatch database have information from DDN devices?

A: Yes although many devices are Virtual Machines and will show up as learned on the uplink ports for the Virtualization server where they are currently running.

Q: What about wireless data?

A: We do not collect MAC address date for UWNet or Eduroam using NetWatch or make that data directly available to Authorized Agents as the wireless network is a large pool of University-owned, personal and guest devices not all under management by any given Authorized Agent. Network Services and CyberSecurity do have logs and records of device MAC address to IP address mappings and one can open a ticket with either group to open an investigation on the location of a particular device MAC address.