Designing and Managing Behavior Models - Using Other Data Sources - NerveCenter's Built-In Triggers -
Using Other Data Sources      An Example Using Built-In Triggers

NerveCenter's Built-In Triggers

When NerveCenter requests a poll, the SNMP GetRequest or the ping that the poll initiates is placed on either NerveCenter's pending SNMP requests list or pending ICMP requests list. NerveCenter waits for a reply from the node or the node's SNMP agent (or from an intervening router). If the node or its SNMP agent sends a non-error reply, then NerveCenter evaluates the poll condition and fires the appropriate trigger.

However, if the node or its SNMP agent makes no response or returns an error -- depending upon the circumstances -- NerveCenter will either retry the request or fire one of its built-in triggers. Those conditions that cause NerveCenter to fire its built-in triggers can be broken down into the following categories:

• SNMP Requests
• Ping Requests
• Matching Errors with Pending SNMP and Ping Requests
• Multi-homed Nodes

  ---

     NerveCenter uses all uppercase letters to designate built-in trigger names.

  ---

For particular information about NerveCenter's built-in triggers, see A List of Built-In Triggers.

For information about the order in which NerveCenter fires built-in triggers, see Built-in Trigger Firing Sequence.

SNMP Requests

NerveCenter retries SNMP requests as many times as configured or until a reply arrives on the SNMP or ICMP socket that NerveCenter can match to a pending request. (NerveCenter uses the number of retries and retry interval specified on the SNMP tab in the NerveCenter Administrator. Refer to the Managing NerveCenter guide or Administrator help for details.)

If the reply is an SNMP error, NerveCenter does not retry the request but returns three built-in triggers with the poll: an ERROR trigger, followed by an SNMP_ERROR trigger, and then finally the appropriate SNMP built-in error trigger. (See A List of Built-In Triggers, for more information.)

If NerveCenter receives no response after the configured number of retries, then NerveCenter fires two built-in triggers: ERROR, followed by SNMP_TIMEOUT. For more information about the order in which NerveCenter fires built-in triggers, see Built-in Trigger Firing Sequence.

Ping Requests

NerveCenter retries ICMP requests as many times as configured or until NerveCenter receives a good, non-error response that it can match to a pending ICMP request. (NerveCenter uses the number of retries and retry interval specified on the SNMP tab in the NerveCenter Administrator. Refer to the Managing NerveCenter guide or Administrator help for details.) If NerveCenter receives no response after the configured number of retries, then NerveCenter fires two built-in triggers: ERROR, followed by ICMP_TIMEOUT. For more information about the order in which NerveCenter fires built-in triggers, see Built-in Trigger Firing Sequence.

After the configured number of retries is exceeded, NerveCenter examines the error list, determines which of the matching errors occurred most often, and selects the last packet received from that set. If there is a tie between two or more types of errors, NerveCenter selects the last error packet received. (NerveCenter does not accumulate timeouts. One or more timeouts is counted as only one timeout.)

Error details are stored in ICMP/IP fields that NerveCenter includes with each instance of ICMP_ERROR that it fires. Using a Perl subroutine or a NerveCenter poll expression, you can extract this data (Type, Code, Destination Address, and Source Address) to learn more specific information about the ICMP error that occurred.

The exception to this rule is when NerveCenter receives an ICMP error that contain values for a net, host, or port unreachable condition (where the ICMP fields Type = 3 and Code = 0, 1, or 3). In this situation, NerveCenter fires an ERROR built-in trigger first, followed by an ICMP_ERROR trigger, and then finally either a NET_UNREACHABLE, NODE_UNREACHABLE, or PORT_UNREACHABLE built-in trigger.

If the poll times out, NerveCenter fires two built-in triggers: ERROR, followed by either an ICMP_TIMEOUT or SNMP_TIMEOUT trigger.

Multiple Errors Examples

For example, you poll a node with addresses A1, A2, A3, A4 and A5 with the number of retries set to three in the NerveCenter Administrator. The replies are as follows:

Original response = ICMP error E1 from address A1

Response from First retry = ICMP error E1 from address A2

Response from Second retry = no reply within retry interval from address A3

Response from Third retry = ICMP error E2 from address A4

Even though error E4 (third retry) was the last error received, NerveCenter discards it and uses error E1 to produce a response, because it occurred most often. The actual data packet that NerveCenter returns with error E1 is from the first retry, because NerveCenter retains only the last packet for each error code. (The packet from the first retry overwrote the packet from the original response because their error codes matched.)

In this example if any of the ICMP errors contain values for a net, host, or port unreachable condition (where the ICMP fields Type = 3 and Code = 0, 1, or 3), NerveCenter fires an ERROR built-in trigger first, followed by an ICMP_ERROR trigger, and then finally either a NET_UNREACHABLE, NODE_UNREACHABLE, or PORT_UNREACHABLE built-in trigger. If error E1 is any other ICMP error, then NerveCenter fires two triggers: first, an ERROR built-in trigger, followed by an ICMP_ERROR built-in trigger that contains data from the first retry packet. For more information about the order in which NerveCenter fires built-in triggers, see Built-in Trigger Firing Sequence.

Consider a second example in which the replies are as follows:

Original response = ICMP error E1 from address A1

Response from First retry = ICMP error E2 from address A2

Response from Second retry = ICMP error E3 from address A3

Response from Third retry = no reply within retry interval from address A4

NerveCenter uses error E3 to produce a response because it was the last error received, and no error type occurred more than once. Even though a timeout occurred on the last response, NerveCenter discards it because an error takes precedence over a timeout.

Built-in Trigger Firing Sequence

NerveCenter Built-in Trigger Firing Sequence shows the order in which NerveCenter fires built-in triggers.

NerveCenter Built-in Trigger Firing Sequence

Matching Errors with Pending SNMP and Ping Requests

Each poll packet that NerveCenter sends on a socket includes a unique identifier (the IP field Sequence Number). When a poll returns ICMP errors within its configured number of retries, NerveCenter collects the error messages that are returned. Each error message includes the sequence number as well as the destination address of the associated node. Certain fields in the ICMP error packet enable NerveCenter to attempt to match SNMP/ICMP error messages with a poll's pending SNMP/ping requests as follows:

• NerveCenter compares a reply on the SNMP socket to its list of pending SNMP requests and attempts to match the reply with the sequence number of an SNMP request. If a match cannot be found with a pending SNMP request, then NerveCenter discards the reply.
• NerveCenter compares a reply on the ICMP socket to its list of pending ICMP requests and attempts to match the reply with the sequence number of an ICMP request. Matching ICMP Replies with ICMP Requests summarizes how NerveCenter attempts to match ICMP replies to ICMP pending requests:

  Matching ICMP Replies with ICMP Requests

  Sequence Number Match?	Destination Address In DB?	Action
  Yes	Yes	NerveCenter fires the appropriate built-in trigger for the poll.
  No	Yes	NerveCenter saves reply to attempt to match with a pending SNMP request.
  No	No	NerveCenter discards the reply.

  If NerveCenter cannot match the sequence number of an ICMP reply with any pending ICMP requests, but NerveCenter recognizes the destination address, the reply is saved because it might be an error response to an SNMP request for that node; therefore, at regular intervals, NerveCenter compares the destination address of saved ICMP error replies with pending SNMP requests. NerveCenter attempts to match each ICMP reply with the destination address of the oldest pending SNMP request. Only after attempting to match ICMP replies with both pending ICMP and SNMP requests does NerveCenter finally discard the reply when it finds no matches.

Multi-homed Nodes

Polling multi-homed nodes will cause NerveCenter to rotate through the address list for that node in the following manner. If the first address returns an ICMP error response, then NerveCenter flags that address as "down" and will not retry the address until NerveCenter has tried all other addresses for this node.

Upon each retry of a poll, NerveCenter chooses the next IP address to poll. If a node has more addresses than the number of allowable retries, then second or subsequent polls of that node will use the current address if it is "up" or the next un-tried address in the list. If all addresses have been tried, then the "down" addresses will be used again. For an SNMP error, NerveCenter flags the address as "up" because NerveCenter did receive a response from the node's agent.

A List of Built-In Triggers

Built-In Triggers lists all the built-in triggers that NerveCenter can fire.

---

   NerveCenter uses all uppercase letters to designate built-in trigger names.

Built-In Triggers

Trigger Name	Meaning
CANNOT_SEND	A local error occurred while NerveCenter was trying to send an SNMP message.
ERROR	An SNMP or ICMP request did not result in a valid response. After firing the ERROR trigger, NerveCenter fires a second trigger that indicates the specific nature of the error.
ICMP_ERROR	Indicates an ICMP error. The ICMP_ERROR trigger contains the ICMP/IP fields from the error message.
ICMP_TIMEOUT	NerveCenter sent an ICMP ping to a node and did not receive a response. This trigger generally indicates that the node in question is down. NerveCenter uses the number of retries and retry interval specified on the SNMP tab in the Administrator. Refer to the Managing NerveCenter guide for details.
ICMP_UNKNOWN_ERROR	NerveCenter sent an ICMP ping to a node and received an invalid response. This trigger is no longer used except for the purpose of backward compatibility with version 3.5. We recommend you use it sparingly in the current version.
INFORM_CONNECTION_DOWN	A NerveCenter Inform host connection with OVPA is down.
INFORM_CONNECTION_UP	A NerveCenter Inform host connection with OVPA was down but is now back up.
INFORMS_LOST	The number of NerveCenter Informs that were unacknowledged and lost, usually while the inform host connection with OVPA was down.
NET_UNREACHABLE	Indicates that the IP routing layer could not find a route to the network containing the polled node, usually because at least one router was down. This trigger indicates nothing about the status of the node. This trigger can be issued only if you have a router between the workstation running NerveCenter and the polled node.
NODE_UNREACHABLE	Indicates that the IP routing layer could not find a route to the destination node. This trigger indicates nothing about the status of the node. This trigger can be issued only if you have a router between the workstation running NerveCenter and the polled node.
PORT_UNREACHABLE	NerveCenter sent a message to a node, and there was no response from the port to which the message was sent.
RESPONSE	NerveCenter sent an SNMP message and received a valid response from the agent on the destination node.
SNMP_AUTHORIZATIONERR	An SNMP v3 authorization error caused because there is a mismatch between one or all of the rows of vacmAccessTable and the packet. Reasons include: context name mismatch (vacmAccessContextPrefix); security model is not used (vacmAccessSecurityModel); incorrect security level (vacmAccessSecurityLevel); unauthorized to read the MIB view for the SNMP context (vacmAccessReadViewName); unauthorized to write to the MIB view for the SNMP context (vacmAccessWriteViewName); unauthorized to notify the MIB view for the SNMP context (vacmAccessNotifyViewName)
SNMP_BADVALUE	NerveCenter tried to set the value of an attribute in a MIB, but the value it supplied was inappropriate for the attribute. The value may have been of the wrong type, of the wrong length, or invalid for some other reason.
SNMP_DECRYPTION_ERROR	The SNMP v3 engine dropped packets because they could not be decrypted. The 32-bit counter, usmStatsDecryptionErrors, is greater than zero.
SNMP_ENDOFTABLE	NerveCenter fires SNMP_ENDOFTABLE when it finds no more rows while performing an SNMP walk of a MIB table. For example, you could walk IfTable to determine the number of DSO interfaces a node contains.
SNMP_GENERR	A GetRequest, GetNextRequest, or SetRequest failed for some unknown reason (general error).
SNMP_NOSUCHNAME	NerveCenter sent to an SNMP agent a GetRequest, a GetNextRequest, or a SetRequest, and the agent that was contacted was unable to perform the requested operation because: The name of the attribute to be read did not match exactly the name of an attribute available for get operations in the relevant MIB view The name of the attribute to be read did not lexicographically precede the name of an attribute available for get operations in the relevant MIB view The attribute to be set was not available for set operations in the relevant MIB view
SNMP_NOT_IN_TIME_WINDOW	The SNMP v3 engine dropped packets because the boots and timeticks sent in the PDU appeared outside of the authoritative SNMP agent's time window. The 32-bit counter, usmStatsNotInTimeWindows, is greater than zero.
SNMP_READONLY	The error readOnly is not defined in RFC 1157. However, some vendors' agents do use this error-status code. As the name implies, the error usually indicates that an agent has received a SetRequest (from NerveCenter, in this case) for an attribute whose access type is read only.
SNMP_TIMEOUT	NerveCenter sent an SNMP message to an agent and did not receive a response. This trigger indicates either that a node's SNMP agent is down or that the node itself is down. NerveCenter uses the number of retries and retry interval specified on the SNMP tab in the Administrator. Refer to the Managing NerveCenter guide for details.
SNMP_TOOBIG	An SNMP agent did not respond normally to a GetRequest, GetNextRequest, or SetRequest from NerveCenter because the size of the required GetResponse would have exceeded a local limitation.
SNMP_UNAVAILABLE_CONTEXT	The SNMP v3 engine dropped packets because the context contained in the message was unavailable. The 32-bit counter, snmpUnavailableContexts, is greater than zero.
SNMP_UNKNOWN_CONTEXT	The SNMP v3 engine dropped packets because the context contained in the message was unknown. The 32-bit counter, snmpUnknownContexts, is greater than zero.
SNMP_UNKNOWN_ENGINEID	The SNMP v3 engine dropped packets because they referenced an snmpEngineID that was not known to the SNMP v3 engine. The 32-bit counter, usmStatsUnknownEngineIDs, is greater than zero.
SNMP_UNKNOWN_USERNAME	The SNMP v3 engine dropped packets because they referenced a user that was not known to the SNMP v3 engine. The 32-bit counter, usmStatsUnknownUserNames, is greater than zero.
SNMP_UNSUPPORTED_SEC_LEVEL	The SNMP v3 engine dropped packets because the requested security level is unknown or unavailable. The 32-bit counter, usmStatsUnsupportedSecLevels, is greater than zero.
SNMP_WRONG_DIGEST	The SNMP v3 engine dropped packets because they didn't contain the expected digest value. The 32-bit counter, usmStatsWrongDigests, is greater than zero.
UNKNOWN_ERROR	Some other error occurred.

---

---

One additional trigger, USER_RESET, is not available from the list of built-in triggers in NerveCenter. NerveCenter fires USER_RESET to trigger another state for an existing alarm instance when you reset the alarm instance using the right-click pop-up menu in the Alarm Summary or Aggregate Alarm Summary windows.