White Papers - Open NerveCenter: Downstream Alarm Suppression - Understanding the Technical Details - Alarms -
Previous: Understanding the Technical Details     Next: Perl Subroutines

Alarms

The downstream alarm suppression behavior model monitors node status using both SNMP and ICMP. This section includes descriptions of the following alarms:

• DwnStrmSnmpStatus Alarm
• DwnStrmIcmpStatus Alarm

  ---

  Note    You must import the downstream behavior models before they become available in NerveCenter Client. From the Server menu in Client, choose Import Objects and Nodes, and then browse to select the node status models.

  ---

DwnStrmSnmpStatus Alarm

This alarm accurately monitors the status of nodes and their SNMP agents by taking into consideration the status of the nodes' parents. This alarm is the same as the DwnStrmSnmpStatus_LogToDB version, except that the DwnStrmSnmpStatus_LogToDB version also logs data on most transitions.

DwnStrmSnmpStatus/DwnStrmSnmpStatus_LogToDB Alarm State Diagram

The following table lists the severity of each state:

Severities of Each State in DwnStrmSnmpStatus

When this alarm is turned on, the following polls and masks cause state transitions:

• ColdStart (trap mask)
• SnmpFastPoll (SNMP get request)
• SnmpPoll (SNMP get request)
• SS_IcmpFastPoll (ICMP echo request, or ping)
• SS_IcmpPoll (ICMP echo request, or ping)
• WarmStart (trap mask)

This alarm uses the following Perl subroutines:

• SS_IcmpError Perl Subroutine
• SetNodeStatus Perl Subroutines
• TestParentStatus Perl Subroutine
• TestParentSetNode Perl Subroutine

  ---

  Note    Before turning this alarm on, NerveCenter must have loaded the relationship data. SeeIdentifying Parent-Child Relationships and Making the Relationship Information Available to NerveCenter.

  ---

The following sections describe the states in the DwnStrmSnmpStatus alarm and the transitions and actions that can happen from those states:

• Ground State
• Error State
• Testing State
• AgentDown State
• Unreachable State
• DeviceDown State

Ground State

In Ground state, the node is reachable and the SNMP agent is up.

As long as the node and agent respond to the SnmpPoll and SnmpFastPoll requests, the agentUp circular transition is triggered. The agentUp transition calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to refresh the update time.

If the node does not respond to the polls, the following triggers can transition the alarm from Ground to Error:

• ICMP_ERROR
• SNMP_TIMEOUT

Transitions to the Error state call the SetNodeStatusTesting Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status to Testing.

ICMP_ERROR also calls the SS_IcmpError Perl subroutine (see SS_IcmpError Perl Subroutine). If the SS_IcmpError Perl subroutine determines that the port is unreachable, it fires SS_PortUnreach. The SS_PortUnreach trigger does the following:

• Transitions the alarm to an AgentDown state
• Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the agent is down
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

Error State

The alarm suppression behavior model uses the Error state to confirm that there is actually a problem (as opposed to a dropped packet, for example). From the Error state, a node can transition back to Ground, to Testing, or to AgentDown.

If the node and agent respond to the SnmpFastPoll request, the agentUpFast transition is triggered. The agentUpFast transition does the following:

• Returns the alarm to Ground state
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If the node still does not respond to the poll, the following triggers transition the alarm from Error to Testing:

• ICMP_ERROR, which calls the SS_IcmpError Perl subroutine (see SS_IcmpError Perl Subroutine)
• SNMP_TIMEOUT

If the SS_IcmpError Perl subroutine determines that the port is unreachable, it fires SS_PortUnreach. The SS_PortUnreach trigger does the following:

• Transitions the alarm to an AgentDown state
• Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the agent is down
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

Testing State

While an alarm is in the Testing state, NerveCenter identifies whether the node is:

• Down
• Unreachable
• Up, but its agent is down

If SS_PortUnreach is triggered by the SS_IcmpError Perl subroutine while the node transitioned from Error to Testing or if SS_nodeUpFast is triggered by SS_IcmpFastPoll, the trigger:

• Transitions the alarm to an AgentDown state
• Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the agent is down
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If SSF_IcmpError is triggered by SS_IcmpFastPoll, the trigger:

• Transitions the alarm to an Unreachable state
• Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the node is unreachable
• Calls the SetNodeStatusUnreachable Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If SS_nodeUpFast results in a circular ICMP_TIMEOUT transition, the TestParentStatus Perl subroutine (see TestParentStatus Perl Subroutine) looks up the status of the parents. If TestParentStatus can determine the node's state based on the parents' status, TestParentStatus fires the appropriate trigger: UnReachable or Down.

• The UnReachable trigger:
  • Transitions the alarm to an Unreachable state
  • Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the node is unreachable
  • Calls the SetNodeStatusUnreachable Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately
• The Down trigger:
  • Transitions the alarm to a DeviceDown state
  • Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the node is unreachable
  • Calls the SetNodeStatusDown Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately
  • Sends an Inform action to notify a network management platform or another NerveCenter of the status of this node

AgentDown State

While an alarm is in the AgentDown state, NerveCenter continues to monitor the node for any changes.

As long as the node responds to the SS_IcmpPoll requests, the SSnodeUp transition is triggered. The SSnodeUp transition calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to refresh the update time.

If the node does not respond to the polls, the following triggers transition the node from AgentDown to Testing:

• ICMP_TIMEOUT
• SS_IcmpError

Each transition calls the SetNodeStatusTesting Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status to Testing.

If NerveCenter receives a warmStart trap or a coldStart trap, or agentUp is triggered in response to an SnmpPoll response, the trigger:

• Transitions the alarm to a Ground state
• Uses the Set Attribute action to turn poll suppression off so NerveCenter can resume all normal polling
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the time of the last status change so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

Unreachable State

While an alarm is in the Unreachable state, NerveCenter continues to monitor the node for any changes.

If NerveCenter receives a coldStart trap or SSnodeUp is triggered by a response to SS_IcmpPoll, the trigger:

• Transitions the alarm to a Ground state
• Uses the Set Attribute action to turn poll suppression off so NerveCenter can resume all normal polling
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If the poll does not get a response and an ICMP_TIMEOUT transition is triggered, NerveCenter calls the TestParentSetNode Perl subroutine (see TestParentSetNode Perl Subroutine), which looks up the status of the parents. If TestParentSetNode can determine the node's state based on the parents' status, TestParentSetNode fires the Down trigger or refreshes the node's update time.

The Down trigger:

• Transitions the alarm to a DeviceDown state
• Calls the SetNodeStatusDown Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately
• Sends an Inform action to notify a network management platform or another NerveCenter of the status of this node

DeviceDown State

While an alarm is in the DeviceDown state, NerveCenter continues to monitor the node for any changes.

If NerveCenter receives a coldStart trap or the SSnodeUpFast transition is triggered by an SS_IcmpFastPoll, the trigger:

• Transitions the alarm to a Ground state
• Uses the Set Attribute action to turn poll suppression off so NerveCenter can resume all normal polling
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the time of the last status change so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If SSF_IcmpError is triggered by SS_IcmpFastPoll, the trigger:

• Transitions the alarm to an Unreachable state
• Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the node is unreachable
• Calls the SetNodeStatusUnreachable Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If the poll does not get a response and a circular ICMP_TIMEOUT transition is triggered, NerveCenter calls the TestParentSetNode Perl subroutine (see TestParentSetNode Perl Subroutine), which looks up the status of the parents. If TestParentSetNode can determine the node's state based on the parents' status, TestParentStatus fires the Unreachable trigger or refreshes the node's update time.

The Unreachable trigger:

• Transitions the alarm to an Unreachable state
• Calls the SetNodeStatusUnReachable Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

DwnStrmIcmpStatus Alarm

This alarm accurately monitors the status of nodes by taking into consideration the status of the nodes' parents. This alarm is the same as the DwnStrmIcmpStatus_LogToDB version, except that the DwnStrmIcmpStatus_LogToDB version also logs data on most transitions.

DwnStrmIcmpStatus/DwnStrmIcmpStatus_LogToDB Alarm State Diagram

The following table lists the severity of each state:

Severities of each state in DwnStrmSnmpStatus

When this alarm is turned on, the following polls cause state transitions:

• IS_IcmpFastPoll (ICMP echo request, or ping)
• IS_IcmpPoll (ICMP echo request, or ping)

This alarm uses the following Perl subroutines:

• SetNodeStatus Perl Subroutines
• TestParentStatus Perl Subroutine
• TestParentSetNode Perl Subroutine

  ---

  Note    Before turning this alarm on, NerveCenter must have loaded the relationship data. SeeIdentifying Parent-Child Relationships and Making the Relationship Information Available to NerveCenter.

  ---

The following sections describe the states in the DwnStrmIcmpStatus alarm and the transitions and actions that can happen from those states:

• Ground State
• Error State
• Testing State
• Unreachable State
• DeviceDown State

Ground State

In Ground state, the node is reachable.

As long as the node responds to the IS_IcmpPoll requests, the ISnodeUp transition is triggered. The ISnodeUp transition calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to refresh the update time.

If the node does not respond to the polls, the following triggers can transition the alarm from Ground to Error:

• ICMP_TIMEOUT
• IS_IcmpError

Transitions to the Error state call the SetNodeStatusTesting Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status to Testing.

Error State

The alarm suppression behavior model uses the Error state to confirm that there is actually a problem (as opposed to a dropped packet, for example). From the Error state, an alarm can transition back to Ground or to Testing.

If the node responds to the IS_IcmpFastPoll request, the ISnodeUpFast transition is triggered. The trigger:

• Returns the alarm to Ground state
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If the node still does not respond to the poll, the following triggers transition the alarm from Error to Testing:

• ICMP_TIMEOUT
• ISF_IcmpError

Testing State

While an alarm is in the Testing state, NerveCenter identifies whether the node is down or unreachable.

If ISnodeUpFast is triggered in response to an IS_IcmpFastPoll poll while the node is in the Testing state, the trigger:

• Transitions the alarm to Ground
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If ISF_IcmpError is triggered in response to an IS_IcmpFastPoll poll while the node is in the Testing state, the trigger:

• Transitions the alarm to Unreachable
• Calls the SetNodeStatusUnreachable Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If ISnodeUpFast results in a circular ICMP_TIMEOUT transition, NerveCenter calls the TestParentStatus Perl subroutine (see TestParentStatus Perl Subroutine) looks up the status of the parents. If TestParentStatus can determine the node's state based on the parents' status, TestParentStatus fires the appropriate trigger: UnReachable or Down.

• The UnReachable trigger:
  • Transitions the alarm to an Unreachable state
  • Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the node is unreachable
  • Calls the SetNodeStatusUnreachable Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately
• The Down trigger:
  • Transitions the alarm to a DeviceDown state
  • Uses the Set Attribute action to suppress the node so the node won't be polled by suppressible polls while the node is unreachable
  • Calls the SetNodeStatusDown Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately
  • Sends an Inform action to notify a network management platform or another NerveCenter of the status of this node

Unreachable State

While an alarm is in the Unreachable state, NerveCenter continues to monitor the node for any changes.

If ISnodeUp is triggered by a response to IS_IcmpPoll, the trigger:

• Transitions the alarm to a Ground state
• Uses the Set Attribute action to turn poll suppression off so NerveCenter can resume all normal polling
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If the poll does not get a response and a circular ICMP_TIMEOUT transition is triggered, NerveCenter calls the TestParentSetNode Perl subroutine (see TestParentSetNode Perl Subroutine), which looks up the status of the parents. If TestParentSetNode can determine the node's state based on the parents' status, TestParentSetNode either fires the Down trigger or refreshes the node's update time.

The Down trigger:

• Transitions the alarm to a DeviceDown state
• Calls the SetNodeStatusDown Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately
• Sends an Inform action to notify a network management platform or another NerveCenter of the status of this node

DeviceDown State

While an alarm is in the DeviceDown state, NerveCenter continues to monitor the node for any changes.

If the ISnodeUpFast transition is triggered by an IS_IcmpFastPoll, the trigger:

• Transitions the alarm to a Ground state
• Uses the Set Attribute action to turn poll suppression off so NerveCenter can resume all normal polling
• Calls the SetNodeStatusUp Perl subroutine (see SetNodeStatus Perl Subroutines) to update the time of the last status change so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If ISF_IcmpError is triggered in response to an IS_IcmpFastPoll poll while the node is in the Testing state, the trigger:

• Transitions the alarm to Unreachable
• Calls the SetNodeStatusUnreachable Perl subroutine (see SetNodeStatus Perl Subroutines) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately

If the poll does not get a response and a circular ICMP_TIMEOUT transition is triggered, the TestParentSetNode Perl subroutine (see TestParentSetNode Perl Subroutine), which looks up the status of the parents. If TestParentSetNode can determine the node's state based on the parents' status, TestParentSetNode fires the Unreachable trigger or refreshes the node's update time.

The Unreachable trigger:

• Transitions the alarm to an Unreachable state
• Calls the SetNodeStatusUnReachable Perl subroutine (see TestParentStatus Perl Subroutine) to update the status so that NerveCenter can evaluate the states of the children of this node, if there are any, accurately