> For the complete documentation index, see [llms.txt](https://docs.gxc.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.gxc.io/docs/network-configuration-and-operations-guides/4g-onyx-portal-4.x-alerts-reference.md).

# 4G Onyx Portal 4.x Alerts Reference

This reference describes Onyx Portal 4G alerts listed in order of type, module, and name in alphabetical order.

<p align="center"><strong>Onyx Portal 4G Predefined Alerts</strong></p>

<table><thead><tr><th width="162.8333740234375">Type</th><th width="150">Module</th><th width="324.5">Alert Name</th><th>Severity</th></tr></thead><tbody><tr><td>Access Point</td><td>C-Plane</td><td><a href="#neighbor-tables-incomplete">Neighbor Tables Incomplete</a></td><td>Major</td></tr><tr><td>Access Point</td><td>C-Plane</td><td><a href="#s1-disconnected">S1 Disconnected</a></td><td>Major</td></tr><tr><td>Access Point</td><td>CU-Plane</td><td><a href="#cell-service-down">Cell Service Down</a></td><td>Major</td></tr><tr><td>Access Point</td><td>CU-Plane</td><td><a href="#das-associated-access-point-cell-service-down">DAS Associated Access Point Cell Service Down</a></td><td>Major</td></tr><tr><td>Access Point</td><td>CU-Plane</td><td><a href="#ipsec-certificate-expiry">IPSec Certificate Expiry</a></td><td>Minor</td></tr><tr><td>Access Point</td><td>CU-Plane</td><td><a href="#ipsec-connection-down">IPSec Connection Down</a></td><td>Major</td></tr><tr><td>Access Point</td><td>Domain Proxy</td><td><a href="#cbsd-lesser-bw-granted">CBSD Lesser BW Granted</a></td><td>Major</td></tr><tr><td>Access Point</td><td>Domain Proxy</td><td><a href="#ap-cbsd-new-grant">CBSD New Grant</a></td><td>Warning</td></tr><tr><td>Access Point</td><td>Domain Proxy</td><td><a href="#ap-cbsd-preferred-frequency-unavailable">CBSD Preferred Frequency Unavailable</a></td><td>Warning</td></tr><tr><td>Access Point</td><td>Domain Proxy</td><td><a href="#ap-cbsd-registration-failure">CBSD Registration Failure</a></td><td>Major</td></tr><tr><td>Access Point</td><td>M-Plane</td><td><a href="#access-point-offline">Offline</a></td><td>Major</td></tr><tr><td>Access Point</td><td>M-Plane</td><td><a href="#tr-069-session-stuck">TR-069 Session Stuck</a></td><td>Major</td></tr><tr><td>Access Point</td><td>M-Plane</td><td><a href="#unexpected-tr-069-fault">Unexpected TR-069 Fault</a></td><td>Minor</td></tr><tr><td>Access Point</td><td>MCU-Plane</td><td><a href="#cell-unwarranted-transmission">Cell Unwarranted Transmission</a></td><td>Major</td></tr><tr><td>Access Point</td><td>MCU-Plane</td><td><a href="#mesh-filter-frequency-mismatch">Mesh Filter-Frequency Mismatch</a></td><td>Major</td></tr><tr><td>DAS</td><td>Domain Proxy</td><td><a href="#das-tp-cbsd-new-grant">TP CBSD New Grant</a></td><td>Warning</td></tr><tr><td>DAS</td><td>Domain Proxy</td><td><a href="#das-tp-cbsd-preferred-frequency-unavailable">TP CBSD Preferred Frequency Unavailable</a></td><td>Warning</td></tr><tr><td>DAS</td><td>Domain Proxy</td><td><a href="#cbsd-preferred-max-power-unavailable">TP CBSD Preferred Max Power Unavailable</a></td><td>Warning</td></tr><tr><td>DAS</td><td>Domain Proxy</td><td><a href="#das-tp-cbsd-registration-failure">TP CBSD Registration Failure</a></td><td>Major</td></tr><tr><td>Mesh Node</td><td>Domain Proxy</td><td><a href="#cpe-cbsd-new-grant">CPE CBSD New Grant</a></td><td>Warning</td></tr><tr><td>Mesh Node</td><td>Domain Proxy</td><td><a href="#cpe-cbsd-preferred-frequency-or-bandwidth-unavailable">CPE CBSD Preferred Frequency or Bandwidth Unavailable</a></td><td>Warning</td></tr><tr><td>Mesh Node</td><td>Domain Proxy</td><td><a href="#cpe-cbsd-registration-failure">CPE CBSD Registration Failure</a></td><td>Major</td></tr><tr><td>Mesh Node</td><td>M-Plane</td><td><a href="#mesh-node-offline">Offline</a></td><td>Critical</td></tr><tr><td>Mesh Node</td><td>MCU-Plane</td><td><a href="#tunnel-down">Tunnel Down</a></td><td>Major</td></tr><tr><td>Mesh Node</td><td>MCU-Plane</td><td><a href="#watchdog-powercycle">Watchdog Powercycle</a></td><td>Minor</td></tr><tr><td>Mesh Node</td><td>Platform</td><td><a href="#mesh-node-cpu-usage">CPU Usage</a></td><td>Minor</td></tr><tr><td>Mesh Node</td><td>Platform</td><td><a href="#mesh-node-disk-usage">Disk Usage</a></td><td>Major</td></tr><tr><td>Mesh Node</td><td>Platform</td><td><a href="#mesh-node-memory-usage">Memory Usage</a></td><td>Minor</td></tr><tr><td>Mesh Node</td><td>Provisioning Server</td><td><a href="#mesh-node-provisioning-server-connection-failure">Provisioning Server Connection Failure</a></td><td>Warning</td></tr><tr><td>Onyx Edge (Core)</td><td>C-Plane</td><td><a href="#authentication-failure">Authentication Failure</a></td><td>Warning</td></tr><tr><td>Onyx Edge (Core)</td><td>C-Plane</td><td><a href="#dhcp-server-failed">DHCP Server Failed</a></td><td>Major</td></tr><tr><td>Onyx Edge (Core)</td><td>C-Plane</td><td><a href="#s1-setup-failure">S1 Setup Failure</a></td><td>Major</td></tr><tr><td>Onyx Edge (Core)</td><td>C-Plane</td><td><a href="#sctp-outbound-stalled">SCTP Outbound Stalled</a></td><td>Major</td></tr><tr><td>Onyx Edge (Core)</td><td>C-Plane</td><td><a href="#ue-attach-failure">UE Attach Failure</a></td><td>Warning</td></tr><tr><td>Onyx Edge (Core)</td><td>C-Plane</td><td><a href="#ue-imsi-imei-validation-failure">UE IMSI-IMEI Validation Failure</a></td><td>Warning</td></tr><tr><td>Onyx Edge (Core)</td><td>C-Plane</td><td><a href="#ue-session-failure">UE Session Failure</a></td><td>Warning</td></tr><tr><td>Onyx Edge (Core)</td><td>CU-Plane</td><td><a href="#ha-peer-down">HA Peer Down</a></td><td>Major</td></tr><tr><td>Onyx Edge (Core)</td><td>CU-Plane</td><td><a href="#ipsec-service-down">IPSec Service Down</a></td><td>Critical</td></tr><tr><td>Onyx Edge (Core)</td><td>CU-Plane</td><td><a href="#ipsec-service-restarting-frequently">IPSec Service Restarting Frequently</a></td><td>Major</td></tr><tr><td>Onyx Edge (Core)</td><td>M-Plane</td><td><a href="#onyx-edge-core-offline">Offline</a></td><td>Critical</td></tr><tr><td>Onyx Edge (Core)</td><td>MCU-Plane</td><td><a href="#service-down">Service Down</a></td><td>Critical</td></tr><tr><td>Onyx Edge (Core)</td><td>Platform</td><td><a href="#onyx-edge-core-cpu-usage">CPU Usage</a></td><td>Minor</td></tr><tr><td>Onyx Edge (Core)</td><td>Platform</td><td><a href="#onyx-edge-core-disk-usage">Disk Usage</a></td><td>Major</td></tr><tr><td>Onyx Edge (Core)</td><td>Platform</td><td><a href="#onyx-edge-core-memory-usage">Memory Usage</a></td><td>Minor</td></tr><tr><td>Onyx Edge (Core)</td><td>Provisioning Server</td><td><a href="#onyx-edge-core-provisioning-server-connection-failure">Provisioning Server Connection Failure</a></td><td>Warning</td></tr><tr><td>Orchestrator</td><td>Domain Proxy</td><td><a href="#sas-connection-failure">SAS Connection Failure</a></td><td>Critical</td></tr></tbody></table>

<p align="center"><strong>Onyx Portal 4G Semi-Custom Alerts</strong></p>

<table><thead><tr><th width="162.8333740234375">Type</th><th width="150">Module</th><th width="325.3333740234375">Alert Name</th><th>Severity</th></tr></thead><tbody><tr><td>Mesh Node</td><td>MCU-Plane</td><td><a href="#mesh-node-rsrp-lower-than-threshold">Mesh Node &#x3C;serial number> RSRP is Lower than Configured Threshold for 5 Minutes</a></td><td>Major</td></tr><tr><td>Mesh Node</td><td>MCU-Plane</td><td><a href="#mesh-node-rsrq-lower-than-threshold">Mesh Node &#x3C;serial number> RSRQ is Lower than Configured Threshold for 5 Minutes</a></td><td>Major</td></tr><tr><td>Mesh Node</td><td>MCU-Plane</td><td><a href="#mesh-node-sinr-lower-than-threshold">Mesh Node &#x3C;serial number> SINR is Lower than Configured Threshold for 5 Minutes</a></td><td>Major</td></tr></tbody></table>

***

## Common Indications

### Alerts Counter Panel Indication

The Onyx Portal Alerts Counter panel displays separate counters for active Critical, Major, and Minor alerts. All other active alerts are grouped under the "Other" counter.

When an alert is triggered, the corresponding counter's value increases by one.

Clicking the Alerts counter panel takes you to the **Alerts** > "Active Alerts" page.

To view an alert's details, click the alert's name.

### Cell Transmission Down Indications

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: Depending on the percentage of affected APs, the icon will be ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FWe1lF03eEjxcTrN0F08O%2Ficon-service-status-amber.png?alt=media\&token=837ad470-9142-40ea-837a-c3b6ea0a043c) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FKdErmYpT8GAMAOmc35de%2Ficon-service-status-red.png?alt=media\&token=ef7101e6-d2dc-4de1-adda-7b85373c2ab3) red.
* Onyx Portal > **Dashboard** > **Subscribers** tile: If the AP results in loss of online subscribers, the connected subscribers count will change.
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: The traffic reduces as the AP is no longer serving.
* Onyx Portal > **Dashboard** > **Data Usage** tile: The usage reduces as the AP is no longer serving.
* Onyx Portal > **Dashboard** > **Access Point SAS Status** tile: Authorized AP count may reduce if this is due to a grant issue. Other counts may increase depending on the issue.
* Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Overview** tab > **Access Point Details** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FHDym4RYJ1k0KqdOA4zaQ%2Ficon-service-status-red.png?alt=media\&token=b1381a35-efb9-4206-9050-5f861fd14c58) red.

***

## Predefined Alerts

{% hint style="info" %}
Predefined alert rules are delivered preconfigured with the Onyx Solution and can only be created, modified, or deleted by GXC.
{% endhint %}

## Access Point Alerts

### Access Point C-Plane Alerts

#### Neighbor Tables Incomplete

**Trigger Condition**

When the AP is unable to populate all Neighbor Frequency Entries or Neighbor Cell Table Entries. The AP has a limit of 8 Neighbor Frequency Entries and 16 Neighbor Cell Table Entries.

**Probable Causes**

Number of discovered neighbors exceeds the AP's table capacity limits.

**Clear Condition**

When all neighbor entries are successfully populated within the table limits.

**Severity**

Major

**System Impact**

Handovers to neighboring cells may fail, potentially causing dropped connections or service degradation for UE moving between coverage areas.

**Indications**

The AP Events page will log an event identifying which Neighbor Frequency Entries or Neighbor Cell Table Entries could not be populated.

**Corrective Actions**

Configure explicit neighbors in the network settings and limit the neighbor count to only the actual neighboring cells to which handover is expected.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Alerts** > **Active Alerts** tab: The alert has cleared.

***

#### S1 Disconnected

**Trigger Condition**

When an AP reports that it is not S1-connected via TR-069 for at least 15 minutes.

**Probable Causes**

* The AP has an issue.
* On the Onyx Edge, the MME/SCTP services are down.

**Clear Condition**

When the AP reports that it is S1-connected.

**Severity**

Major

**System Impact**

The AP stops transmitting.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fm1p4m8quSwwCDVZzrq9S%2Ficon-service-status-red.png?alt=media\&token=1235ab65-85e0-4d13-8890-9f0dadcd1e44) red, although it may take some time to change.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Online** icon: If a Mesh Node is connected via this AP, depending on the percentage of Mesh Nodes affected, the **Online** icon will turn ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FLIx5hT9nbOazfz5ejp1L%2Ficon-online-status-red.png?alt=media\&token=7957a723-25a0-4d71-8a51-a18fd9043e7b) red or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fbk4y8swJJ1nqdmAVrpKq%2Ficon-online-status-amber.png?alt=media\&token=fb82e757-16f2-43d7-aac2-45c0c6901f63) amber, although it may take some time.
* Onyx Portal > **Dashboard** > **Subscribers** tile: If the AP results in loss of online subscribers, the connected subscribers count will change.
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: The traffic will reduce as this AP is no longer serving.
* Onyx Portal > **Dashboard** > **Data Usage** tile: The usage will reduce as this AP is no longer serving.
* Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Overview** tab > **Service Status** and **Interface Status** icons: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FildlFpLVdN53RD6cZKnT%2Ficon-service-status-red.png?alt=media\&token=6d1f9041-788d-4b06-b64f-446f9abbdcfa) / ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FYbjfrpRDkj9a42ElvBvt%2Ficon-interface-status-red.png?alt=media\&token=a38dda7c-25d8-4fe3-877d-50147021db6a) red, although it may take some time for the **Service Status** icon to change.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Overview** tab > **Onyx Edge Details** tile > **Interface Status** icon: Depending on the number of APs that disconnected, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F0tPOPvuovncha5EUJHm7%2Ficon-interface-status-red.png?alt=media\&token=3a74b244-b5cb-46a6-97ca-39659db7ad99) red or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Far4xPhlgZYVYSoxtNR4L%2Ficon-interface-status-amber.png?alt=media\&token=2cdbf3e5-7691-414f-91f7-87b5eee5e774) amber.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Online** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FmQnXwgnmSvUTsr9uEB5G%2Ficon-online-status-red.png?alt=media\&token=6c5d575b-f98f-440a-bb73-d5ed6f3ff1f8) red, if it is connected via the affected AP.

**Corrective Actions** (perform in order)

1. Check if the Onyx Edge's services are healthy. If the services are restarting, contact GXC Technical Support.
2. Check if any maintenance activity is currently in progress.
3. Check the AP's IPSec status (if applicable).
4. Check the AP's events and alerts and resolve any faults (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Events & Alerts** tab). Any faults will be represented as "unexpected\_fault\_received" event, and new alerts as "enb\_alarm\_raised" event.
5. Check the AP's logs and resolve any errors (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Logs** tab).
6. Reboot the AP.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

**Related Alerts**

* Onyx Edge (Core) MCU-Plane [Service Down](#service-down) Alert — Triggered if all APs connected to the Onyx Edge are S1 Disconnected.

***

### Access Point CU-Plane Alerts

#### Cell Service Down

**Trigger Condition**

When an AP cell does not transmit for at least 15 minutes.

**Probable Causes**

The grant is either unavailable or unauthorized, preventing the AP from transmitting.

**Clear Condition**

When the cell starts transmitting again.

**Severity**

Major

**System Impact**

The AP stops transmitting.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* See [Cell Transmission Down Indications](#cell-transmission-down-indications).

**Corrective Actions**

1. Check if any maintenance activity is currently in progress.
2. Check the AP's IPSec status (if used).
3. Check the AP's events and alerts and resolve any faults (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Events & Alerts** tab). Any fault will be represented as an "unexpected\_fault\_received" event, and new alert as an "enb\_alarm\_raised" event.
4. Check the AP's logs and resolve any errors (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Logs** tab).
5. If SAS is enabled, check if grants are available and authorized for the AP to transmit (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Overview** tab > SAS Status display).
6. Check the AP's DP logs for any failures and take necessary action (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Logs** tab > **DP Logs** section).
7. If grants are not available or not authorized, if possible, change the preferences to obtain grants (Onyx Portal > **Equipment** > **Access Point** tab > edit AP's details > **RAN** tab).
8. If grants are still not available for the minimum acceptable preferences, transmission will not be possible until resources become available with SAS again.
9. Reboot the AP.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

***

#### DAS Associated Access Point Cell Service Down

**Trigger Condition**

Cell associated with a DAS has not transmitted for at least five minutes.

**Probable Causes**

Grants for all DAS Transmission Points (TPs) are unavailable or unauthorized.

**Clear Condition**

When the cell resumes transmission.

**Severity**

Major

**System Impact**

The AP and all associated DAS TPs stop transmitting.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* See [Cell Transmission Down Indications](#cell-transmission-down-indications).
* The power granted for DAS TPs is lower than what can be configured on the associated AP.

**Corrective Actions** (perform in order)

1. Check if any maintenance activity is in progress.
2. Check the AP's IPSec status (if applicable).
3. Review the AP's events and alerts and resolve any faults (Onyx Portal > **Equipment** > **Access Points** tab > view AP's details > **Events & Alerts** tab). Faults will appear as "unexpected\_fault\_received"; new alerts will appear as "enb\_alarm\_raised".
4. Review AP logs and resolve any errors (Onyx Portal > **Equipment** > **Access Points** tab > view Access Point details > **Logs** tab).
5. Check if grants are available and authorized for all associated DAS TPs to transmit (Onyx Portal > **Equipment** > **DAS** > view DAS's details > SAS Status display).
   1. Check DP logs for per-TP failures (Onyx Portal > **Equipment** > **DAS** > view DAS's details > **Logs** tab > **DP Logs**).
   2. If grants are missing or not authorized, adjust preferences, if possible, to obtain grants (Onyx Portal > **Equipment** > **Access Points** tab > edit AP's details > **RAN** tab).
   3. If grants remain unavailable for the minimum acceptable preferences, transmission will not resume until SAS resources become available.
6. Reboot the AP.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

**Related Alerts**

* DAS TP [CBSD Registration Failure](#das-tp-cbsd-registration-failure)

***

#### IPSec Certificate Expiry

**What Does This Alert Mean?**

* The IPSec certificate for the AP is expiring soon. This will result in the AP losing S1 connectivity with the MME/Onyx Edge.
* Note that this alert only captures the certificate as maintained by the Orchestrator. It is the user's responsibility to upload the current certificate on the AP. This alert does not track the actual certificate in the AP.

**Trigger Condition**

When an AP's IPSec certificate is expiring in three days or less or has already expired.

**Clear Condition**

When a new IPSec certificate is generated and uploaded to the AP.

**Severity**

Minor

**Corrective Actions**

Regenerate a new certificate and upload it to the AP.

***

#### IPSec Connection Down

**What Does This Alert Mean?**

The AP has not yet established a successful IPSec connection with the Onyx Edge. As a result, S1 signaling activity is unlikely to occur.

**Trigger Condition**

IPSec tunnel between AP and Onyx Edge is not yet up or has gone down.

**Clear Condition**

When the IPSec tunnel between the AP and Onyx Edge is up.

**Severity:** Major

***

### Access Point Domain Proxy Alerts

#### CBSD Lesser BW Granted

**Trigger Condition**

When a CBSD has been granted less bandwidth than desired, and it continues to be in this state for at least five minutes.

**Clear Condition**

When either of the following conditions occur:

* The CBSD gets the desired bandwidth in a grant.
* The CBSD loses the grant.

**Severity**

Major

**System Impact**

The network capacity is reduced from what was planned.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: Traffic will reduce as bandwidth is reduced.
* Onyx Portal > **Dashboard** > **Data Usage** tile: Usage will not increase at the expected rate due to reduced bandwidth.

**Corrective Actions**

1. Check if any maintenance activity is currently in progress.
2. In the AP's DP logs, check Spectrum Inquiry Response to determine available frequencies (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Logs** tab > **DP Logs** section).
3. If Strict Preference is enabled, disable it, if possible (Onyx Portal > **Equipment** > **Access Point** tab > edit AP's details > **RAN** tab > **Strict Preference** parameter), or change the frequency preference to obtain frequency with higher bandwidth (Onyx Portal > **Equipment** > **Access Point** tab > edit AP's details > **RAN** tab).
4. Contact GXC Technical Support.

**Resolution Verification**

1. All tiles have returned to their pre-incident state.
2. In the AP's details, verify the granted bandwidth (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Overview** tab > **RAN** tile > **Bandwidth (MHz)** parameter).

**Related Alerts**

Access Point Domain Proxy [CBSD New Grant](#ap-cbsd-new-grant) Alert — Triggered in conjunction with the "CBSD Lesser BW Granted" alert.

***

#### CBSD New Grant <a href="#ap-cbsd-new-grant" id="ap-cbsd-new-grant"></a>

**Trigger Condition**

When new grants have been received within the last one hour.

**Clear Condition**

When no new grants have been received for at least one hour.

**Severity**

Warning

**System Impact**

This alert indicates a new grant has been received from SAS. For an AP that is already transmitting, it could change the transmission frequency, bandwidth, power, etc. The operator should verify the new grant will not affect the required KPIs.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: Depending on the grant authorization and previous transmission state of the AP, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F2xxlqY2KuMufXQuKW3FC%2Ficon-service-status-green.png?alt=media\&token=db1f821f-7450-405b-be9f-461d2dfbef9d) green or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FM1NxR5hHsBqHgbivnbJU%2Ficon-service-status-amber.png?alt=media\&token=33c8f18f-2627-4e1b-abed-e7af8f2ad5ae) amber.
* Onyx Portal > **Dashboard** > **Subscribers** tile:
  * If the new grant comes with a lower power level, subscribers at the edge may lose connection.
  * If the AP was not previously transmitting and the grant got authorized as well, previously unconnected subscribers in the coverage area can get connected.
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: The traffic may change depending on AP's previous transmission state, grant authorization, and UE behavior. For example, a reduced bandwidth will not transmit as much data as before.
* Onyx Portal > **Dashboard** > **Data Usage** tile: The usage may change depending on the AP's previous transmission state, grant authorization, and UE behavior.
* Onyx Portal > **Dashboard** > **Access Point SAS Status** tile: Granted/Authorized CBSDs count may change depending on the CBSD's previous state.
* Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Overview** tab > **RAN** tile: SAS Status will move to Granted and generally to Authorized if no previous grants were available.
* Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Overview** tab > **Access Point Details** tile > **Service Status** icon: If the AP was not transmitting previously, the icon can be ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FcnCwMXwkQWe9TgqAvG9G%2Ficon-service-status-green.png?alt=media\&token=b08e1eb6-d057-4310-85e2-e30839cc9dce) green.

**Corrective Actions**

None.

**Resolution Verification**

None required for initial alert for an AP.

**Related Alerts**

Access Point Domain Proxy [CBSD Lesser BW Granted](#cbsd-lesser-bw-granted) Alert — Triggered if the new grant does not have the desired bandwidth.

**Additional Information**

* This is a notification when a new grant is received from SAS.
* As each grant change is a separate grant success, this alert can also be used to track grant changes. However, note that the very first grant is also tracked.

***

#### CBSD Preferred Frequency Unavailable <a href="#ap-cbsd-preferred-frequency-unavailable" id="ap-cbsd-preferred-frequency-unavailable"></a>

**Trigger Condition**

* In SC/DC mode, none of the frequencies in the preferred list are available with SAS.
* In CA mode, no non-overlapping frequency pair from the preferred list is available with SAS.

**Clear Condition**

Any of the following:

* Preferred frequencies are available again.
* CBSD is deregistered.

**Severity**

Warning

**System Impact**

* Network radio transmissions may not occur as planned, potentially causing interference from neighboring cells.
* This alert indicates that transmission on preferred frequencies is not fully possible. If Strict Preference is enabled, the AP will stop transmitting.
* In other cases, the AP will continue transmitting on whatever frequency is received via SAS grant.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* In case Strict Preference is disabled (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details), the granted frequency would not be part of the preferred list.
* In case Strict Preference is enabled, see [Cell Transmission Down Indications](#cell-transmission-down-indications).

**Corrective Actions**

1. Change the preference list (Onyx Portal > **Equipment** > **Access Point** tab > edit AP's details > **RAN** tab).
2. The available frequency can be found in the "Spectrum Inquiry Response" log type (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Logs** tab > **DP Logs** section > "Spectrum Inquiry Response" log type). Filter for "From SAS To DP logs".

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

#### CBSD Registration Failure <a href="#ap-cbsd-registration-failure" id="ap-cbsd-registration-failure"></a>

**Trigger Condition**

CBSD registrations have failed within the past hour.

**Clear Condition**

No CBSD registrations have failed for at least an hour.

**Severity**

Major

**System Impact**

The AP becomes non-functional, as it cannot obtain any grants to transmit.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > SAS Status: Will show Registered in red ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FXQ7Izs9N5vXoAqD0664M%2Fimage.png?alt=media\&token=750c63fc-94be-4035-8440-9a738790219b). It may show amber in the case of Dual Carrier mode and one CBSD is registered.
* See [Cell Transmission Down Indications](#cell-transmission-down-indications).

**Corrective Actions**

1. Check the AP's DP logs for the failure reason and take necessary action (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Logs** tab > **DP Logs** section).
2. The most probable cause is a mismatch of parameters between the Onyx Portal and the SAS Portal. Verify the CBSD registration on the Federated Wireless Portal has the correct information. Verify the FCC ID, Serial Number, CBSD Type and Category, Location, and Antenna parameters.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

### Access Point M-Plane Alerts

#### Offline <a href="#access-point-offline" id="access-point-offline"></a>

**Trigger Condition**

When no TR-069 Inform messages have been received for the past 20 minutes (15 minutes in alert condition + five minutes for changing the metric). This may occur, for example, if the AP is powered off.

**Clear Condition**

When TR-069 connection is re-established with the AP.

**Severity**

Major

**System Impact**

* Management access to the AP will be down. Configurations, alerts, events, metrics, etc. between the Onyx Portal and the AP will not be in sync.
* User traffic may be unavailable depending on the mode of failure.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Access Point** tile > **Online** icon: This AP is offline. Depending on other APs in the network, the **Online** icon may change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FygQOSD4mSOeelAB3dDgV%2Ficon-online-status-red.png?alt=media\&token=d5d793a9-9200-409f-841f-2aac11dc66c8) red or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FQ1nsY8k3m48Xkz9wri5C%2Ficon-online-status-amber.png?alt=media\&token=a9964ef2-be34-4e9a-929c-cf6249ef975c) amber. Hover information shows the offline APs.
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: Depending on other APs in the network, the **Service Status** icon may change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FWO5yHRDvhWoRGHigRe1R%2Ficon-service-status-grey.png?alt=media\&token=56a09e37-b11f-4cc9-bac1-761b1bd66df8) grey or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FflOL1yikiMpbNSKJjPUk%2Ficon-service-status-amber.png?alt=media\&token=a1913748-63b0-4b9b-8f70-ac003069c22c) amber.
* Onyx Portal > **Dashboard** > **Mesh Node** tile: If the AP service is down and it provides connection for the Mesh Node, the Mesh Node will also go offline.
* Onyx Portal > **Dashboard** > **Subscribers** tile: If the AP results in loss of online subscribers, the connected subscribers count will change.
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: If S1 is down, the traffic will commensurately reduce.
* Onyx Portal > **Dashboard** > **Data Usage** tile: If S1 is down, the usage will commensurately reduce.
* Onyx Portal > **Dashboard** > **Access Point SAS Status** tile: For SAS-enabled APs that are not online for the last four hours:
  * If unregistered, it will not get registered until it is online.
  * Any grants will be relinquished, and the AP will move to Registered state.
* Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Overview** tab > **Access Point Details** tile > **Online** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FGwVfxQAGVZzuKxutX8sk%2Ficon-online-status-red.png?alt=media\&token=f9a063df-d045-47eb-8f76-02da9b5d4fe7) red. All other statuses will turn grey.

**Corrective Actions**

1. Check if any maintenance activity is currently in progress.
2. Check for any power outages at the equipment's location.
3. Verify that the AP is associated with the correct Onyx Edge.
4. Verify network connectivity between the AP and the Onyx Edge.
5. Get assistance from local AP technician.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

**Related Alerts** Onyx Edge (Core) MCU-Plane [Service Down](#service-down) Alert — Triggered if all APs connected to the Onyx Edge are offline.

***

#### TR-069 Session Stuck

**Trigger Condition**

The Access Point TR-069 session has started but not completed for more than 15 minutes, indicating the session is stuck in an error loop.

**Clear Condition**

The TR-069 session completes successfully.

**Severity**

Major

**System Impact**

* Configuration between the AP and the Onyx Portal will not be in sync; as a result, the AP will not function as expected.
* Potential loss of traffic.

**Indications**

* Onyx Portal > Dashboard > common counter indications.
* AP Details page:
  * AP Properties section:
    * **Config/FSM State** parameter will show **STUCK**
    * Hover over Config/FSM State to see Engineering Debug Info:
      * **Last Session Start Time** — Confirms when the session got stuck
      * **Last Session End Time** — Will be old or --, confirming the session did not complete

**Corrective Actions**

Check enodebd logs for errors related to the stuck TR-069 session.

**Resolution Verification**

* From the Dashboard:
  * The Alerts count has returned to the condition prior to the incident.
  * All tiles have returned to the condition prior to the incident.

**Related Alerts**

* [Access Point Offline](#access-point-offline)
* [Access Point Unexpected TR-069 Fault](#unexpected-tr-069-fault)

***

#### Unexpected TR-069 Fault

**Trigger Condition**

When unexpected TR-069 faults are received from AP in the last one hour.

**Probable Causes**

* One or more of the commands issued by the Onyx Edge (enodebd-service) was rejected by the AP.
* This alert could also trigger when grant details and the AP neighbor configurations are not compatible.

**Clear Condition**

When no unexpected TR-069 faults are received from the AP for at least one hour.

**Severity**

Minor

**System Impact**

* Configuration between the AP and the Onyx Portal will be out of sync; as a result, the AP will not be functioning as expected.
* Potential loss of traffic.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Access Point** tile > **Online** icon: If the problem causes the AP to go offline, depending on other APs, the **Online** icon may be ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FXwq1q80kbPjoWszKlxeB%2Ficon-online-status-red.png?alt=media\&token=08f931ec-593d-48d8-afaa-b8f54ede0c9f) red or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F4ak6MIPYSCgh53z5mtQd%2Ficon-online-status-amber.png?alt=media\&token=7a187c1c-1876-43dc-bdea-c9b882ed9552) amber. Hover information shows the offline APs.
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: If the problem repeats on every TR-069 session, the **Service Status** icon can fluctuate between ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FIGaIrd2Y1iTOjmnaIug0%2Ficon-service-status-red.png?alt=media\&token=0521d7df-c57d-4d03-8e01-70c3d58dfc30) red and ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F2OsP0Mzo8ewzEtVd3IAj%2Ficon-service-status-green.png?alt=media\&token=cd914d11-6e72-4281-a301-5e0045cbf0c6) green, or stay red, depending on the point in session at which the fault happens.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Online** icon: If the fault is preventing enabling transmission in a currently non-transmitting AP, the Mesh Node will go offline ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FYhejSkYN5EWzHEhkT6PF%2Ficon-online-status-red.png?alt=media\&token=99aa2737-dc10-4206-8e09-f2b32723ae80).
* Onyx Portal > **Dashboard** > **Subscribers** tile: If the AP results in loss of online subscribers, the connected subscribers count will change.
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: If S1 is down, the traffic will commensurately reduce.
* Onyx Portal > **Dashboard** > **Data Usage** tile: If S1 is down, the usage will commensurately reduce.
* Onyx Portal > **Dashboard** > **Access Point SAS Status** tile: For SAS-enabled APs having recurrent faults for the last four hours:
  * If unregistered, the AP will not get registered until it becomes free of faults.
  * If the AP has grants, they will be relinquished, and the AP will move to Registered state.
* Onyx Portal > **Dashboard** > **Backhaul (SGi) Health** tile: Only affected when AP is already in a non-transmitting state and fault is preventing the Onyx Edge from enabling transmission.

**Corrective Actions**

1. Check if any maintenance activity is currently in progress.
2. Navigate to the Events view (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Events & Alerts** tab > **Events** section). The event associated with the fault typically provides additional details. It is possible that the fault is due to misconfiguration, and the event details usually identify the specific misconfigured parameter.
3. Check the AP's logs and resolve any errors (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Logs** tab).
4. Check and update the specific parameter in the Onyx Portal and the AP equipment.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

**Related Alerts**

Access Point M-Plane [Offline](#access-point-offline) Alert

***

### Access Point MCU-Plane Alerts

#### Cell Unwarranted Transmission

**Trigger Condition**

When even though transmission is disabled, an AP's cell transmits for at least 15 minutes.

**Clear Condition**

When the cell stops transmission.

**Severity**

Major

**System Impact**

* The AP cell may transmit when not expected, potentially causing interference to neighboring radio equipment.
* This may result in a violation of federal spectrum regulations.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fo8hqgv3fZAZuCcS4UGPX%2Ficon-service-status-amber.png?alt=media\&token=5f461f67-c74c-4c02-90a7-d922ff9f173f) amber.

**Corrective Actions**

1. Check if any maintenance activity is currently in progress.
2. Check the AP's events and alerts and resolve any faults (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Events & Alerts** tab). Any faults will be represented as "unexpected\_fault\_received" event, and new alerts as "enb\_alarm\_raised" event.
3. Check the AP's logs and resolve any errors (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Logs** tab).
4. Reboot the AP.
5. If the condition does not clear, contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

***

#### Mesh Filter-Frequency Mismatch

**Trigger Condition**

* The Mesh Node filter and frequency of its attached Mesh AP are in an incompatible state for at least five minutes.
* The frequency for Mesh AP attached to the Mesh Node is not compatible with filter settings on the Mesh Node. The filter setting is determined by the frequency of the Gateway AP to which the Mesh Node is attached.

**Clear Condition**

The Mesh Node filter and frequency of its attached Mesh AP become compatible.

**Severity**

Major

**System Impact**

* The Mesh system will not work, and the user may not get much feedback.
* The Mesh Node small cell filter will reject frequencies used by the Mesh AP, perhaps generating a VSWR fault on the AP, or the small cell hardware may not fault and instead simply fail to connect to any UE in the area (UE signal will be filtered out).
* Some devices can be damaged by Tx energy being returned.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).

**Corrective Actions**

* Check the frequency of the Gateway AP to which the Mesh Node is connected.
  * The frequencies of the Gateway AP and the Mesh Node's Mesh AP must be in opposite halves of the CBRS band (3550–3700 MHz).
  * The CBRS band is divided at 3625 MHz, where Low Band refers to the 3550–3625 MHz range, and High Band refers to the 3625–3700 MHz range.
* If the Mesh AP is in the same half of the CBRS band as the Gateway AP, move the Mesh AP to the opposite half.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

## DAS TP Alerts

### DAS TP Domain Proxy Alerts

#### CBSD New Grant <a href="#das-tp-cbsd-new-grant" id="das-tp-cbsd-new-grant"></a>

**Trigger Condition**

When new grants have been received from SAS in the last hour.

**Clear Condition**

When no new grants have been received for at least an hour.

**Severity**

Warning

**System Impact**

* For a DAS TP that is already transmitting, a new grant may change the transmission frequency, bandwidth, or power level.
* If sufficient power is not granted, transmission may be affected.
* Operators should verify that the new grant parameters will not adversely affect required KPIs.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Dashboard: Common Dashboard Counter Indication.
* DAS details page > SAS Status: Status will change to green or amber based on grant authorization state and the previous transmission state of the DAS TP.
* Subscribers tile:
  * If the new grant comes with a lower power level, subscribers at the edge may lose connection.
  * If the AP was not previously transmitting and the DAS grant got authorized as well, previously unconnected subscribers in the coverage area can get connected.
* CBRS SAS Status tile: The Granted/Authorized CBSDs count may change depending on the CBSD's previous state.
* DAS overview: SAS status will move to Granted and generally to Authorized if no previous grants were available.

**Corrective Actions**

None required. This is an informational alert.

**Resolution Verification**

None required for initial alert.

**Related Alerts**

* DAS TP [CBSD Registration Failure](#das-tp-cbsd-registration-failure)
* DAS TP [CBSD Preferred Frequency Unavailable](#das-tp-cbsd-preferred-frequency-unavailable)

**Additional Information**

* This alert serves as a notification when a new grant is received from SAS.
* Each grant change triggers this alert, allowing operators to track all grant modifications.
* The initial grant for a DAS TP also triggers this alert.

***

#### CBSD Preferred Frequency Unavailable <a href="#das-tp-cbsd-preferred-frequency-unavailable" id="das-tp-cbsd-preferred-frequency-unavailable"></a>

**Trigger Condition**

For DAS TPs, when the preferred frequency selected for the associated AP is unavailable from SAS.

**Clear Condition**

When any of the following occurs:

* The preferred frequency becomes available again.
* The CBSD is deregistered.

**Severity**

Warning

**System Impact**

* The associated AP and DAS TP will stop transmitting because the preferred frequency is not available from SAS.
* Subscribers in the coverage area will lose connectivity.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* DAS Associated AP Overview (Onyx Portal > **Equipment** > **Access Points** tab > view AP's details): The AP's configured frequency will not appear in the available SAS spectrum.

**Corrective Actions**

Change the AP frequency to an available frequency (Onyx Portal > **Equipment** > **Access Points** tab > edit AP > **RAN** tab).

To identify available frequencies:

* View the Available Spectrum graph for the DAS TP (Onyx Portal > **Equipment** > **DAS** tab > view DAS's details), or
* Check the Spectrum Inquiry Response log (Onyx Portal > **Equipment** > **DAS** tab > view DAS's details > **Logs** tab > **DP Logs** section > **Spectrum Inquiry Response** log type, filter for "From SAS To DP" logs).

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

#### CBSD Preferred Max Power Unavailable

**Trigger Condition**

When the preferred maximum power configured for the DAS TP is unavailable from SAS.

**Clear Condition**

When any of the following occurs:

* The preferred maximum power becomes available again.
* The CBSD is deregistered.

**Severity**

Warning

**System Impact**

* Radio transmissions are not operating as planned, potentially due to interference from neighboring cells.
* This alert indicates reduced transmission coverage caused by lower transmit power granted by the SAS.

**Indications**

DAS Associated AP Overview (Onyx Portal > **Equipment** > **DAS** tab > view DAS's details): The **Preferred Max Power** and **Granted Power** values will differ.

**Corrective Actions**

1. Identify available power levels: Check the Spectrum Inquiry Response log (Onyx Portal > **Equipment** > **DAS** tab > view DAS's details > **Logs** tab > **DP Logs** section > **Spectrum Inquiry Response** log type, filter for "From SAS To DP" logs).
2. Change the Preferred Max Power to an available value (Onyx Portal > **Equipment** > **DAS** tab > edit DAS's details > Transmission Point Details).

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Verify from the Dashboard that the alert has cleared.
* Confirm that the **Preferred Max Power** and **Granted Power** values match on the DAS details page (Onyx Portal > **Equipment** > **DAS** tab > view DAS's details).

***

#### CBSD Registration Failure <a href="#das-tp-cbsd-registration-failure" id="das-tp-cbsd-registration-failure"></a>

**Trigger Condition**

When CBSD registrations have failed within the past hour.

**Clear Condition**

When no CBSD registrations have failed for at least an hour.

**Severity**

Major

**System Impact**

The DAS TP is not transmitting because it cannot obtain grants from SAS without successful registration.

**Indications**

DAS Details Page (Onyx Portal > **Equipment** > **DAS** tab > view DAS's details > SAS Status): SAS State will display as Registered in red ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FHQaeqNJGmjHvnpBCD4Uj%2Ficon-sas-status-authorized-red.png?alt=media\&token=3a82933f-954d-4d39-9217-72b1478bb128) for the affected DAS TP.

**Corrective Actions** (perform in order)

1. Check the DAS DP logs for the failure reason and take the required action (Onyx Portal > **Equipment** > **DAS** tab > view DAS's details > **Logs** tab > **DP Logs** section).
2. The most probable cause is a parameter mismatch between the Onyx Portal and the SAS Portal.
3. Verify that the CBSD registration on the Federated Wireless portal has correct information. Verify FCC ID, serial number, CBSD type and category, location, and antenna parameters.

**Resolution Verification**

* Verify from the Dashboard that the alert has cleared.
* Confirm in the DAS details page (Onyx Portal > **Equipment** > **DAS** tab > view DAS's details > SAS Status) that the SAS State displays Registered in green ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FAnZbWCv9fd9UhZe6TuRp%2Fimage.png?alt=media\&token=43c39166-dff7-4e79-8538-63515311c3f0).

***

## Mesh Node Alerts

### Mesh Node Domain Proxy Alerts

#### CPE CBSD New Grant

**Trigger Condition**

When using the GM02 Mesh Node, one or more new grants have been received within the last hour.

**Clear Condition**

When no new grants are received within the last one hour.

**Severity**

Warning

**System Impact**

* This alert is triggered upon receiving a new grant from SAS. For a Mesh Node that is already in transmission, it may result in a change in transmission power.
* Operators should verify that the new grant will not affect the required KPIs.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: Based on grant authorization state and previous transmission state of the Mesh Node, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FOHNDqXplsIgLtKeBXaQL%2Ficon-service-status-green.png?alt=media\&token=b5ec69f6-5d1c-4a53-8a7a-fdac383601d5) green or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FytQM2MHvSYpVZ5GipljL%2Ficon-service-status-amber.png?alt=media\&token=7bc8c920-e4ae-4828-982f-4a035decd410) amber.
* Onyx Portal > **Dashboard** > **Access Point SAS Status** tile: Granted/Authorized CBSDs count may change depending on the CBSD's previous state.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab: If no previous grants were available, SAS Status will move to Granted state and generally to Authorized state.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Service Status** icon: If the Mesh Node was not transmitting previously, it can change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fx9TB9XIgCo80mg2MZJq0%2Ficon-service-status-green.png?alt=media\&token=5ac9ddcf-fa77-49e0-8961-a56bfbe27c88) green.

**Corrective Actions**

None.

**Resolution Verification**

None required for initial alert for a Mesh Node.

***

#### CPE CBSD Preferred Frequency or Bandwidth Unavailable

**Trigger Condition**

When using the GM02 Mesh Node, if the configured backhaul frequency or bandwidth to which the Mesh Node is connected becomes unavailable.

**Clear Condition**

When the preferred frequencies or bandwidth become available again.

**Severity**

Warning

**System Impact**

The Mesh Node is transmitting at reduced capacity and power and will not support any data traffic.

**Indications**

The GM02 Mesh Node will be online but does not have a grant.

**Corrective Actions**

1. Change the Mesh Node's backhaul frequency or bandwidth and reconnect.
2. The available frequency can be found in the "Spectrum Inquiry Response" log type (Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Logs** tab > **DP Logs** section > "Spectrum Inquiry Response" log type). Filter for "From SAS To DP logs".

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

#### CPE CBSD Registration Failure

**Trigger Condition**

When using the GM02 Mesh Node, if one or more CPE CBSD registration attempts have failed within the last one hour.

**Clear Condition**

When no CPE CBSD registration failures occur for at least one hour.

**Severity**

Major

**System Impact**

The Mesh Node will become non-functional, as it cannot obtain any grants to transmit.

**Indications**

* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > SAS Status will show **Registered** in red ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FzLj4bB4Do1fn85XiYSFv%2Fimage.png?alt=media\&token=f749cbdc-3cae-4990-980d-24faf473eb44).
* The Mesh Node will be reported offline. See the Mesh Node M-Plane [Offline](#mesh-node-offline) alert.

**Corrective Actions**

1. Check the Mesh Node's DP logs for the failure reason and take necessary action (Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Logs** tab > **DP Logs** section).
2. The most probable cause is a parameter mismatch between the Onyx Portal and the SAS Portal. Verify the CPE CBSD registration on the Federated Wireless Portal has the correct information. Verify the FCC ID, Serial Number, CBSD Type and Category, Location, and Antenna parameters.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

### Mesh Node M-Plane Alerts

#### Offline <a href="#mesh-node-offline" id="mesh-node-offline"></a>

**Trigger Condition**

When a Mesh Node has remained offline for at least 10 minutes.

**Clear Condition**

When the Mesh Node comes back online.

**Severity**

Critical

**System Impact**

Configurations, alerts, events, metrics, etc. between the Onyx Portal and Mesh Node will not be in sync.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Online** icon: Depending on whether other Mesh Nodes are offline, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FaG2Nh6ANp7rfrIv9hWP0%2Ficon-online-status-amber.png?alt=media\&token=b3bf2124-4a80-4725-998f-243aa070aeae) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FU3shYkxRCNaxLKs5WZWL%2Ficon-online-status-red.png?alt=media\&token=31d58312-8a68-4447-a53c-707b962d9745) red.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Service Status** icon: Depending on whether other Mesh Nodes are offline, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FeTm7hjL3leOH7dqo0Emx%2Ficon-service-status-amber.png?alt=media\&token=9a27f1e5-9eba-4370-8ff7-d058fd623716) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FDYn8wFL6Teltxq7R1pUp%2Ficon-service-status-grey.png?alt=media\&token=894fec5a-5592-4b06-8e67-23a219afb84e) grey.
* Onyx Portal > **Equipment** > **Mesh Node** tab > Mesh Nodes list > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FdNHyf7MRU1e6f2Gx1Kaz%2Ficon-compute-resource-status-red.png?alt=media\&token=5dfd2390-c349-47da-b253-9bf7e1c43fe6) red.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FykxNnTdhG9ThAxt0zMUR%2Ficon-compute-resource-status-red.png?alt=media\&token=403396d5-b350-4d38-8975-69948bcc4b9f) red.

**Corrective Actions**

1. Check the Mesh Node's Internet connectivity.
2. Verify that the subscriber associated with the Mesh Node is connected and active.
3. If Internet connectivity is confirmed, ensure the Mesh Node is properly provisioned. Log on to the Mesh Node and run the `checkin_cli.py` command. Investigate and resolve any errors in the output.
4. Verify power supply to the Mesh Node equipment.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

***

### Mesh Node MCU-Plane Alerts

#### Tunnel Down

**Trigger Condition**

When the Mesh Tunnel between the Mesh Node and the Onyx Edge remains down for at least five minutes.

**Clear Condition**

When the Mesh Tunnel between the Mesh Node and the Onyx Edge is restored.

**Severity**

Major

**System Impact**

The Mesh AP behind the Mesh Node will not be configured, preventing UE from attaching to it.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Mesh Node** tab > Mesh Nodes list > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FzHCm2Xw9fqaBO5ZxAlyD%2Ficon-compute-resource-status-red.png?alt=media\&token=f9547d7e-5625-492b-88a9-d33f58bc3787) red.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FXmwuQOkxa2YCkY98JQuC%2Ficon-compute-resource-status-red.png?alt=media\&token=7c216fbd-b314-4892-9822-0c8f92cf092e) red.

**Corrective Actions**

1. Check the Mesh Node's Internet connectivity.
2. Check if the subscriber associated with the Mesh Node is connected and active.
3. Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

***

#### Watchdog Powercycle

**Trigger Condition**

When the Mesh Node is power cycled by the Watchdog service within the last 10 minutes.

**Clear Condition**

When the Mesh Node Watchdog service has not triggered a power cycle on the Mesh Node for at least 10 minutes.

**Severity**

Minor

**System Impact**

* The mesh network will be affected until the Mesh Node reconnects and comes back online.
* UE will be unable to attach to the Mesh AP.

**Indications**

Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).

**Corrective Actions**

* Watchdog is triggered due to issues with the "cpe\_controller" service or modem identification. If the alert persists, contact GXC Technical Support.
* Check the alert's details for the cause.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

### Mesh Node Platform Alerts

#### CPU Usage <a href="#mesh-node-cpu-usage" id="mesh-node-cpu-usage"></a>

**Trigger Condition**

When a Mesh Node's CPU usage exceeds 70% of capacity in the last five minutes.

**Clear Condition**

When the Mesh Node's CPU usage drops below 70% of capacity.

**Severity**

Minor

**System Impact**

* Delays may occur in signaling and data processing at the Mesh Node.
* If CPU usage continues to rise, unexpected system behavior may occur.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Mesh Node** tab > Mesh Nodes list > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FRUxC3pkamaLwjIVFv0Cc%2Ficon-compute-resource-status-red.png?alt=media\&token=5b38d343-8631-4dca-b6b5-a5652b141d9b) red.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FLzx10kYUXsuhzjGpgngW%2Ficon-compute-resource-status-red.png?alt=media\&token=341872cd-21cf-4459-b7fe-2323e9211917) red.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FKooyzRw3V8hH9uhQijgW%2Ficon-compute-resource-status-green.png?alt=media\&token=f322ccad-d3ef-4216-84d0-d67e384ab7f5) green.

***

#### Disk Usage <a href="#mesh-node-disk-usage" id="mesh-node-disk-usage"></a>

**Trigger Condition**

When a Mesh Node's disk usage exceeds 70% of capacity.

**Clear Condition**

When the Mesh Node's disk usage drops below 70% of capacity.

**Severity**

Major

**System Impact**

Delays in signaling and data processing at the Mesh Node.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Mesh Node** tab > **Mesh Node** list > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FZGq9itSvORArJjDX4Ayl%2Ficon-compute-resource-status-red.png?alt=media\&token=eb84ed92-14b9-440a-99a2-fe1808b452ca) red.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FFuLjvcsLqDuc39C61OuL%2Ficon-compute-resource-status-red.png?alt=media\&token=5da8109b-2fb8-41d1-9459-190e538f8c91) red.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fu0gFIJ722dP8FkBMXs2J%2Ficon-compute-resource-status-green.png?alt=media\&token=7050e286-81fb-4c6e-8e9b-b2826e554242) green.

***

#### Memory Usage <a href="#mesh-node-memory-usage" id="mesh-node-memory-usage"></a>

**Trigger Condition**

When a Mesh Node's memory usage exceeds 70% of capacity.

**Clear Condition**

When the Mesh Node's memory usage drops below 70% of capacity.

**Severity**

Minor

**System Impact**

Delays in signaling and data processing at the Mesh Node.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Mesh Node** tab > Mesh Nodes list > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F3ESxDiBr0KD4kRrzSUBq%2Ficon-compute-resource-status-red.png?alt=media\&token=17c30002-f9d0-47ae-8396-5b3700a0ebef) red.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F3NgezXZyfwk7wXO1KlAH%2Ficon-compute-resource-status-red.png?alt=media\&token=4e3aa68d-d0df-4920-8ca2-929a3c8253a5) red.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.
* Onyx Portal > **Equipment** > **Mesh Node** tab > view Mesh Node's details > **Overview** tab > **Mesh Node Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fimtud18CpycnvdYwQSas%2Ficon-compute-resource-status-green.png?alt=media\&token=1765a66a-4ae6-4d93-9c19-cf42b671499d) green.

***

### Mesh Node Provisioning Server Alerts

#### Provisioning Server Connection Failure <a href="#mesh-node-provisioning-server-connection-failure" id="mesh-node-provisioning-server-connection-failure"></a>

**Trigger Condition**

When a Mesh Node has not successfully checked in with the Provisioning Server within the past 35 minutes.

**Clear Condition**

When the Mesh Node successfully checks in with the Provisioning Server.

**Severity**

Warning

**System Impact**

* A new Mesh Node will be unable to connect to the Onyx Portal or begin operations.
* For an existing Mesh Node:
  * If the keys have changed, it will be unable to get the updated keys from the Provisioning Server and will be unable to connect to the Onyx Portal.
  * If the keys have not changed, there will be no impact.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Online** icon:
  * If the Mesh Node's unique keys have changed, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FioI0gPMznykfImkccoOb%2Ficon-online-status-red.png?alt=media\&token=7317c6b6-2ee7-4cda-b02b-f2688b49430c) red.
  * If the Mesh Node's unique keys have not changed, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FlNIcVQFtsQAByMbPNKHm%2Ficon-online-status-green.png?alt=media\&token=72e44ad1-b16d-42e4-b480-1c2efc50a6e1) green.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

## Onyx Edge (Core) Alerts

### Onyx Edge (Core) C-Plane Alerts

#### Authentication Failure

**Trigger Condition**

When one or more UE authentication failures (excluding "unknown user") occur within the last five minutes.

**Clear Condition**

When no UE authentication failures (excluding "unknown user") occur for at least five consecutive minutes.

**Severity**

Warning

**System Impact**

UE will not be able to connect to the network.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Control Plane Transaction Status** tile: Shows Auth Request failures.

**Corrective Actions**

1. Typically, the cause is misconfiguration on the UE's SIM with its auth parameters (Onyx Portal > **Subscribers** > view subscriber's details > **AuthKey/Auth OPC**).
2. Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Affected UE attach as required and pass data traffic.

***

#### DHCP Server Failed

**Trigger Condition**

When the DHCP server, responsible for assigning IP addresses to UE, has been unavailable for five minutes.

**Clear Condition**

When the DHCP server becomes available.

**Severity**

Major

**System Impact**

* UE may not be able to obtain IP addresses for data connectivity.
* New data sessions will be impacted while existing sessions may continue to function.

**Indication**

Onyx Portal > **Dashboard** > common counter indications.

**Corrective Actions**

Check the DHCP server health in the network.

**Resolution Verification**

Onyx Portal > Alerts Counter Panel: Alert counts revert to pre-incident levels.

***

#### S1 Setup Failure

**Trigger Condition**

When there is an S1 Setup Request failure in the last five-minute window.

**Clear Condition**

When there are no S1 Setup Request failures in the last five-minute window.

**Severity**

Major

**System Impact**

AP will not be able to connect with the Onyx Edge.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Onyx Edge** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FXZC0y1BYRTqlKRPl1vlR%2Ficon-service-status-amber.png?alt=media\&token=4a9882de-1f18-4883-8cab-7ee3254a21ee) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FvLG2DAFqXSm0H4wlYxGr%2Ficon-service-status-red.png?alt=media\&token=d7d5b288-91f3-420a-9e69-761c70ee17e0) red, depending on how other APs are connected both in this and other Onyx Edge.
* Onyx Portal > **Dashboard** > **Access Point** tile > **Online** icon: May change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FCpCjyeXnCqLU5jE5AWLv%2Ficon-service-status-amber.png?alt=media\&token=d1ea1991-38bb-403b-b5e2-d93d728b9255) amber depending on whether the affected AP is a donor AP resulting in loss of connectivity to a Mesh AP.
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FarterMmLIZxS4sz808gc%2Ficon-service-status-amber.png?alt=media\&token=4c71f8b8-f5c8-4bb0-b0ee-c529e9ed2323) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fy7T5EaFjiYKHugIL4bJQ%2Ficon-service-status-red.png?alt=media\&token=afc64271-928f-42bc-b199-433e717c8a05) red depending on other AP statuses.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Online** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FIVYzlZSlo3WcN6akHE7g%2Ficon-online-status-amber.png?alt=media\&token=4fa12b0a-4380-4fa4-8fc6-c9f827ad7b96) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FyNJnjttRJUmBmQCJW24s%2Ficon-online-status-red.png?alt=media\&token=5c54ea53-efe8-4aca-b961-cf94d89e8c9c) red depending on if this is a donor AP going offline.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FXisV01YUqxxMjIfFIAJ2%2Ficon-service-status-amber.png?alt=media\&token=b35470e8-1ade-4288-b0f6-218b17508724) amber or grey depending on whether other Onyx Edge are offline.
* Onyx Portal > **Dashboard** > **Subscribers** tile: Total subscribers impacted by this Onyx Edge will change to unknown.
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: Reduces by the extent of traffic handled by the Onyx Edge.
* Onyx Portal > **Dashboard** > **Data Usage** tile: Reduces by the extent of traffic handled by the Onyx Edge.
* Onyx Portal > **Dashboard** > **Backhaul (SGi) Health** tile: Reduces by the extent of traffic handled by the Onyx Edge.

**Corrective Actions**

1. Check if any maintenance activity is currently in progress.
2. Check if any recent configuration changes have been made.
3. Check the AP's IPSec status (if used).
4. Verify that the TAC configuration between the Network and AP are the same (Onyx Portal > **Network** > **EPC** section > **TAC** parameter) and (Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > **Overview** tab > **RAN** tile > **TAC** parameter).

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

***

#### **SCTP Outbound Stalled**

**Trigger Condition**

SCTP inbound ordered chunks have been increasing over the last 15 minutes while SCTP outbound ordered chunks have remained flat.

**Clear Condition**

Outbound SCTP chunks resume normal flow, indicating the core is processing and responding to signaling messages again.

**Severity**

Major

**System Impact**

* The NGAP/NAS signaling layer on the Onyx Edge core is stalled — SCTP packets are arriving from the access network but the core is not sending any responses back.
* New UE procedures (attach, registration, handover, PDU session establishment) will fail because signaling responses are not being sent.
* Existing sessions may continue to pass user-plane traffic for a time, but any control-plane operation (e.g., service request, paging response, session modification) will time out.
* If the condition persists, APs may tear down their SCTP associations and trigger S1/NG Setup failures, leading to a broader service outage.

**Indications**

* Onyx Portal > **Dashboard** > common counter indications
* Onyx Portal > **Dashboard** > **Onyx Edge** tile > **Service Status** — May change to Red if the stall persists and signaling associations are lost
* Onyx Portal > **Dashboard** > **Subscribers** — Active subscriber count may begin to decline as UE procedures fail
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** — Control-plane stall may eventually impact data traffic as sessions cannot be established or modified

**Corrective Actions**

Contact GXC Support. There is no automatic recovery action for this condition.

**Resolution Verification**

From the Dashboard:

* The alerts count has returned to the condition prior to the incident.
* SCTP outbound chunk counters are incrementing normally.
* UE attach/registration procedures are succeeding again.

***

#### UE Attach Failure

**Trigger Condition**

When UE attach requests failed for reasons other than unknown user/IMEI validations in the last 15 minutes.

**Clear Condition**

When no UE attach requests fail for reasons other than unknown user/IMEI validations for at least 15 minutes.

**Severity**

Warning

**System Impact**

UE will be unable to attach to the network.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Control Plane Transaction Status** tile: Shows Attach Request failures.

**Corrective Actions**

1. Verify the subscriber/IMSI is configured in the Onyx Portal (Onyx Portal > **Subscribers**).
2. Verify the subscriber's service status is set to Active (Onyx Portal > **Subscribers** > view subscriber's details > **Subscriber Details** section > **Service** parameter).
3. Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Affected UE attach as required and pass data traffic.

***

#### UE IMSI-IMEI Validation Failure

**Trigger Condition**

When a UE attach failure occurs due to IMSI-IMEI validation failure in the last 15 minutes.

**Clear Condition**

When there are no UE attach failures due to IMSI-IMEI validation failures in the last 15 minutes.

**Severity**

Warning

**System Impact**

Any of the following:

* Genuine UE are not able to attach to the network due to incorrect configuration.
* Illegal attempt to use the network.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Control Plane Transaction Status** tile: Displays Attach Request failures.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Events & Alerts** tab: Displays IMEI-mismatch failures.

**Corrective Actions**

1. Check the Onyx Edge's Events (Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Events & Alerts** tab > **Events**). Filter with "*stream: mme*" and look for "*Attach Reject*" events. This will show the offending IMSI and IMEI.
2. If the IMEI is not acceptable against the IMSI, there is nothing further to be done in the Onyx Portal. The user can check who is trying to attach with the wrong device.
3. If the IMEI is configured incorrectly, correct it (Onyx Portal > **Subscribers** > edit subscriber's details > **Subscriber** tab > **IMEI Verification** > **IMEI** parameter).

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Affected UE, if genuine, attach as required and pass data traffic.

***

#### UE Session Failure

**Trigger Condition**

When there are UE PDU Session Creation failures in the past five minutes.

**Clear Condition**

When there are no UE PDU Session Creation failures for at least five minutes.

**Severity**

Warning

**System Impact**

UE will not be able to access data services through CBRS.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Control Plane Transaction Status** tile: Shows Create Session Request failures.

**Corrective Actions**

1. Check if the same APNs are configured in the subscriber and UE configurations (Onyx Portal > **Subscribers** > view subscriber's details > **Traffic Policy** section > **Active APNs**).
2. Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Affected UE attach as required and pass data traffic.

***

### Onyx Edge (Core) CU-Plane Alerts

#### HA Peer Down

**Trigger Condition**

When, in an Onyx Edge pool, the Primary ACS Onyx Edge detects a peer Onyx Edge is down. A peer Onyx Edge can be marked as "down" for the following reasons:

* S1 interface is down
* SGi interface is down
* Both S1 and SGi interfaces are down
* The peer Onyx Edge is powered down

The Primary ACS Onyx Edge detects a peer is down within 10 minutes of it going down.

**Clear Condition**

When the Primary ACS Onyx Edge detects that the peer Onyx Edge is operational.

**Severity**

Major

**System Impact**

When a peer Onyx Edge is reported as down, if the S1/SGi interface is also down, it will not be able to handle any UE traffic and is flagged as non-operational. A complete blackout will occur if all the Onyx Edge in a pool become non-operational.

**Indications**

Common Dashboard indications.

**Corrective Actions**

1. Check the Onyx Edge pool's status (Onyx Portal > **Equipment** > **Onyx Edge** tab > **Onyx Edge Pool** tile > view pool's details > **HA Connectivity Status** icon):
   * ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FQBHBnEOIonV6n1L1UpRC%2Ficon-interface-status-green.png?alt=media\&token=d6fce60f-e836-4299-a4b1-fd2d4ae4a386) green — One Onyx Edge is functioning as the Primary ACS, and the others as Standby ACS.
   * ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F3ujIEqcoYEtUFyCGwXUR%2Ficon-interface-status-amber.png?alt=media\&token=2cd08986-7738-4a85-a4b2-5ee93736f0f5) amber — One Onyx Edge is functioning as the Primary ACS, but one or more of the other Onyx Edge are non-functional.
   * ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FxLQLnX7NvrNF8KnYmctO%2Ficon-service-status-red.png?alt=media\&token=43d4d5a9-8fd0-4792-9ae2-8314a968f57e) red — All Onyx Edge are non-functional, or their status is not available.
   * &#x20;![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FmkURSMVfyvixm51SQVym%2Ficon-service-status-grey.png?alt=media\&token=1ff38e69-050c-4436-98e9-18c7a11f29ee) grey — All Onyx Edge are offline.
2. Check if all the Onyx Edge in the pool are online (Onyx Portal > **Equipment** > **Onyx Edge** tab > **Onyx Edge Pool** tile > view pool's details > **Associated Onyx Edge** tile > **Online** column > **Online** icon):
   * ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FOPqx90HIHOAK2mWzFW67%2Ficon-online-status-green.png?alt=media\&token=1610ad96-889b-488a-a89c-a97e92a22a92) green — The Onyx Edge has checked in with the Onyx Portal in the last five minutes.
   * ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FQKgsPs4abw9SX69P9NFH%2Ficon-online-status-red.png?alt=media\&token=de13ad48-a3d9-4d46-b0db-ee797bcdf35a) red — The Onyx Edge has not checked in with the Onyx Portal in the last five minutes.
3. If the S1 interface is down, check the S1 connection between the peer Onyx Edge and APs. In a healthy S1 interface, the Onyx Edge should be able to ping each AP (Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Actions** > **Command** > **Ping**). Verify physical connections, switch configuration, and VLAN assignments for both the Onyx Edge and APs.
4. If the SGi interface is down, check Internet connectivity. In a healthy SGi interface, the Onyx Edge should be able to ping the Internet (8.8.8.8) (Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Actions** > **Command** > **Ping**). Verify physical connections, switch configuration, and VLAN assignments.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

***

#### IPSec Service Down

**Trigger Condition**

IPSec endpoint in the Onyx Edge is down and is not listening for new IPSec connection requests from APs.

**Clear Condition**

IPSec service is ready and is listening for connection requests from the APs.

**Severity**

Critical

**System Impact**

The network is down as the Onyx Edge is not ready to receive AP connections.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Overview** tab > **Onyx Edge Details** tile > **IPSec Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F8BE9is61hyk8G0nGyhIt%2Ficon-ipsec-status-red.png?alt=media\&token=faaca8d6-42f5-4add-ae32-87335ad8bc3a) red.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.

***

#### IPSec Service Restarting Frequently

**Trigger Condition**

When the IPSec service on the Onyx Edge is restarting too frequently.

**Clear Condition**

When the IPSec service remains stable.

**Severity**

Major

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Overview** tab > **Onyx Edge Details** tile > **IPSec Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FHIoUrWZBS8tgt90PlVyL%2Ficon-ipsec-status-red.png?alt=media\&token=83c7a953-bfd5-40ea-83e5-62f250aee867) red or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FXMU6Oso1PDXfJMuFPmuX%2Ficon-ipsec-status-amber.png?alt=media\&token=31a478bb-b5f9-4df3-81ed-bd00c1c2bbf6) amber.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.

***

### Onyx Edge (Core) M-Plane Alerts

#### Offline <a href="#onyx-edge-core-offline" id="onyx-edge-core-offline"></a>

**Trigger Condition**

When an Onyx Edge remains offline for at least 10 minutes.

**Clear Condition**

When the Onyx Edge comes back online.

**Severity**

Critical

**System Impact**

Configurations, alerts, events, metrics, etc. between the Onyx Portal and the Onyx Edge will be out of sync.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Onyx Edge** tile > **Online** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FrsluxL74zLTf4VXZ41gY%2Ficon-online-status-amber.png?alt=media\&token=ac3e2e42-7e5f-4ff8-ae16-c2f70fd9ead3) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FDMYkeRtKJeWaspB5jttx%2Ficon-online-status-red.png?alt=media\&token=774300ba-70d3-44ec-b7cc-350883bdad4b) red, depending on whether other Onyx Edge are offline.
* Onyx Portal > **Dashboard** > **Onyx Edge** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F2taHWf85URpdtQ5k6WXy%2Ficon-service-status-amber.png?alt=media\&token=cfb3d3ee-b585-4fa0-a909-561148fc9ddf) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fizn57Ad9iK3xscRKvJSm%2Ficon-service-status-grey.png?alt=media\&token=239e5dd9-6ba1-44f7-bdf2-cab113524b54) grey, depending on whether other Onyx Edge are offline.
* Onyx Portal > **Dashboard** > **Access Point** tile > **Online** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FdgkyllbDd3KCwtFISz2T%2Ficon-online-status-amber.png?alt=media\&token=9f896acc-b2b7-47d8-99c2-365d65e0888f) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FrVlz7Llth1dgJP5a4FQe%2Ficon-online-status-grey.png?alt=media\&token=cfe001e0-7d02-4b57-9525-7bfd211c3465) grey, depending on whether other Onyx Edge are offline.
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F2f7WvGohnh6rBU99ctLv%2Ficon-service-status-amber.png?alt=media\&token=1d329960-59a1-4cee-8b65-2df2ea1640b1) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FIUd2JmBUnSYSMx5K1cji%2Ficon-service-status-grey.png?alt=media\&token=a13fc857-15b5-41b9-90a0-e34c3e12b995) grey, depending on whether other Onyx Edge are offline.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Online** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FvIwS24nKBKC5Szhtstmr%2Ficon-online-status-amber.png?alt=media\&token=f89af9c3-4aff-48a6-9680-7f192b7e43a9) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F2mR1pTXaQ9Eo8NcVNUU8%2Ficon-online-status-grey.png?alt=media\&token=8cfb45b5-b189-410f-b745-ee33ba8be026) grey, depending on whether other Onyx Edge are offline.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Service Status** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F7ETvGheTGF4EKVlJzfXo%2Ficon-service-status-amber.png?alt=media\&token=ec541987-0bcf-4ec7-ab0e-5bbf7e4a27a5) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FRPz0iBrgBT3hQlMrk7ji%2Ficon-service-status-grey.png?alt=media\&token=02a86c0b-651a-486f-9550-4b957f7a92b2) grey, depending on whether other Onyx Edge are offline.
* Onyx Portal > **Dashboard** > **Subscribers** tile: Total subscribers impacted by this Onyx Edge will change to Unknown.
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: Reduces by the extent of traffic handled by the Onyx Edge.
* Onyx Portal > **Dashboard** > **Data Usage** tile: Reduces by the extent of traffic handled by the Onyx Edge.
* Onyx Portal > **Dashboard** > **Backhaul (SGi) Health** tile: Reduces by the extent of traffic handled by the Onyx Edge.

**Corrective Actions**

1. Check the Onyx Edge's Internet connectivity on the SGi interface. This cannot be done from the Onyx Portal when the Onyx Edge is offline. If another system, such as a remote laptop in the local network is accessible, ping the Onyx Edge to check if there is a local network issue, and then log on to the Onyx Edge to verify Internet connectivity.
2. At the Onyx Edge's location, verify power and connectivity to the equipment.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

***

### Onyx Edge (Core) MCU-Plane Alerts

#### Service Down

**Trigger Condition**

When any of the following is true:

* All APs are offline for five minutes.
* All APs are S1-disconnected for five minutes.
* Any critical services have restarted more than three times in the last five minutes.
* Any critical services are down for five minutes.

**Clear Condition**

When the trigger condition is resolved.

**Severity**

Critical

**System Impact**

The Onyx Edge may be offline or may not be functioning as expected, potentially causing UE to be unable to obtain service.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Onyx Edge** tile > **Service Status** icon: Depending on how other APs are connected both in this and other Onyx Edge, and how they are affected, will change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F4mIshEGGZWXHm9wtHGUW%2Ficon-service-status-amber.png?alt=media\&token=952cabbe-323f-456b-be21-97136d402169) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FbupKZnGTUABPY7NKMk7x%2Ficon-service-status-red.png?alt=media\&token=87ec7f4f-84a5-4309-bfda-eeca5f6c1c6a) red.
* Onyx Portal > **Dashboard** > **Access Point** tile > **Online** icon: Depending on whether APs are affected, may change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FaLe87X2fY6x92emTLxO8%2Ficon-online-status-amber.png?alt=media\&token=a4e60b64-ca9b-41f9-a4de-28f9c09c30b1) amber.
* Onyx Portal > **Dashboard** > **Access Point** tile > **Service Status** icon: Depending on whether APs are affected, will change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F5OnIXZbJJvVHakZvNuZS%2Ficon-service-status-amber.png?alt=media\&token=101f65c5-5c3c-407f-8ede-b582a777df38) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fku8IcyfN6z2Y7YPreT6G%2Ficon-service-status-red.png?alt=media\&token=28d23c87-75a7-40c3-9e0d-6771114ea295) red.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Online** icon: Depending on whether Mesh Nodes are affected, will change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FaGSrHwnBASSdXvFleWRD%2Ficon-online-status-amber.png?alt=media\&token=62cf9da4-af65-427f-83a7-655e3bd8127e) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FfJJpcf5bAiBRLp0pADPg%2Ficon-online-status-red.png?alt=media\&token=38ba2886-1895-4418-bef5-106966602e7e) red.
* Onyx Portal > **Dashboard** > **Mesh Node** tile > **Service Status** icon: Depending on whether Mesh Nodes are offline, will change to ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FBOKiVfn2u4FGamk6eYcR%2Ficon-service-status-amber.png?alt=media\&token=2c63f55a-56cb-45ab-88dd-c5be3beb7623) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fr3eGEqU6WD41A9Iya9EZ%2Ficon-service-status-grey.png?alt=media\&token=4c06ce41-8ef4-40bc-ab34-8f67ca10bbcc) grey.
* Onyx Portal > **Dashboard** > **Subscribers** tile: Total subscribers impacted by this Onyx Edge may change to Unregistered.
* Onyx Portal > **Dashboard** > **Downlink/Uplink Traffic** tile: Reduces by the extent of traffic handled by the Onyx Edge.
* Onyx Portal > **Dashboard** > **Data Usage** tile: Reduces by the extent of traffic handled by the Onyx Edge.
* Onyx Portal > **Dashboard** > **Backhaul (SGi) Health** tile: Reduces by the extent of traffic handled by the Onyx Edge.

**Corrective Actions**

1. Check if the Onyx Edge is properly provisioned.
2. If an AP is offline and the Access Point M-Plane [Offline](#access-point-offline) alert has triggered:
   1. Check for network connectivity issues between the AP and the Onyx Edge.
   2. Check if the AP is associated with the correct Onyx Edge.
3. If an AP is S1-disconnected and the Access Point C-Plane [S1 Disconnected](#s1-disconnected) alert has triggered:
   1. Check the AP's IPSec status.
   2. Check for any Access Point faults and alarm events (**Events & Alerts** tab).
4. If the above alerts have not been triggered, restart the Onyx Edge's services (Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Actions** > **Restart Services**).

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

**Related Alerts**

* Access Point M-Plane [Offline](#access-point-offline) Alert
* Access Point C-Plane [S1 Disconnected](#s1-disconnected) Alert
* Onyx Edge (Core) M-Plane [Offline](#onyx-edge-core-offline) Alert

***

### Onyx Edge (Core) Platform Alerts

#### CPU Usage <a href="#onyx-edge-core-cpu-usage" id="onyx-edge-core-cpu-usage"></a>

**Trigger Condition**

When an Onyx Edge's CPU usage exceeds 70% of capacity in the last five minutes.

**Clear Condition**

When the Onyx Edge's CPU usage drops below 70% of capacity in the last five minutes.

**Severity**

Minor

**System Impact**

* Possible delays in signaling and data processing at the Onyx Edge.
* If CPU usage continues to rise, unexpected behaviors may be seen.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Onyx Edge** list > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FeiahheeTwlSQ1SMV4FZH%2Ficon-compute-resource-status-red.png?alt=media\&token=151ef5b5-d6b7-4c07-a8bc-9ad344b75397) red.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Overview** tab > **Onyx Edge Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FnyNM5yYImxHiDJ9ytHEi%2Ficon-compute-resource-status-red.png?alt=media\&token=ccee9bdb-dd75-4269-ba06-f0c2d72327ec) red.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Onyx Edge** list > **Compute Resource** icon: Depending on usage, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fk9NDLnm2orGFvQRnv3yR%2Ficon-compute-resource-status-amber.png?alt=media\&token=1581c993-415c-44fb-b79f-e80e779b8ba8) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FfjRW83XJTYEe4YF15kRG%2Ficon-compute-resource-status-green.png?alt=media\&token=95704eb4-6b75-4319-a462-e2ab551103ad) green.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Overview** tab > **Onyx Edge Details** tile > **Compute Resource** icon: Depending on usage, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fb79iplwW4OWs0urSoX2q%2Ficon-compute-resource-status-amber.png?alt=media\&token=7b87a840-a924-4d2b-86e7-8b7c68f4964c) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Ft0gvtrglgu49fRSAHVVU%2Ficon-compute-resource-status-green.png?alt=media\&token=687faf0a-ed81-49c7-96da-a0e8ef28219e) green.

***

#### Disk Usage <a href="#onyx-edge-core-disk-usage" id="onyx-edge-core-disk-usage"></a>

**Trigger Condition**

When an Onyx Edge's disk usage exceeds 70% of capacity in the last five minutes.

**Clear Condition**

When the Onyx Edge's disk usage drops below 70% of capacity in the last five minutes.

**Severity**

Major

**System Impact**

* If the disk usage continues to rise, it will eventually cause downtime on the Onyx Edge.
* The Onyx Edge might stop collecting critical logs that will be necessary to debug issues.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Onyx Edge** list > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F6fTNa5zPBmqubCsw4Wqx%2Ficon-compute-resource-status-red.png?alt=media\&token=e795531f-afc7-427f-8ce0-7566d7d85200) red.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Overview** tab > **Onyx Edge Details** tile > **Compute Resource** icon: ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FG5CmDsdjDq5iQOsNHEP4%2Ficon-compute-resource-status-red.png?alt=media\&token=692f0433-8540-4702-8a75-a83db3cf91fa) red.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Onyx Edge** list > **Compute Resource** icon: Depending on usage, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FAQ6gVdq0fjrJpC2XPai4%2Ficon-compute-resource-status-amber.png?alt=media\&token=3a3a3a75-3e36-43be-acf9-5e2f46095d0d) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fr4OUPoYadON0xiPu68lj%2Ficon-compute-resource-status-green.png?alt=media\&token=462247c6-40de-489f-840f-4f4f83acdb54) green.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Overview** tab > **Onyx Edge Details** tile > **Compute Resource** icon: Depending on usage, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F4ISemHbOH2iGf3Sn95Xk%2Ficon-compute-resource-status-amber.png?alt=media\&token=79c3e6ee-d9d4-4515-a6eb-b7780e62a20e) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2Fho1xfND7uNSmcGHLRxY7%2Ficon-compute-resource-status-green.png?alt=media\&token=acaa612d-c699-4063-9f54-f05797571469) green.

***

#### Memory Usage <a href="#onyx-edge-core-memory-usage" id="onyx-edge-core-memory-usage"></a>

**Trigger Condition**

When an Onyx Edge's memory usage exceeds 70% of capacity.

**Clear Condition**

When the Onyx Edge's memory usage drops below 70% of capacity.

**Severity**

Minor

**System Impact**

If memory usage continues to rise, it could lead to downtime on the Onyx Edge.

**Indications**

Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).

**Corrective Actions**

If a significant portion of memory is being consumed by a few processes, try restarting the corresponding systemd services to see if memory usage recovers.

1. To sort processes by memory usage, run the following command:

   ```
   top -o RES
   ```
2. To find the corresponding systemd service, run the following command:

   ```
   systemctl status <PID>
   ```
3. To restart a service, run the following command:

   ```
   systemctl restart <service name>
   ```

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > **Onyx Edge** list > **Compute Resource** icon: Depending on usage, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F5HK4ZgVuWUMDKNTLfWtT%2Ficon-compute-resource-status-amber.png?alt=media\&token=3330998b-bb6e-40d8-a9ac-42a4dbccfc8f) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FCuvIFYfk5bngZTiKuSzz%2Ficon-compute-resource-status-green.png?alt=media\&token=a54f6ea5-c90c-4b00-93fd-9d512813b463) green.
* Onyx Portal > **Equipment** > **Onyx Edge** tab > view Onyx Edge's details > **Overview** tab > **Onyx Edge Details** tile > **Compute Resource** icon: Depending on usage, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F6eFrOMt2JcNKpXKioYpV%2Ficon-compute-resource-status-amber.png?alt=media\&token=90b9459f-d859-485e-bcb1-76f3bc628ea0) amber or ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2F3IrVgiuNC0UnJT3kihDl%2Ficon-compute-resource-status-green.png?alt=media\&token=8c173fb8-cc4b-4c10-a2d8-df352d723849) green.

***

### Onyx Edge (Core) Provisioning Server Alerts

#### Provisioning Server Connection Failure <a href="#onyx-edge-core-provisioning-server-connection-failure" id="onyx-edge-core-provisioning-server-connection-failure"></a>

**Trigger Condition**

When an Onyx Edge has not successfully checked in with the Provisioning Server within the past 35 minutes.

**Clear Condition**

When the Onyx Edge successfully checks in with the Provisioning Server.

**Severity**

Warning

**System Impact**

* A new Onyx Edge will be unable to connect to the Onyx Portal or begin operations.
* For an existing Onyx Edge:
  * If the keys have changed, it will be unable to get the updated keys from the Provisioning Server and will be unable to connect to the Onyx Portal.
  * If the keys have not changed, there will be no impact.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* Onyx Portal > **Dashboard** > **Onyx Edge** tile > **Online** icon:
  * If the Onyx Edge's unique keys have changed, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FSFfPWBQHv45X4ZC1gXWt%2Ficon-online-status-red.png?alt=media\&token=124f363c-77f7-4b5e-a365-1aef752e7b1e) red.
  * If the Onyx Edge's unique keys have not changed, ![](https://4071075005-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FZc9hpHmiTCrh1sn4mWXn%2Fuploads%2FTOwchH1Zx9ewSGrKHlXe%2Ficon-online-status-green.png?alt=media\&token=b35dfd52-c22d-4df5-b8b6-ee92ac96bd22) green.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Alert counts revert to their pre-incident levels.
* Onyx Portal > **Dashboard**: All tiles revert to their pre-incident states.

***

## Orchestrator Alerts

### Orchestrator Domain Proxy Alerts

#### SAS Connection Failure

**Trigger Condition**

Any of the following:

* Could not connect to SAS for at least three times in the last 15 minutes.
* At least one HTTP error from SAS in the last 15 minutes.

**Clear Condition**

When there are no SAS connection errors and no HTTP errors for at least 15 minutes.

**Severity**

Critical

**System Impact**

This could potentially mean the entire network is down, at least those APs that are dependent on SAS for their grants.

**Indications**

* Onyx Portal > [Alerts Counter Panel Indication](#alerts-counter-panel-indication).
* In case of constant connection errors, the transmit timer could expire and result in the cells going down. See [Cell Transmission Down Indications](#cell-transmission-down-indications).
* Onyx Portal > **Equipment** > **Access Point** tab > view AP's details > SAS status: The SAS state may not progress if not authorized, and the Authorized state may eventually revert back to Granted state on timer expiry.

**Corrective Actions**

Contact GXC Technical Support.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Onyx Portal > **Dashboard**: All tiles return to their pre-incident states.

**Additional Information**

* There are two kinds of errors:

  * **HTTP errors** — Should not occur under normal conditions; typically indicate an SAS-side issue or fundamental breakage in DP.
  * **Transport-level errors** — May occur due to connectivity issues (unable to reach SAS) or security issues (server-side or client-side authentication failures).

  The type of error can be determined from the alert labels.
* SAS Connection Failure alerting generates a single consolidated alert at the network level instead of individual alerts per AP.

***

## Semi-Custom Alerts

Semi-custom alert rules are predefined by GXC but allow limited configuration changes, such as threshold values. These rules retain their core logic as defined by GXC but can be adjusted to better fit a specific network's operational needs.

### Semi-Custom Mesh Node MCU-Plane Alerts

#### **Mesh Node&#x20;*****\<serial number>*****&#x20;RSRP is Lower than Configured Threshold for 5 Minutes**

**Trigger Condition**

When a Mesh Node's RSRP remains below its configured RSRP threshold continuously for five minutes.

**Clear Condition**

When the Mesh Node's RSRP rises to or above its configured threshold and remains stable for five minutes.

**Severity**

Major

**System Impact**

Indicates weak signal coverage or degraded connectivity between the Mesh Node and serving cell, which can lead to reduced throughput or service drops.

**Indications**

* RSRP metric on the Mesh Node details page is below the configured limit (default: -100 dBm).
* End devices may experience reduced link stability.

**Corrective Actions**

* Check the Mesh Node's placement and antenna alignment.
* Investigate physical obstructions or environmental changes affecting signal propagation.
* Validate serving cell configuration and parameters.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Confirm the Mesh Node's RSRP is above threshold in real-time monitoring.

**Related Alerts**

* [Mesh Node \<serial number> RSRQ is Lower than Configured Threshold for 5 Minutes](#mesh-node-rsrq-lower-than-threshold)
* [Mesh Node \<serial number> SINR is Lower than Configured Threshold for 5 Minutes](#mesh-node-sinr-lower-than-threshold)

***

#### **Mesh Node&#x20;*****\<serial number>*****&#x20;RSRQ is Lower than Configured Threshold for 5 Minutes**

**Trigger Condition**

When a Mesh Node's RSRQ remains below its configured RSRQ threshold continuously for five minutes.

**Clear Condition**

When the Mesh Node's RSRQ rises above the configured threshold and remains stable for five minutes.

**Severity**

Major

**System Impact**

Indicates poor signal quality and/or high interference, potentially causing reduced spectral efficiency and degraded user experience.

**Indications**

* RSRQ metric on the Mesh Node's details page is below the configured limit (default: -15 dB).
* Throughput and session stability may be impacted even if RSRP is acceptable.

**Corrective Actions**

1. Identify and mitigate interference sources.
2. Check cell neighbor relations and handover parameters.
3. Verify antenna configuration and sector assignments.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Confirm the Mesh Node's RSRQ is above threshold in real-time monitoring.

**Related Alerts**

* [Mesh Node \<serial number> RSRP is Lower than Configured Threshold for 5 Minutes](#mesh-node-rsrp-lower-than-threshold)
* [Mesh Node \<serial number> SINR is Lower than Configured Threshold for 5 Minutes](#mesh-node-sinr-lower-than-threshold)

***

#### **Mesh Node&#x20;*****\<serial number>*****&#x20;SINR is Lower than Configured Threshold for 5 Minutes**

**Trigger Condition**

When a Mesh Node's SINR remains below the configured SINR threshold continuously for five minutes.

**Clear Condition**

When the Mesh Node's SINR rises above the configured threshold and remains stable for five minutes.

**Severity**

Major

**System Impact**

Indicates low signal-to-interference-plus-noise ratio, which can lead to reduced modulation schemes, slower data rates, and increased error rates.

**Indications**

* SINR metric on the Mesh Node's details page is below the configured limit (default: 13 dB).
* Possible increase in retransmissions and packet errors.

**Corrective Actions**

1. Investigate interference sources in the serving frequency band.
2. Optimize antenna orientation and tilt.
3. Adjust frequency plan or channel allocation if persistent.

**Resolution Verification**

* Onyx Portal > Alerts Counter Panel: Returns to pre-incident alert counts.
* Confirm the Mesh Node's SINR is above threshold in real-time monitoring.

**Related Alerts**

* [Mesh Node \<serial number> RSRP is Lower than Configured Threshold for 5 Minutes](#mesh-node-rsrp-lower-than-threshold)
* [Mesh Node \<serial number> RSRQ is Lower than Configured Threshold for 5 Minutes](#mesh-node-rsrq-lower-than-threshold)

## Related Documentation

* *4G Onyx Portal Operations Guide*
* *GXC Glossary*

## Contact GXC

To get in touch with GXC, please visit <https://gxc.io/contact-us/>.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.gxc.io/docs/network-configuration-and-operations-guides/4g-onyx-portal-4.x-alerts-reference.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.