How to determine impact of the Alert Generation and Delivery Failures Incident

Last updated: January 18, 2025

How can I determine if I was impacted during the Alert Generation and Delivery Failures Incident?

The incident impacted the generation and delivery of alerts for detections with an events threshold value greater than one. If you suspect this incident has affected your alerts, follow the steps below to assess the impact and ensure that your systems were functioning as expected.

Step 1: Identify the rule_id of the detections with threshold > 1

CI/CD Users:

  • Please use the following command to identify the affected detections:

    grep -r "Threshold:" | grep -v "Threshold: 1$" | grep "yml"

Console Users:

  • Please go to your Panther console and click Detections -> Upload -> Download all entities

  • In your terminal, please run the following command:

    grep -r "Threshold:" | grep -v "Threshold: 1$" | grep "yml"

Step 2: Query for the impacted alerts

Once you have the list of rule_id, you can get all the potentially impacted alerts by running the following query in the Panther Data Explorer:

select * from panther_signals.public.correlation_signals 
where p_ruled_id IN (<list>) 
and p_occurs_between('2025-01-13', current_timestamp)

Please note that the alerts returned by this query are not impacted if:

  • An alert has an alert id that exists in your Panther console

  • There weren't enough results to meet the alert threshold associated with a given alert id