QUESTION

Why do I see alerts in the PANTHER_VIEWS.PUBLIC.ALL_RULE_MATCHES table that are not visible in the Panther UI or retrievable through the API?

Is there a way to filter this table to identify indicators/events that met the rule threshold and those events leading up to it?

ANSWER

This can happen for detections using a deduplication threshold. Panther writes the events before reaching the threshold in case we do reach it. If the alert_id never reaches the threshold, we don't "trigger" it, but we need to have the rule matches stored before we "trigger" the alert. Behind the scenes, this process involves two tables: a "hot" table for the deduplication, and a "final" table that you see in the UI and from which we send to alert destinations. Alerts that never reach the threshold aren’t present in the "final" alerts table but still generate rule matches.

To determine the events leading up to the threshold, you can check all the alerts/rule matches for that specific ID that occurred in the 60 minutes prior to the alert limiter kicking in. You can use a query like this:

select * 
from panther_views.public.all_rule_matches  
where p_rule_id = RULEID  
and p_occurs_between(1_HR_BEFORE_LIMITER_ALERT, TIME_OF_LIMITER_ALERT)

This will retrieve all the events that contributed to the rule. However, keep in mind that the limiter triggers on individual alerts, not rule matches. If you want to view the individual alerts themselves, you can get the alert IDs with:

select distinct p_alert_id 
from panther_views.public.all_rule_matches  
where p_rule_id = RULEID  
and p_occurs_between(1_HR_BEFORE_LIMITER_ALERT, TIME_OF_LIMITER_ALERT)

You should be able to search for the alerts in the UI as well, but if not, you can use the query above and then view the alerts by manually entering the ID into the URL bar.