Alerts failed to fire after changing a Panther detection's deduplication period
Issue
While testing a detection, I changed the deduplication period from a long period to a shorter one. Then I sent test data that should have triggered some alerts, but none fired when I expected them. Then, later on, alerts started firing successfully, in accordance with the new configuration.
Resolution
After reducing the deduplication period for a detection from an original period to a new period, the system may need to wait as long as the original period before the new period takes effect. For example, if you changed the deduplication period from 24 hours to 1 hour, and you made this change at 9:00AM on Monday, in the worst case scenario you may need to wait until 9:00AM on Tuesday to see alerts. Once this time has passed, the deduplication will continue as described in our documentation.
Cause
This issue occurs because of a Panther bug.
The deduplication (dedup) period and threshold apply to p_alert_creation_time and p_alert_update_time. Values for these fields can be found by searching for Rule Matches on the Panther Console's Search page, or by searching the panther_rule_matches table in the Data Explorer. The dedup period gets re-checked every time p_alert_update_time changes (roughly every time a new rule match arrives). However, the dedup threshold is set at p_alert_creation_time, and is never updated until the original alert expires (which occurs at (whatever the dedup period was at p_alert_creation_time) after p_alert_creation_time ).
Our engineering team plans to address this bug. If you encounter this issue, please contact Panther support.