Built-in retry mechanism for alert destinations in Panther

QUESTION

Do alert destinations have a built-in retry mechanism in Panther? If so, how does it behave?

ANSWER

If the initial attempt to deliver an alert fails, Panther will automatically attempt to re-deliver it. After breaching a certain threshold of alert delivery failures, a system health alert will be generated and sent to any alert destinations configured to receive System Error alerts. This is described in the section Alert Delivery Failure from our documentation. Afterward, you'll have to go to that particular System Error and check the Non-Delivered Alerts tab to see what alerts failed to deliver.

Alert deliveries are batched and processed in a queue. Any delivery failures are thrown back onto the queue and we will retry processing them every 15+ minutes, up to 10 times for a max of 3 days. Alert retry logic is not user-configurable.  In addition, there’s not exactly a back-off, but if the queue has other items to be processed, it may take longer than 15 minutes to retry again.

If you'd like to find more details on how to manually retry an alert, you can check our article How can I retry an alert delivery if it failed?.