Do alert destinations have a built-in retry mechanism in Panther? If so, how does it behave?
If the initial attempt to deliver an alert fails, Panther will automatically attempt to re-deliver it. After breaching a certain threshold of alert delivery failures, a system health alert will be generated and sent to any alert destinations configured to receive
System Error alerts. This is described in the section Alert Delivery Failure from our documentation. Afterward, you'll have to go to that particular
System Error and check the
Non-Delivered Alerts tab to see what alerts failed to deliver.
Alert deliveries are batched and processed in a queue. Any delivery failures are thrown back onto the queue and we will retry processing them every 15+ minutes, up to 10 times for a max of 3 days. There’s not exactly a back-off, but if the queue has other items to be processed, it may take longer than 15 minutes to retry again.
If you'd like to find more details on how to manually retry an alert, you can check our article How can I retry an alert delivery if it failed?.