Why does Data Replay keep failing on alert simulation in Panther?

Last updated: April 4, 2025

Issue

What could be causing my Data Replay to fail on alert simulation in Panther?

Resolution

  1. Please ensure your use case is within the following limitations:

    • Time range is configurable but must be within the last 15 days.

    • Time range must be older than 24 hours.

    • The maximum supported replay size is 20GB.

    • The replay must complete in under an hour.

    • Access to the DynamoDB cache is blocked to prevent polluting production data.

    • Network calls from within a rule are blocked.

    • Enrichment is not supported.

  2. If your scenario is free from these limitations:

    1. Set the timeframe to the same day, for one hour.

    2. If the Data Replay completes successfully, take note of how many alerts were generated.

    3. If the number of alerts is relatively small (10 - 15), you may consider increasing the timeframe.

    4. If the number of alerts is quite high (close to 1000), please only adjust the time frame such that it does not exceed 1000 generated alerts.

Since there is no way to know in advance exactly how many alerts will be generated during Data Replay, you may need to try different time windows to find the right balance. We recognize this as a current limitation of Data Replay and we are working towards improving this for the future.

Cause

Data Replay may fail if there are more than 1000 alerts generated within one hour. This is due to the Alert Limiter which automatically kicks in, even during Data Replay.