When a log file in an S3 bucket is overwritten, does Panther process the new version?
Panther treats an overwritten file in S3 as a new log entry, regardless of whether S3 bucket versioning is enabled.
This means that Panther will capture both the original version and the new version of the overwritten file, treating them as separate logs. This behavior ensures that no log entries are missed, but it may result in duplicate logs if the overwritten file contains the same content as the original.
Managing Log Duplicates
To effectively manage log duplicates resulting from overwritten files, it is recommended to implement logic at the log source level that filters out duplicates before sending the logs to Panther. This logic can involve comparing log entries based on unique identifiers, such as timestamps or log IDs.