Error "event exceeds maximum size: event at offset 1 is larger than xx Bytes" on Panther AWS Config History log source
Issue
When trying to ingest my AWS Config History Logs, I get the error "event exceeds maximum size: event at offset 1 is larger than 15728640 Bytes"
on my Log Source.
Resolution
To resolve this issue set a prefix to exclude the Config History context, which will reduce the ingested file size.
Based on the default format of the AWS Config History Log:
1111111111_Config_us-west-2_ConfigHistory_AWS::EC2::VPC_111111T11111Z_111111111111Z_1.json.gz
An exclusion pattern such as *_Config_*ConfigHistory*
could be applied.
Cause
Due to their content, the AWS Config History Logs are usually large files. This issue occurs because the AWS Config history Logs' uncompressed size exceeds Panther's maximum allowed limit size.