Error "event exceeds maximum size: event at offset 1 is larger than xx Bytes" on Panther CloudTrail log source

Issue

I get the following error from my CloudTrail log source in Panther:

"error": "event exceeds maximum size: event at offset 1 is larger than X Bytes

Resolution

To resolve this issue:

Run the following query to check if the CloudTrail log source is currently ingesting logs:

SELECT
*
FROM
panther_logs.public.aws_cloudtrail
WHERE
p_occurs_between('2023-02-08 00:00:00Z','2023-02-10 23:59:00Z') AND p_source_label = '<log-source-name>'
ORDER by p_parse_time DESC
LIMIT 10

Make sure to customize the query based on your own timeframe for the unclassified event.
- where p_occurs_between('<start-date-of-unclassified-event>','<current-date>') e.g 'YYYY-MM-DD HH:MM:SSZ' / '2023-02-16 23:59:00Z'

2. If logs after the unclassified event are ingesting, you can then backfill that unclassified event using our S3SNS tool. See📄 Can I backfill the logs of a new log source into Panther?

Note: The unhealthy banner may still appear for 2-4 days after the unclassified event occured, but the query above will confirm if your log source is currently ingesting as expected! See📄 After fixing an unhealthy log source, why do I still get an error banner in the Panther Console?

Cause

This issue occurs when the CloudTrail log source schema is disabled and re-enabled. There are processing rules that help handle larger log files, but the temporary disable may cause the rules to not take effect for a short bit of time.