Error: "Query timeout after scanning x B from x S3 objects (Total Listed: x)" when trying to infer a custom schema from the S3 data receiver in Panther
Issue
When trying to infer a custom schema from my S3 log source in Panther the following error occurs: Query timeout after scanning 0 B from 0 S3 objects (Total Listed: 171,000)
Resolution
To resolve this issue, try shortening the time period to help reduce the query results:
Cause
This happens when Panther is trying to explore S3 raw data with rare prefixes or rare substring matches and the query times out. The error details iclude the total listed S3 objects, the scanned S3 objects, as well as the total volume scanned (uncompressed).