Issue

When trying to onboard Amazon EKS Audit logs using a Panther managed log source, the following classification error occurs:

Failed to classify event as 'Amazon.EKS.Audit': 'apiVersion' is a required field but it is missing
‘auditID’ is a required field but it is missing
‘kind’ is a required field but it is missing
‘level’ is a required field but it is missing
‘requestReceivedTimestamp’ is a required field but it is missing
‘sourceIPs’ is a required field but it is missing
‘stage’ is a required field but it is missing
‘stageTimestamp’ is a required field but it is missing
‘user’ is a required field but it is missing

This causes the log source to become unhealthy, and reprocessing events only provides a temporary fix before reverting back to the unhealthy state.

Resolution

To resolve this issue:

1. Navigate to your Kinesis Data Firehose delivery stream configuration.

2. Enable the following two settings:

   • Turn on decompression (Decompress source records from Amazon CloudWatch Logs)

   • Turn on message extraction (Extract message fields only from log events)

3. Leave Turn on data transformation unchecked, as this requires a Lambda function.

4. If you cannot modify these settings on your existing stream, recreate the Firehose delivery stream with both Turn on decompression and Turn on message extraction enabled from the start.

   

For detailed setup instructions, refer to Panther's AWS EKS documentation.

Cause

This issue occurs when EKS audit logs are sent to Panther in CloudWatch Logs subscription filter format, which includes a wrapper with messageType, logGroup, and logEvents fields. The actual EKS audit events are nested inside the logEvents[].message fields and must be extracted before Panther can parse them as individual Kubernetes audit events.

Without enabling decompression and message extraction in Kinesis Data Firehose, Panther receives the wrapped format instead of the individual audit events, causing the classification to fail due to missing required fields.