Skip to main content
Panther Knowledge Base

How come no data is coming in for a new S3 log source in Panther?

Issue

When trying to onboard a new custom S3 log source the following issue occurs:

No data is being pulled from the log source.

Resolution

To resolve this issue:

  1. Is there new data coming into the S3 bucket?
    Add new data to the bucket and see if "Last Received Data" updates for your new log source on the Configure > Log Sources page
  2. Does the data coming into the bucket match the log type's schema?
    Check for log classification errors by clicking on your new custom S3 log source and clicking "Health" on the upper left-hand side of your Panther Console, or by using the following Data Explorer Query:
    1. select * from panther_monitor.public.classification_failures
      where p_occurs_since('48 hours') -- adjust as needed
      and p_source_label = 'your-new-s3-log-source'
      order by p_parse_time desc
      limit 10
  3. Was the source set up via the CloudFormation wizard or manually?
    Check to make sure all IAM roles for the source have permissions and are correctly hooked into Panther.
    If the log source was configured manually, check the CloudFormation or Terraform template to ensure all roles have the correct permissions to write notifications.
     

Cause

This issue occurs when

  1. No new data is coming into the log source's S3 bucket
    Panther only pulls data that comes into the bucket after the source has been onboarded. Old data will not be pulled in.
  2. The new data coming into the S3 bucket doesn't match the schemas for any of the log types associated with that log source
    This can happen when logs do not include required fields or are not formatted the way specified in the schema and cannot be parsed. (JSON vs CSV, etc.)
  3. When set up manually and editing the CF or Terraform template, values can get deleted or roles may have conflicting names with existing AWS IAM role names. It's good to double-check when debugging that the template is complete.