Troubleshooting log ingestion issues in Panther
QUESTION
How can I troubleshoot log ingestion issues with a log source?
ANSWER
Follow the tips below while troubleshooting log ingestion issues in Panther:
- Confirm whether Panther is receiving any data.
- You can check the log source. It should say “Last data received X minutes ago” if you are expecting it to receive logs every few minutes.
- If it's a Data Transport source, such as an S3 bucket, check to verify that the source has received data since setting up the ingestion with Panther. See this KB article for more information: How come no data is coming in for a new S3 log source in Panther?
- If there is data received:
- The issue could be an error from Panther reading the source (e.g., the log is encrypted and Panther expects it to not be, or the log file is not in the right format). To check for this, you can run a query in Data Explorer to search for the log source name in the
panther_monitor.data_audit
table. - The issue could be a misclassification error where Panther successfully reads it but fails to classify. To check for this, you can run a query in Data Explorer to search for the log source name in the
panther_monitor.classification_failures
table.
- The issue could be an error from Panther reading the source (e.g., the log is encrypted and Panther expects it to not be, or the log file is not in the right format). To check for this, you can run a query in Data Explorer to search for the log source name in the
If there is no data received yet then follow the suggestions below. Note that this could include seeing "Last data received X hours ago" in a situation where you are expecting data every few minutes.
- Check whether bucket notifications are set up correctly. If there are no bucket notifications, then Panther will display “No data received yet” next to the log source. See this KB article for more information: Panther Log Source error: "Bucket notifications are not properly configured"
- If you created an IAM role manually for an S3 log source and you are not receiving data, see this article: No data flow or errors after creating IAM role manually for S3 source in Panther
Please also see these related articles: