Why can’t I find logs in the data lake after ingesting data using a custom schema in Panther?

Issue

When trying to create a custom schema and ingest data through my log source, I see the data go in, but I can't find it in the data lake.

Resolution

To resolve this issue:

• Check the fields in your schema to verify they aren't using special characters outside of Snowflake's Identifier Requirements.
• If you have a field with square brackets, please try transforming your data to remove the square brackets from these fields. If they are in a sub-field of another field, consider pulling in the higher level field as a string, or if the value of this field is not absolutely necessary, consider leaving the field out of your schema.

Cause

Panther tries to modify fields when they don't conform to Snowflake's Identifier Requirements, but if there is a gap, then the data won't save to the Data Lake.