Issue

I just created a new Lookup Table using a Custom Schema. Every time I try to ingest a CSV file using this Lookup Table, I see errors like this:

lookup update failed for TestLookUps: upload failed for lookup ad2827e3-378c-40ef-8e97-52304906f26f into : failed to parse line 1: {“payload”:“id,display_name,first_name,last_name...}”,“errors”{“id”:“Custom.Custom.testingApp”,“error”:“parse failed: readObjectStart: expect { or n, but found i, error found in #1 byte of ...|id,display_|..., bigger context ...|id,display_name,first_name,last_name,|...“}],“p_log_type”:“Classification.Failures”,“p_row_id”:“fe4e37704159a6dba48fcda11201",“p_event_time”:“2022-01-01T10:00:00”,“p_parse_time”:“2022-01-01T10:00:01”}

Resolution

To resolve this issue, you can use pantherlog to infer a schema. 

  1. If needed, install pantherlog.

  2. Run pantherlog infer sample_logs.csv > schema.yml to generate a new schema and save it in schema.yml.

  3. Upload the new schema to a similarly-named Lookup Table and ingest the CSV data again.

If you already have a schema and you're seeing an error like the one above, you can spot-check a sample of data and see if it matches the lookup table's schema. If a field name in the data doesn't match the name in the schema, you can adjust the schema to match and then try again to ingest the data.

If these options don't work, contact the Panther support team for further assistance.

Cause

This issue can occur for a number of reasons, but one common situation is when there is something in the schema that doesn't line up with the data, or vice versa.