How do I resolve "upload failed for lookup" and "parse failed" errors when uploading a CSV to Panther?
Issue
I just created a new Lookup Table using a Custom Schema. Every time I try to ingest a CSV file using this Lookup Table, I see errors like this:
lookup update failed for TestLookUps: upload failed for lookup ad2827e3-378c-40ef-8e97-52304906f26f into : failed to parse line 1: {“payload”:“id,display_name,first_name,last_name...}”,“errors”{“id”:“Custom.Custom.testingApp”,“error”:“parse failed: readObjectStart: expect { or n, but found i, error found in #1 byte of ...|id,display_|..., bigger context ...|id,display_name,first_name,last_name,|...“}],“p_log_type”:“Classification.Failures”,“p_row_id”:“fe4e37704159a6dba48fcda11201",“p_event_time”:“2022-01-01T10:00:00”,“p_parse_time”:“2022-01-01T10:00:01”}
Resolution
To resolve this issue, you can use pantherlog to infer a schema.
- If needed, install pantherlog.
- Run
pantherlog infer sample_logs.jsonl > schema.yml
to generate a new schema and save it in schema.yml. - Upload the new schema to a similarly-named Lookup Table and ingest the CSV data again.
If this doesn't work, contact the Panther support team for further assistance.
Cause
This issue can occur for a number of reasons, but one common situation is when there is something in the schema that doesn't line up with the data, or vice versa.