Skip to main content
Panther Knowledge Base

Why do I get an error when trying to ingest zst compressed files in Panther?

  1. Last updated
  2. Save as PDF

Issue

When trying to ingest zst compressed data from my S3 log source into Panther I see an error message such as:

failed to read JSON array

or

error found in #1 byte of ...|(\ufffd/\ufffd\u0000|...

Resolution

Double check to ensure your zst compressed files were not compressed using a dictionary. If they were compressed with a dictionary, please modify the system that sends these files to S3 such that they are compressed without using a dictionary.

Cause

If you are ingesting compressed data, Panther will try to decompress the files prior to classifying the events in your files. If Panther doesn't detect that your files are compressed, it will continue to try to read them as is. For this reason, if you have compressed files that Panther fails to decompress, you might see errors that look like Panther is attempting to read a binary file.

Specifically for zst compression, this could happen if you compress your files using a dictionary. Panther does not support decompressing files that were compressed using a dictionary, and therefore you might see errors like the above. See here for more details on Panther's supported compression formats.

If you did not compress your input files with a dictionary, please reach out to Panther's support team and share how these files were compressed.