Skip to main content
Panther Knowledge Base

How can I investigate log classification errors in Panther?

  1. Last updated
  2. Save as PDF

QUESTION

If you see a System Health Notification alert for a log source, such as the following:

Source has turned Unhealthy
Log(s) cannot be properly classified into attached schemas

You may want to see a sample of alerts to investigate the logs or schemas involved. How can you find out what this alert is for? 

ANSWER

  1. Open the Panther Console and go to Integrations > Log Sources.
  2. Find an alert from the log source in question, by scrolling or typing the name of the source in the box at the top that says "Filter Log Sources by text."
  3. Once you've found an alert, look at the end of its error message and click the "View Alerts" link there. This can reveal additional information about the state of the log source.

Alternatively, in the Data Explorer, use this query to check for new classification errors:

select * from panther_monitor.public.classification_failures
 where p_occurs_since('48 hours') -- adjust as needed
 and p_source_label = 'Your log source name'
 order by p_parse_time desc
 limit 10

 If your Panther instance doesn't use Snowflake, omit .public from the database name(s).

Classification Errors Explained
  • error: This will help identify which fields are causing the log event and schema to not classify (normalize) with each other.
  • id: This is the schema that failed to classify (normalize) with the log event.
  • payload: This is the log event that failed to classify (normalize) with the schema.

Screenshot 2023-09-11 at 2.02.43 PM.png

In this example, the error is saying that the schema Custom.Intercom.Webhook is expecting Field_1_Id as a required field in every log event. In the payload, there is no valid Id field which is causing the classification error. Removing the required: true tag from the schema, as shown in the screen shot below, should resolve this classification error.
Screenshot 2023-09-11 at 2.29.47 PM.png

Troubleshooting for Panther Supported Logs 
  • Confirm that the payload is similar enough to the schema (id).
    • Errors may occur due to the third party's latest updates which includes adding or deprecating fields from their latest log versions.
  • Once confirmed, please provide the fields above error, id and payload for Panther Support to also verify and then make changes to the Panther Supported Log Schema.
    • Any release notes or updated documentation that states there have been changes to the logs from the third party will help Panther Support make the necessary changes!
Troubleshooting for Custom Logs
  • Please confirm that the payload is similar enough to the schema (id). 
    • Errors may occur due to the third party's latest updates which includes adding or deprecating fields from their latest log versions and/or your S3 source may be ingesting metadata payloads to which no schema was created for.
  • Once confirmed, users check the error against yourpayload and schema (id) to resolve the classification error.
  • If you still need help troubleshooting, please provide the fields above error, payload and entire schema for Panther Support to take a look. Here's an example of the entire schema: 
    • fields:
  - name: type
    type: string
    required: true
    description: "Value is 'notification_event'"
  - name: id
    type: string
    required: true
    description: "The Intercom defined id representing the notification."
  - name: self
    type: string
    description: "The Intercom defined URL for the subscription."
  - name: created_at
    type: timestamp
    timeFormats:
        - unix
    isEventTime: true
    required: true
    description: "The timestamp the notification was created."
  - name: topic
    type: string
    required: true
    description: "Corresponds to a topic."
  - name: data
    type: object
    required: true
    fields:
      - name: item
        type: json
        description: "The data associated with the notification, which will have a 'type' field."