Queries are running slowly when using TO_OBJECT in Data Explorer

Last updated: September 3, 2024

Issue

When trying to run a query in the Data Explorer, the query runs slowly and takes 5-10 minutes or longer. My query includes Snowflake's TO_OBJECT feature, as shown below:

SELECT distinct(TO_OBJECT(event):event:event_data:event_kind) FROM panther_logs.public.custom_data_source
WHERE p_occurs_since('2 weeks')

Resolution

To resolve this issue:

  1. Try omitting the TO_OBJECT piece of the query. Try to find another way to find the data you're looking for.

  2. If TO_OBJECT is the only way to find the desired information in your data, reduce the total size of the data by using LIMIT or several, separate queries using p_occurs_between, e.g. p_occurs_between(current_date - 1, current_timestamp) and then p_occurs_between(current_date - 1, current_date - 2) and so on. 

  3. You can check our relevant article Why might Data Explorer in Panther crash or freeze when running a query?.

  4. If these solutions still don't accelerate your queries, reach out to Panther support for additional assistance.

Cause

TO_OBJECT is a computationally expensive operation because it makes a copy of all processed data before generating its results. When querying a lot of data, this can cause extremely high query times.