QUESTION

How can I improve the performance of queries when retrieving alert data from Panther's data lake, and what are the best practices for querying alert information?

ANSWER

Alert data is loaded into Snowflake tables via Snowpipe with minimal latency. There are several ways to optimize query performance when retrieving alert data in Panther:

Use the signal_alerts table

Instead of querying the all_rule_matches view, use the panther_signals.public.signal_alerts table. This table:

• Is a Snowflake hybrid table optimized for row-based storage.

• Uses indexes for efficient queries.

• Has a primary key of (CREATIONTIME, ALERTID) for faster lookups.

• Performs better for queries with alert ID constraints.

Optimize query performance

When querying Panther tables other than signal_alerts, follow these best practices:

• Consider including a constraint on p_event_time to improve query speed.

• For the all_rule_matches view, include time-based filters to reduce full table scans.

If you frequently query over large time periods and need to scan large amounts of data, you may consider increasing warehouse size, which typically speeds up query performance.