Do Panther Lookup Tables enrichments happen for all events or only rule matches?

QUESTION

Do Lookup Table enrichments happen for each incoming event or do they only happen for events which have matched a rule and triggered an alert? I may want to use custom lookup tables for filtering events which might be actionable.

ANSWER

Enrichment happens before log events are sent to the detections engine, so every incoming log event with a match in your lookup table will be enriched. If a match is found, a p_enrichment field is appended to the event and accessed within a detection using a deep_get.This is the structure of p_enrichment fields:

'p_enrichment': {
      <name of lookup table>: {
        <key in log that matched>: <matching row looked up>,
		<key in log that matched>: <matching row looked up>
	  },
}

You can see this in practice in the screenshot below. For example, if you needed to grab the title string nested within p_enrichment in a detection, you would use deep_get(event, "p_enrichment", "1Password Translation", "item_uuid", "title") . 

The data in p_enrichment  isn't stored in panther_logs because panther_logs tables are populated pre-enrichment, but if a rule has a match, and that event has enrichment data, then p_enrichment will be present in panther_rule_matches  (due to rule matches being generated after enrichment & detections engine). Lookup tables are essentially tables in panther_lookups , and these tables are created based on the schema you provide just like with any other log source. So to use enrichment data in a query, you can write SQL joins against like this query template in our docs.

with logs as 
(select * from my_logs), 
lookup as (select * from my_lookup_table) 
select logs.fieldA, lookup.fieldB 
from logs join lookup on logs.selector_field = lookup.key_field

To request the ability to use enrichment data without it being dependent on rule matches, please reach out to the Panther Support team for a feature request.