Does Panther ingest updates to Azure Monitor files in Blob Storage?

QUESTION

Azure Monitor creates a file in Blob Storage once an hour, and appends new events to that file as they occur. If Panther's ingestion is triggered by file creation, does Panther still ingest any events appended after the file is created?

ANSWER

Yes, through the process is a little different that you'd expect.

Panther ingests Azure Monitor logs via the following process:

1. Azure Monitor creates a new log file for the current hour.
2. Panther recieves an OBJECT CREATE notification from Azure for the new file.
3. Panther sets an internal timer to check on this object at the end of the current hour.
4. Azure Monitor continues appending events to the log file in Blob Storage.
5. At the end of the hour, Panther retrieves the file and ingests the contents.

Since Panther waits until the end of every hour to parse Azure Monitor files, we ensure that we don't miss any events. However, this can lead to increased latency for Monitor logs. Maximum latencies of ~1 hour are expected.