Process Search Negation Guide

How do negation queries work?

By definition, negation queries ask the search to provide any results that do NOT match a certain criteria. So, if a customer were to submit the query NOT field_name:"blah", any process that did not have field_name:"blah" would technically be a valid result. With this logic, any process record that doesn’t have field_name populated at all would technically be a valid result.

Example: Take the following process records

process_a: {
    "field_1": "value"
    "field_2": "value"
}

process_b: {
    "field_1": "value"
}

If we were to submit the query field_1:"value" AND NOT field_2:"value", then process_record_b would be a valid result.

Why do negation queries sometimes provide incorrect results?

As explained above, if a process does not contain a field whose value is being negated, it is by default a valid search result.

In Enterpise EDR, the sensor will send an event bundle every 1 to 2mins depending on the OS or if the event bundle reaches 1MB in size. The event bundle contains events for all processes on the device which the backend synthesizes into process records grouped by process_guid. Long-running processes have multiple (and sometimes many) records that are captured at different times to make sure that we get all information through the process’ life. Those records will be combined and aggregated to form what you see in the console as a process. However, process records may have fields absent from them that are present in other process records due to the sensor not being fully initialized after a sensor restart. As such a negated search will match a process record with a missing field causing the process to unexpectedly appear in the results.

Example: Consider the following two process records for the same process

process_record_a_1: {
    "process_cmdline": "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"
    "process_name": "chrome.exe"
}


process_record_a_2: {
    "process_name": "chrome.exe"
}

As shown here, the process_record_a_1 does not have a process_cmdline. Sometimes this can happen (often due to a machine rebooting, a fresh sensor installation or sensor update), where our EEDR sensors do not have enough information to attribute a specific process_cmdline to the process record. What this would mean is that if you were to run the following query on these two process records:

NOT process_cmdline:"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"

You would get the following result:

process_record_a_2: {
    "process_name": "chrome.exe"
}

Even though process_record_a_1 has a process_cmdline that is excluded from the search, process_record_b does meet all the criteria of the search. If you were to dig deeper into this process using others of our API’s that do not contain this negation, you would learn that the process does have process_cmdline:"process_cmdline" which could cause confusion, because it was surfaced as a search result where that was not the case.

This problem only affects some of our schema fields, particularly those that are metadata fields, and should never change over the course of a process’ life but for reasons explained above can sometimes be reported as empty by the sensor. Though this is rare, it does happen, and can cause this problem. The following fields can have this issue, though it is a very small percentage of the data that would show signs of this problem.

Which fields are susceptible to this problem?

This is only a problem for our process searches (not enriched events), and it is known to be an issue for the following fields:

device_group_id
device_internal_ip
device_policy
device_sensor_version
parent_cmdline
parent_cmdline_length
parent_publisher_state
parent_name
parent_pid
process_cmdline
process_cmdline_length
process_company_name
process_file_description
process_hash
process_issuer
process_name
process_original_filename
process_product_name
process_product_version
process_publisher_state
process_publisher
process_service_name
process_username

What workarounds are available for this?

Right now, the best workaround is to add a clause to the query to ensure that only process records with the negated field present are included in the search results. See below for a modification to the query referenced above.

Problematic Query:

NOT process_cmdline:"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"

Successful Query:

process_cmdline:* AND NOT process_cmdline:"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"

Last modified on March 9, 2022