✋

Writing your own detection model? We are here to help.

Chat with Netography's Detection Engineers in the #fusion-detections channel in Netography's Discord community, or send your question to Support. See: ✋ Need More Help?.

Overview

Detection Models generate events in Fusion when they are triggered based on the traffic ingested and the detection model configuration. This page helps you understand how detection models work so you can customize existing detection models and create your own.

To see short descriptions of every Detection Model configuration field, see: Detection Model Quick Reference Guide.

To add a new detection model and view basic configuration field information, see Adding a Detection Model

General Configuration

• Name: Unique name for Detection Model.
• Description: Brief description of the Detection Model's purpose and function.
• Categories: One or more Detection Categories. See Detection Categories.
• Traffic Type: Flow or DNS - determines what type of traffic this Detection Model is for.
• Enable Detection Model: Determines if the Detection Model is active.
• Enable Policies and Integrations: If Disabled, response policies and response integrations will not be executed when this Detection Model generates an event.

Traffic Match

The traffic match section defines the traffic ingested into Fusion to which this detection model will be applied. Multiple rows can be added using the + button to specify unique NQL Expressions for different Flow or DNS types.

• NQL Search:
  • Search Against: The Flow or DNS traffic types to apply the NQL Expression to. all will be used for all Flow or DNS types except those specified in a separate row.
  • NQL Expression: The NQL to use to filter the traffic included in the Detection Model. C
• Discards: Excludes traffic that would otherwise match the NQL Expression defined in NQL Search.

Thresholds

Thresholds operate on the traffic that matches in the Traffic Match section.

Track By Fields

Track By Fields specify how data is aggregated in a detection model. For example, if srcip is specified as a Track By field, each distinct source IP aggregates data, which is then evaluated against the thresholds to determine if an event should be triggered.

Multiple Track By fields can be specified in a detection model. The more Track By fields specified, the higher the cardinality (the number of unique values). With more unique values, the data aggregated into each unique value is reduced.

🪣

Cardinality: Understanding Data Uniqueness

Cardinality refers to the uniqueness of elements within a dataset. In the context of data aggregation, it represents the number of distinct values a field can have. A field with high cardinality contains many unique values (e.g., a list of IP addresses where each entry is different), while a field with low cardinality has a limited set of unique values (e.g., a list of country codes with only a few repeated options). Higher cardinality typically means more precise or granular data, while lower cardinality results in more generalized groupings.

Examples

Source IP (Low Cardinality): If you track only by source IP, you may see fewer unique groupings because multiple connections can originate from the same IP. For instance, if there are 10 devices, each with a unique source IP, you'd have 10 unique entries.

Source IP and Destination IP (Higher Cardinality): If you track by both source IP and destination IP, the number of unique groupings increases. For example, a single source IP connecting to 5 different destination IPs would now result in 5 unique entries for that source IP alone, raising the overall uniqueness.

Source IP, Source Port, Destination IP, Destination Port (Highest Cardinality): Adding more fields, such as source port and destination port, creates even more unique groupings. A single source IP connecting to multiple destination IPs on different ports results in a much larger set of unique combinations. For example, the same source IP connecting to 5 destination IPs using 3 different source ports and 2 destination ports could result in up to 30 unique combinations (5 x 3 x 2).

Auto Thresholds and multiple Track By fields

Auto Thresholds generate baselines for each Track By field, and the more Track By fields added, the more values that need to be tracked for Auto Thresholding (ie higher cardinality). If the cardinality is too high, you will exceed the resource limits in Fusion for auto thresholds, and your detection model will fail to generate baselines.

Thresholds: Severity, Threshold, and Rollup Period

Threshold Severity

Severity is one of Low, Medium, or High and indicates the magnitude of an event produced by a Detection Model. A Detection Model can specify multiple Thresholds, each with a different Severity and Threshold configuration.

Threshold

Threshold is the core of a detection model and specifies what a detection does. A threshold is one or more operations over data, including (but not limited to) average, maximum, minimum, divide, or count. Unless auto thresholds is enabled, a detection model specifies a predetermined value in the threshold. When the threshold value is satisfied within a specified Rollup Period, an event is generated.

Rollup Period

Rollup Period specifies a sliding window (in seconds) over which the operation in a threshold configuration is applied to each aggregation (as specified in the Track By Fields). A threshold operates over all the traffic records ingested into Fusion meeting the Traffic Match criteria within the time window specified in the rollup period. A detection model effectively is always looking backward in time over the Rollup Period.

The default Rollup Period of 300 seconds looks backward from the most recent aggregation and the four previous records (one aggregation is recorded for every 60 seconds of traffic) (300 seconds / 60 = 5).

Putting this all together, a Track By Fields configuration is a bucket that collects the data matching the NQL Search filter in the Traffic Match section. The Threshold specifies a test or tests to perform over the data in each aggregation (such as sum(bits) > 50). The Rollup Period specifies the time frame (sliding window back in time) over which the operation is performed. For a threshold of sum(bits) > 50, the Rollup Period adds all the bits values in each aggregation and if the sum is greater than 50 over the previous 300 seconds, an event is generated.

🤖 Auto Thresholding

Auto Thresholding automates the adjustment of event-triggering thresholds in detection models using machine learning to adapt to baseline traffic patterns. For more details, see: Auto Thresholding.