NQL Overview and Syntax

The Netography Query Language Explained

Refer to the NQL Quick Reference Guide for all the NQL syntax in one table.

Overview

The Netography Query Language (NQL) is the basis for accomplishing many tasks in Fusion.

  • Searching for flows, DNS, events, audits, or block records
  • Filtering statistics and aggregations
  • Defining custom Detection Models to create an event

Becoming familiar with NQL will help you take full advantage of the Fusion platform.

📘

Comparing NQL to Other Query Languages

NQL is designed specifically for network traffic and security analytics, with features optimized for querying this type of data. Unlike SQL, which is a general-purpose query language for relational databases, NQL is tailored to operate on high-throughput network data with a focus on real-time analysis. Similar to Splunk's SPL (Search Processing Language) and Elasticsearch's Query DSL, NQL provides specialized operators and pattern matching capabilities, making it well-suited for dynamic network environments. However, NQL's syntax and rules are distinct, reflecting its purpose-built nature for network security operations.

Using NQL in Fusion

In the Fusion Portal, NQL can be used In the Global Filter (the bar at the top of the Portal), on the Search pages, in Detection Models, or to define widgets inside custom dashboards. It can be used in the API as the value for the search parameter if it exists.

Applying NQL

Results on each page are filtered by the Global Filter, the combination of the date/time range you select and the NQL filter you enter. NQL can also be used on the Search page (in the Investigate section of the left-hand menu bar).

📘

The circle with the inside it in the Global Filter changes from gray to blue when you need to apply a new value

Blue means you changed a value but have not yet applied it. Apply the filter by clicking the return arrow ( ) icon, or pressing Enter in the NQL text box.

Using NQL Presets

If you click the floppy disk ( ), it brings up a list of NQL queries divided into these sections:

  • Recent Queries (Flow only), My Presets, Company Presets, System Presets

Search for a preset

The empty text box that appears above the preset list after clicking the floppy disk is a search for presets - start typing in that box to filter the list of presets to find the one you are looking for quickly.

Save NQL to My Presets

Save the NQL from the global filter to My Presets by clicking the floppy disk , selecting the + Save Current NQL row, and then giving it a name (or you can leave it Untitled).

Duplicate, Edit, and Delete Presets

Hover over a preset, and 3 icons will appear on the right-hand side of the preset row. These icons allow you to edit, duplicate, or delete a preset. Only the icons you have permission to perform the actions of will appear (eg you will not see the delete icon on a System Preset).

Search using NQL Intersections

The Search page in the Investigate section of the left-hand menu enables you to perform advanced searches using NQL Intersections. An NQL Intersection allows for the logical combination of multiple independent NQL queries, focusing on the commonality between their results based on specified fields. This approach differs fundamentally from writing a single NQL query that combines conditions using logical operators like AND.

Intersections are a powerful capability for:

Complex Filtering: When you need to filter data across multiple dimensions that don’t necessarily fit into a single query string

Correlation of Data: When you want to correlate different aspects of data, such as matching IPs across different conditions (e.g., ports, protocols) without enforcing all conditions simultaneously.

NQL Syntax

NQL Examples

Use the system presets by clicking to see example NQL.

DescriptionNQL
IP Reputation Matchessrciprep.count > 0 OR dstiprep.count > 0
Only Privileged Portsdstport < 1024
Not Broadcast IPsdstip != 255.255.255.0/24
TCP Ports Scantcpflags.syn == true and tcpflags.ack == true and srcport > 1024

For additional examples, see NQL Examples, or watch the Video walkthroughs NQL video.

NQL Basics

Using the Netography Query Language is like writing a programming conditional inside an if ( ) statement. The statement should be constructed logically, from left to right, comparing fields to values.

Rules and Limitations

Spacing and Parentheses

  • Operators must be surrounded by spaces, e.g. srcip == 10.0.0.1 is valid, while srcip==10.0.0.1 is not valid.
  • Whitespace is not optional in NQL. For example, input==1 is invalid, but input == 1 is valid since the NQL parser will not parse the first example if the whitespace is missing.
  • NQL accepts nested parentheses. For example, ((this) || (that)) && other is a valid NQL statement. However, this can be considered hard to read, so it is suggested to reduce the use of parentheses when possible, e.g. (this || that) && other
  • Logic must be unambiguous. e.g. A && B || C will fail. Use parentheses () to prevent ambiguity

Comparisons

  • Only integer fields can use numerical comparisons: < <= > >=

CIDR Notation

  • IP fields can be searched with CIDR notation if desired. For example, 10.0.0.0/24 will match 10.0.0.1

📘

CIDR (Classless Inter-Domain Routing)

CIDR is a method for allocating IP addresses and routing Internet Protocol packets. CIDR uses a suffix (e.g., /24) to specify the exact number of bits that represent the network portion of the address. For example, 192.168.0.0/24 defines a network with addresses ranging from 192.168.0.0 to 192.168.0.255. CIDR is commonly used in routing, firewall rules, and network segmentation to precisely define IP address ranges.

Operators

What is an operator?

In the context of NQL, operators are used to define how fields should be compared or matched against values, allowing users to filter, search, and analyze data according to specific criteria.

Boolean Operators

BooleanDescriptionExample
&& ANDlogical ANDthis && that
ORlogical ORthis OR that
!NOT must precede expressions in parenthesis!(srcip == 10.0.0.1)

Comparison Operators

ComparisonDescription
==equals
!=not equals
<=less than or equals to
<=less than
>=greater than or equals to
>greater tha

For additional examples, see NQL Examples, or watch the Video walkthroughs NQL video.

Pattern Matching (Wildcards, RegEx, Fuzzy) in NQL

In NQL, pattern matching is performed using regular expressions, wildcards, and fuzzy matching. The operators =~ and !~ specify these matches and their negative counterparts.

ℹ️

NQL fields supporting pattern matching

Pattern matching is only supported for the fields listed below.

CategoryFields
Flowdstiprep.categories srciprep.categories tags
DNSanswers.rdata query.domain query.host query.name query.publicsuffix
Eventsipinfo.iprep.categories summary tags
Auditdescription

Wildcards

Wildcards use special characters to match patterns of text.

Wildcards: Operator Syntax and Meaning

OperatorSyntaxMeaningExample Field
=~*atMatches zero or more characters.query.name =~ *at
!~*atNegative match for zero or more characters.query.name !~ *at
=~?atMatches any single character before "at".query.name =~ ?at
!~?atNegative match for any single character before "at".query.name !~ ?at

Note: Avoid beginning patterns with * or ? as this can lead to performance issues due to increased iterations.

Regular Expressions (regex)

Regular expressions (regex) allow for flexible and complex search patterns. In NQL, you can use regex to match patterns with precise rules.

Note: Avoid beginning patterns with * or ? as this can lead to performance issues due to increased iterations.

Regex: Examples

query.domain =~ net_  
query.domain =~ netog.i?  
query.domain =~ net~  
query.domain =~ /[a-z]_/  
query.domain =~ /[a-z]\_/ && query.name == neto

Regex: Operator Syntax and Meaning

OperatorSyntaxMeaningExample Field
=~/.*at/Matches zero or more characters.query.name =~ /net.*\.com/
!~/.*at/Negative match for zero or more characters.query.name !~ /net.*\.com/

Regex: Text Boundary Anchors

Boundary anchors specify positions in the text. The start and end characters are implicitly applied to each search and cannot be modified:

RegexpMeaningDescriptionExample
^Start of a line or stringMatches the start of a string^cat matches catdog
$End of a line or stringMatches the end of a stringcat$ matches dogcat

Regex: Choice and Grouping

Choice and grouping operators allow you to match one or more alternatives or group expressions:

RegexpMeaningDescriptionExample
xyx followed by yMatches "xy"abc\ matches "abc".
x OR yx or yMatches "x" or "y"ye` matches "axe" or "aye"
abc(def)?GroupingMatches "abc" or "abcdef"abc(def)? matches abc and abcdef but not abcd

Regex: Repetition (Greedy and Non-Greedy)

Repetition operators match a specified number of occurrences:

RegexpMeaningDescription
x*Zero or more occurrences of xMatches zero or more of "x". Example: a* matches "", "a", "aa".
x+One or more occurrences of xMatches one or more of "x". Example: a+ matches "a", "aa".
x?Zero or one occurrence of xMatches zero or one of "x". Example: a? matches "" or "a".
x{n,m}Between n and m occurrences of xMatches between n and m of "x". Example: a{2,4} matches "aa", "aaa", "aaaa".
x{n,}n or more occurrences of xMatches n or more of "x". Example: a{2,} matches "aa", "aaa".
x{n}Exactly n occurrences of xMatches exactly n of "x". Example: a{3} matches "aaa".

Regex: Character Classes

Character classes match specific sets or ranges of characters:

RegexpMeaningDescription
.Matches any single characterMatches any character except a newline. Example: c.t matches "cat" and "cot".
[abc]Matches any single character in the setMatches one of "a", "b", or "c". Example: [aeiou] matches any vowel.
[^abc]Matches any single character not in the setMatches any character except "a", "b", or "c". Example: [^aeiou] matches consonants.
[a-z]Matches any single character in the rangeMatches any character between "a" and "z". Example: [a-z] matches any lowercase letter.

Regex: Special Characters

Special characters perform specific functions within regex patterns:

RegexpMeaningDescription
\Escape characterEscapes special characters to be treated as literals. Example: \. matches a literal period.

Regex: Reserved Characters

The following characters are reserved as operators and need to be escaped: . ? + * | { } [ ] ( ) " \

Fuzzy Matching

Fuzzy matching accounts for variations in spelling by calculating the Levenshtein distance between terms.

🤔

What is the Levenshtein Distance?

The Levenshtein distance is a metric for measuring the difference between two strings. It calculates the minimum number of single-character edits (insertions, deletions, or substitutions) required to change one string into another. For example, the Levenshtein distance between "kitten" and "sitting" is 3, because it requires three edits: replacing 'k' with 's', replacing 'e' with 'i', and adding 'g'. This metric is useful for fuzzy matching, where you want to find matches that are close, but not exact.

Fuzzy Matching: Operator Syntax and Meaning

OperatorSyntaxMeaningExample Field
=~cat~Matches terms with an automatically calculated distance.query.name =~ cat~
!~cat~Negative fuzzy match with automatic distance.query.name !~ cat~
=~cat~2Matches terms with a maximum distance of 2 changes.query.name =~ cat~2
!~cat~2Negative match with a maximum distance of 2 changes.query.name !~ cat~2