Configure NetoFlow

You can run NetoFlow with the default configuration, which should be sufficient for most deployments.

Modifying the configuration

NetoFlow uses a layered configuration that will read configuration from the /etc/netoflow.conf file, if it exists, and from environment variables next (environment variables take precedence).

Follow the steps in the install instructions for your chosen deployment method to edit and pass the configuration settings:

Install NetoFlow (container) or Install NetoFlow (Linux package)

Default configuration settings are in/etc/netoflow.conf

The default configuration settings are contained in /etc/netoflow.conf.example. This file is installed with the Linux software packages or can be downloaded for container deployments:

curl -O https://neto-downloads.s3.amazonaws.com/netoflow/netoflow.conf.example

Configuration settings

SectionKeyDescriptionDefault Value
generaludpbufferSize of the UDP buffer1048576
generalstatsintervalInterval in seconds to report stats60
generalflushintervalInterval in seconds to flush data5
generalflushcountNumber of records to flush2000
generalretriesNumber of retries for failed operations5
generalretryintervalInterval in seconds between retries5
generaldumprecordsOutput to stdout instead of Netographyfalse
generaltemplatedbPath to the template database/var/tmp/netoflow.db
generaldownsampleDownsample rate for data10
generalapihostAPI host for data ingestionhttps://api-ingest.netography.com
generaldevicecreationurlURL for device creation/api/v1/device
generalaudithostHost for audit logshttps://api.netography.com
generalauditurlURL for audit logs/api/v1/audit
flowlistenAddress and port to listen for flow data0.0.0.0:2055
flowworkersNumber of worker threads4
flowqueueSize of the queue for incoming data100000
sysloglevelLogging levelinfo
syslogserverSyslog server address and portlocalhost:514
apienabledEnable or disable the APItrue
apiportPort for the API8080
sflowpayloadInclude payload in sFlow datafalse
teesenabledEnable or disable teesfalse
teesupdateintervalInterval in seconds to update tees300
teestargetsList of target addresses for tees
endpointnetosecretAPI key encoded as netosecret
proxyenabledEnable or disable proxyfalse
proxyurlProxy URL`
filterdefaultDefault action for the filterallow
filterlogdenyLog denied packetsfalse
flowfilteractionAction for the flow filter
flowfilterdstnetsDestination networks for the flow filter
deviceflowsrcipSource IP address for flows
devicedownsampleDownsample rate for device data

[tees] - Teeing flows

The NetoFlow Connector can tee flow packets to an array of targets.  That is, it can deliver the packets to multiple destinations while preserving the source IP of the device that sent the flows to NetoFlow. This is useful if you want to send flow to an array of tools within the infrastructure.  Teeing in NetoFlow accomplishes this while preserving this visibility to the other tools that are receiving flows.  Refer to the [tees] section of the configuration example below for an example.

[general] - Flow delivery performance tuning

In the general section of the configuration, there are several options to modify the behavior flow delivery to the cloud.  By default, flows will be delivered to the cloud when either of 2 conditions is met:

  • a 5-second flush interval timer
  • 2,000 flow records. 

One might wish to modify this behavior depending on how much traffic the NetoFlow Connector handles.  Consult Netography Support for help tuning these values if needed.

[[flowfilter]] - Flow Filters

You can define flow filters to exclude flow data from the traffic delivered to Fusion.  Flow filters behave like firewall rules where you have a default behavior and then set an action in individual rules. 

  • Flow filters can contain a combination of the following array fields: srcnets, dstnets, srcports, dstports, protocols, srcasns, dstasns
  • The fields within a flow filter are processed as an and. If any array member matches within a field, that field is a match.  All fields must match for a flow filter to match. 
  • Dropped records are logged at the debug level. 
  • You can have many flow filters as they are currently not capped.  This is primarily CPU-bound, but the impact should be minimal. 
  • Refer to the flowfilter example in the configuration example below. Note the filter default setting, which defaults to "allow," as the normal case is to send Netography all flows.

📘

A flow filter must contain an action and at least 1 set of criteria to match on.

[general]/[device] downsample Flow Sampling with NetoFlow

NetoFlow can perform client-side downsampling. If you have sampling being performed at your network devices and want to pass through the already sampled flow, this additional downsampling is not needed.

You can configure downsampling in 2 ways:

  1. Apply a global downsample to any device NetoFlow sees flow from. This is set in the [general] section with the downsample key (or NETO__GENERAL__DOWNSAMPLE environment variable).
  2. Apply a device-specific downsample by adding a [device] block to the configuration and specify the flowsrcip and the downsampleto apply to it.  See the example configuration below.
  • Specific device downsample configurations will override the global downsample.  The global downsample will only be used if there is no device-specific configuration. 
  • Valid downsample values are integers between 2 and 10,000.

**Note:  If you are downsampling on a device that doesn't send the sample rate (i.e. the sample rate appears to be 0), you must also configure the sample rate when adding that device in the Fusion Portal or you will receive incorrect metrics for that device. See Ingest NetFlow/sFlow via the NetoFlow Connector.

Configuring NetoFlow to view flow rates before delivering flow to Fusion

It is possible to configure NetoFlow to acquire the flows per second sent by a device without delivering the flow records to Fusion. This is useful for understanding your internal flow volume. The easiest way to achieve this is to set a default filter of "deny" like such:

[filter]
default = "deny"
logdeny = false

Then, use the NetoFlow statistics API to read the flows per second. This will provide per-device statistics. See: Reading statistics from NetoFlow API.

Configuration example

[general]
udpbuffer         = 1048576  # size of the UDP buffer, default: 1048576
statsinterval     = 60       # interval in seconds to report stats, default: 60
flushinterval     = 5        # interval in seconds to flush data, default: 5
flushcount        = 2000     # number of records to flush, default: 2000
retries           = 5        # number of retries for failed operations, default: 5
retryinterval     = 5        # interval in seconds between retries, default: 5
dumprecords       = false    # output to stdout instead of netography, default: false
templatedb         = "/var/tmp/netoflow.db"  # path to the template database, default: "/var/tmp/netoflow.db"
#downsample        = 10      # downsample rate for data, default: 10
#apihost           = "https://api-ingest.netography.com"  # API host for data ingestion, default: "https://api-ingest.netography.com"
#devicecreationurl = "/api/v1/device"  # URL for device creation, default: "/api/v1/device"
#audithost         = "https://api.netography.com"  # host for audit logs, default: "https://api.netography.com"
#auditurl          = "/api/v1/audit"  # URL for audit logs, default: "/api/v1/audit"

[flow]
listen          = "0.0.0.0:2055"  # address and port to listen for flow data, default: "0.0.0.0:2055"
workers         = 4               # number of worker threads, default: 4
queue           = 100000          # size of the queue for incoming data, default: 100000

[syslog]
level           = "info"          # logging level, default: "info"
server          = "localhost:514" # syslog server address and port, default: "localhost:514"

[api]
enabled         = true  # enable or disable the API, default: true
port            = 8080  # port for the API, default: 8080

[sflow]
payload         = false  # include payload in sFlow data, default: false

#[tees]
#enabled         = false  # enable or disable tees, default: false
#updateinterval  = 300    # interval in seconds to update tees, default: 300
#targets         = [ "10.0.0.1:2055" ]  # list of target addresses for tees, default: [ "10.0.0.1:2055" ]

[endpoint]
netosecret       = ""  # API key encoded as netosecret

[proxy]
enabled         = false  # enable or disable proxy, default: false
url             = "http://10.0.0.1:3121"  # proxy URL

[filter]
default = "allow"  # default action for the filter, default: "allow"
logdeny = false    # log denied packets, default: false  

#[[ flowfilter ]]
#action = "deny"  # action for the flow filter, default: "deny"
#dstnets = ["10.0.0.0/24"]  # destination networks for the flow filter

#[[device]]
#flowsrcip      = "192.168.2.254"  # source IP address for flows
#downsample     = 100              # downsample rate for device data

Setting configuration in environment variables

Configuration can be set in the environment instead of the /etc/netoflow.conf file.

  • If you are using the container, ensure you are following the instructions in Install NetoFlow (container) to map the environment variables into the container correctly.

❗️

You can not set the teestargets, flowfilter, or device sections in environment variables today

These settings involve a list or array (ie multiple values), which is not parsed from environment variables today. If you are modifying these configuration settings, you must modify the netoflow.conf file.

Environment variable naming convention

Use this format for environment variables:
NETO__SECTION__KEY=value

  • NETO is the prefix for all configuration environment variables
  • Replace SECTION with the configuration section name. This is the value in square brackets [] in the conf file above the setting.
  • Replace KEY with the name of the configuration setting.

ℹ️

The separator used is two underscores __. If you use one underscore _ it won't work.

Environment variables mapping

The correct environment variable names to set for each configuration item in the netoflow.conf.examplefile is shown below.

NETO__GENERAL__UDPBUFFER=1048576  
NETO__GENERAL__STATSINTERVAL=60  
NETO__GENERAL__FLUSHINTERVAL=5  
NETO__GENERAL__FLUSHCOUNT=2000  
NETO__GENERAL__RETRIES=5  
NETO__GENERAL__RETRYINTERVAL=5  
NETO__GENERAL__DUMPRECORDS=false  
NETO__GENERAL__TEMPLATEDB="/var/tmp/netoflow.db"  
NETO__GENERAL__DOWNSAMPLE=10  
NETO__GENERAL__APIHOST="https://api-ingest.netography.com"  
NETO__GENERAL__DEVICECREATIONURL="/api/v1/device"  
NETO__GENERAL__AUDITHOST="https://api.netography.com"  
NETO__GENERAL__AUDITURL="/api/v1/audit"  
NETO__FLOW__LISTEN="0.0.0.0:2055"  
NETO__FLOW__WORKERS=4  
NETO__FLOW__QUEUE=100000  
NETO__SYSLOG__LEVEL="info"  
NETO__SYSLOG__SERVER="localhost:514"  
NETO__API__ENABLED=true  
NETO__API__PORT=8080  
NETO__SFLOW__PAYLOAD=false  
NETO__TEES__ENABLED=false  
NETO__TEES__UPDATEINTERVAL=300  
NETO__ENDPOINT__NETOSECRET=""  
NETO__PROXY__ENABLED=false  
NETO__PROXY__URL="http://10.0.0.1:3121"  
NETO__FILTER__DEFAULT="allow"  
NETO__FILTER__LOGDENY=false