Azure Virtual network (VNet) Flow Log Setup
Netography Fusion ingests Virtual network (VNet) flow logs from Azure via an Azure Storage account. The steps to integrate with Azure are:
- Register Microsoft Insights provider (in each Azure subscription containing virtual networks you are onboarding).
- Create a storage account in Azure (for each region you are onboarding virtual networks).
- Create a flow log for the virtual network in Azure (for each virtual network you are onboarding).
- In Fusion, Add Azure VNet as a new flow source (for each virtual network you are onboarding).
In addition to ingesting VNet flow logs, you may want to enrich them with context from Azure resources by adding the Microsoft Azure context integration.
Using Terraform to automate onboarding
Access Netography's Terraform automation at our GitHub repo: https://github.com/netography/neto-onboarding. For access to the repo, email your GitHub ID to [email protected].
The instructions linked from this page are suitable for onboarding one or a small number of cloud accounts manually or using as a reference for building automation for larger scale deployment. In addition to these instructions, Netography provides a Terraform project,
neto-onboard,
that provides Netography Fusion Cloud Onboarding Automation for AWS Organizations, Azure Tenants, and GCP Organizations.Each cloud has 2 Terraform deployment options - full and simple.
The simple deployment deploys all the resources needed to integrate the cloud to Fusion and perform context enrichment in a deployment. You specify a target set of accounts/subscriptions/projects at deployment-time. You can redeploy the automation to change the scope of monitoring or when you need to onboard new accounts or networks to Fusion. This is suitable for a trial or if you have a relatively static cloud environment or one with a limited number of accounts and networks.
The full deployment provides the following:
- Enables and configure AWS VPC flow logs, Azure VNet flow logs, and GCP VPC flow logs based on a simple policy and tags that defines which VPC/VNet are in scope.
- Deploy all the infrastructure required to integrate to Fusion across multiple accounts (AWS), subscriptions (Azure), and projects (GCP) in a single deployment
- Adds VPCs/VNets configured for flow logging to Netography Fusion as traffic sources.
- Deploys a single AWS Lambda function, Azure Function, or Google Function that provides context enrichment across all the accounts/subscriptions/projects as an outbound push from your cloud to the Fusion API, eliminating the need to add context integrations from the Fusion portal, to grant Netography permissions to directly enumerate resource properties, or to add individual context integrations in Fusion for each cloud account.
- Monitor for VPC/VNet changes and trigger enabling and configuring flow logs, and onboarding to Fusion new VPCs/VNets that are in scope, and offboarding VPCs/VNets that are removed or no longer in scope.
Prerequisites
- Access to the Azure subscription(s) to onboard with an
Owner
orContributor
role, or a custom role with the specific permissions required for each step.- To register Microsoft Insights provider requires
/register/action
operation permissions for the Insights provider. The permission is included in theOwner
andContributor
roles. - To create flow logs for a virtual network requires
Microsoft.Network/networkWatchers/configureFlowLog/action
permission. The permission is included in theOwner
,Contributor
, andNetwork contributor
roles. - To create a storage account requires
Microsoft.Storage/storageAccounts/*
permission. The permission is included in theOwner
,Contributor
, andStorage account contributor
role.
- To register Microsoft Insights provider requires
- Azure Network Watcher must be enabled in the subscription and region for which the virtual network flow logs are enabled. This is enabled by default in Azure, but if you previously chose to opt out of Network Watcher automatic enablement, you must manuallly enable Network Watcher in each subscription and region containing virtual networks you are onboarding to Fusion. See Enable or Disable Azure Network Watcher.
- If Azure Policy is in use, you may be restricted from performing these steps, even if you have the
Azure Global Administrator
role. If this is the case, you will receive an AzureRequestDisallowedByPolicy
error. See Resolve errors for request disallowed by policy.
Microsoft Azure Instructions
1. Register Microsoft Insights Provider
You can skip this step if VNet flow logs are already enabled or if the Microsoft.Insights
provider is already registered in the Azure subscription.
Microsoft.Insights
provider must be registered in the virtual network's Azure subscription. You only need to perform this action once for each subscription containing virtual networks being monitored.
Follow these steps to register the Microsoft.Insights
provider: Microsoft Register Insights provider page.
Azure Console Steps
- Enter subscriptions in the search box at the top of Azure Console and select Subscriptions from the results.
- In the Subscriptions list, select the Azure subscription that you wish to enable the provider for.
- Under Settings, select Resource providers.
- Enter insight in the filter box.
- Confirm the status of the Microsoft.Insights provider displayed is Registered. If the status is NotRegistered, select the Microsoft.Insights provider then select Register.
2. Create a Storage Account for each region
If you are using the Azure Console to perform these steps, you can create a new storage account while creating the flow logs in the next step and skip this step.
Azure writes flow logs to an Azure storage account, and Fusion reads flow logs from the Azure storage account. Create a storage account for each region that contains virtual networks you are onboarding.
If you are onboarding multiple subscriptions in a single Azure tenant, you can have 1 set of storage accounts per region in a single centralized logging subscription and direct the flow logs from any subscription in the tenant to the corresponding storage account for that region.
Storage Account Configuration
Field | Value |
---|---|
Subscription | The same subscription as the virtual network, or a subscription in the same Azure tenant |
Resource Group | Any existing resource group, or create a new one (e.g. rg_neto_logging ) |
Storage Account Name | Any unique name (e.g. st_neto_vnetlogs_westus ) |
Region | The same region as the virtual network(s) |
Performance Tier | Standard |
Redundancy | Locally-redundant Storage (LRS) |
All other settings can use Azure's default configuration. The Advanced > Security > Enable storage account key access
setting must remain in its default setting,True
, for Azure Network Watcher to write flow logs to the storage account and Fusion to read flow logs from the storage account.
Restricting Azure Storage Account access to Netography's allowed IPs
The
Advanced > Networking > Network Access
setting for a storage account in Azure has a default value ofEnable public access from all networks
. This setting allows any IP to attempt to authenticate with an access key to the storage account. It does not allow unauthenticated access to the storage account.To further secure the storage account, restrict access to only the Netography Fusion Poller IPs required to read the flow logs. The IPs to allow are listed in the Netography Fusion Portal in Settings > Account Overview.> System Allow Lists > Pollers.
Create virtual network rules to restrict IP access to these IPs and grant access to the trusted Azure service
Microsoft.Network
to allow Azure Network Watcher to write the flow logs to the account. See: Configure Azure Storage firewalls and virtual networks > Grant access from a virtual network and Configure Azure Storage firewalls and virtual networks > Grant access to trusted Azure services.
3. Create a Flow Log for each Virtual network
You can skip this step if VNet flow logs are already enabled.
Follow these steps using the configuration settings below: Create a flow log section of the Manage VNET flow page.
Flow Log Configuration
Field | Value |
---|---|
Project details | |
Subscription | Select the Azure subscription of your virtual network that you want to log |
Flow Log Type | Select Virtual Network then select the virtual networks |
Flow Log Name | You can use the default name of{ResourceName}-{ResourceGroupName}-flowlog or enter your own |
Instance details | |
Subscription | Select the Azure subscription of the storage account to write flow logs to |
Storage Accounts | Select the storage account that you want to write flow logs to.. If you skipped step 2 above, select Create a new storage account. |
Retention (days) | 1 |
You can adjust the retention period to retain logs within the Azure storage account based on your organization's requirements.
Azure Console Steps
- In the search box at the top of the portal, enter network watcher. Select Network Watcher from the search results.
- Under Logs, select Flow logs.
- In Network Watcher | Flow logs, select + Create or Create flow log blue button.
- On the Basics tab of Create a flow log.
- Select Review + create.
- Review the settings, and then select Create.
For more information related to managing VNet Flow Logs in Azure, refer to Microsoft's Create, change, enable, disable, or delete virtual network flow logs using the Azure portal article.
Netography Fusion Instructions
4. Add a new Azure VNet flow source to Fusion
In the Fusion portal, click the gear icon to go to Settings, navigate to Traffic Sources, click Add Traffic Source, select Azure VNet, and fill out the form using the configuration below.
Azure VNet Flow Source Configuration
The following fields are specific to the Azure VNet configuration.
All of these field values can be located in the Azure Portal by going to Network Watcher, expanding the Logs section, selecting Flow Logs, and finding the row in the table for the flow log you are adding. The value to use is either directly visible in the table, or can be found by following the links noted in the table below.
Field | Description | Azure Network Watcher Flow Logs Table Field To Use |
---|---|---|
Region | Azure region the Vnet and storage account are located in (they are always the same) | Location |
Container Name | Storage account container name. Use the value insights-logs-flowlogflowevent for all standard configurations. | Storage account > Data Storage > Containers |
Subscription ID | Virtual network subscription ID | Subscription name > Overview |
Resource Group | Virtual network Resource Group name | Resource group |
Network Watcher | Network Watcher Name (e.g. NetworkWatcher_eastus/FlogLog_vnet2 ) | Name - The network watcher name is in parentheses |
Flow Log | Flow Log Name (eg FlowLog_vnet2 ) | Name - The flow log name is the value excluding what is in parentheses |
Account Name | Storage Account's Access Name | Storage account |
Account Key | Storage Account's Access Key to authenticate | Storage Account > Security + Networking > Access keys > Key |
Updated 3 months ago