Read time: 10 minutes

Installation Notes

This will guide you through how to setup a custom integration so that Elastic can parse Unifi Network Firewall logs.

This will comprise of 3 main steps:

Setup a custom integration.
Configure a pipeline which will parse events.
Configure Unifi for data ingestion.

This has been tested on a Unifi Dream Machine. Your mileage on other appliance types may vary.

After running this setup, traffic will be parsed into the following fields:

Syslog timestamp (log.syslog.timestamp)
Firewall rule description (rule.description)
Inbound interface (observer.ingress.interface.name)
Outbound interface (observer.egress.interface.name)
Source MAC address (source.mac)
Source IP address (source.ip)
Destination IP address (destination.ip)
Packet length / bytes (network.bytes)
Type of Service (TOS) (network.type)
Source port (source.port)
Destination port (destination.port)
Protocol / transport (network.transport)
GeoIP enrichment (source) (source.geo.*)
GeoIP enrichment (destination) (destination.geo.*)
Event category (event.category = network)
Event kind (event.kind = event)
Event type (conditional) (event.type = connection, only when both source.ip and destination.ip are present)

Requirements

Administrative access to Kibana.
Administrative access to Unifi Network.
At least one host, managed by fleet with an already assigned agent policy.
The managed agent must be able to accept logs on port 8514 from your Unifi appliance.

Setting up the custom integration

This guide assumes that you have already configured a fleet agent policy that is already managing hosts. The policy ID for this must be obtained first:

Navigate to Fleet > Agent Policies
Identify the policy which controls agents that will receive Unifi logs.
Select the policy and obtain its ID from the URL bar. For example, in this instance: app/fleet/policies/cd2cb24a-02ba-46da-ba38-c8011fa58ade, the policy is cd2cb24a-02ba-46da-ba38-c8011fa58ade.

Now navigate to the console under dev tools. Copy the below contents, modifying it with your specific fleet policy, and run this to create the integration. Dev tools should indicate a 200 - Success

This command will setup a custom integration that is configured to listen on port 8154.


POST kbn:/api/fleet/package_policies
{
  "policy_ids": [
    "<insert_fleet_policy_id_here>"
  ],
  "package": {
    "name": "udp",
    "version": "2.3.0"
  },
  "name": "UnifiNetworkFirewall",
  "description": "Unifi Network Firewall Logs",
  "namespace": "",
  "inputs": {
    "udp-udp": {
      "enabled": true,
      "streams": {
        "udp.udp": {
          "enabled": true,
          "vars": {
            "listen_address": "0.0.0.0",
            "listen_port": "8514",
            "use_logs_stream": false,
            "data_stream.dataset": "unifi_network",
            "max_message_size": "10KiB",
            "keep_null": false,
            "tags": [
              "UnifiNetwork"
            ],
            "syslog": true,
            "syslog_options": "field: message\n#format: auto\n#timezone: Local\n",
            "preserve_original_event": false,
            "custom": ""
          }
        }
      }
    }
  }
}

*You may also change the listen address from 0.0.0.0 to the specific local IP of the agent you intent to deploy this integration against`

Next, we will configure a custom pipeline on the newly created integration. The job of the pipeline is to extract key network information and map this to Elasticsearch fields.

Similar to before, paste the below in the development console to set this up

PUT _ingest/pipeline/logs-unifi_network@custom
{
  "description": "Parse and ECS map UniFi Network Firewall logs",
  "version": 1,
  "processors": [
    {
      "grok": {
        "field": "event.original",
        "patterns": [
          "%{SYSLOGTIMESTAMP:log.syslog.timestamp}"
        ],
        "ignore_failure": true,
        "description": "Grok Timestamp"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "DESCR=\"%{DATA:rule.description}\""
        ],
        "ignore_failure": true,
        "description": "Grok Firewall Rule Description"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "IN=%{WORD:observer.ingress.interface.name}"
        ],
        "ignore_failure": true,
        "description": "Grok Inbound Interface"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "OUT=%{WORD:observer.egress.interface.name}"
        ],
        "ignore_failure": true,
        "description": "Grok Outbound Interface"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "MAC=%{MAC:source.mac}"
        ],
        "ignore_failure": true,
        "description": "Grok MAC address"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "SRC=%{IP:source.ip}"
        ],
        "ignore_failure": true,
        "description": "Grok Source IP"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "DST=%{IP:destination.ip}"
        ],
        "ignore_failure": true,
        "description": "Grok Destination IP"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "LEN=%{NUMBER:network.bytes:long}"
        ],
        "ignore_failure": true,
        "description": "Grok Packet Length"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "TOS=%{BASE16NUM:network.type}"
        ],
        "ignore_failure": true,
        "description": "Grok Type of Service"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "SPT=%{NUMBER:source.port:long}"
        ],
        "ignore_failure": true,
        "description": "Grok Source Port"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "DPT=%{NUMBER:destination.port:long}"
        ],
        "ignore_failure": true,
        "description": "Grok Destination Port"
      }
    },
    {
      "grok": {
        "field": "message",
        "patterns": [
          "PROTO=%{WORD:network.transport}"
        ],
        "ignore_failure": true,
        "description": "Grok Protocol"
      }
    },
    {
      "geoip": {
        "field": "source.ip",
        "target_field": "source.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "destination.ip",
        "target_field": "destination.geo",
        "ignore_missing": true
      }
    },
    {
      "set": {
        "field": "event.category",
        "value": "network"
      }
    },
    {
      "set": {
        "field": "event.type",
        "value": "connection",
        "if": "ctx?.source?.ip != null"
      }
    },
    {
      "set": {
        "field": "event.kind",
        "value": "event"
      }
    }
  ]
}

The integration components have been configured, and have been assigned to a fleet policy in order for agents to receive ingest logs.

Configure Unifi Syslog

We will now configure syslog to send to a destination IP of an agent that is managed by Elastic and is running the newly deployed integration.

Login to Unifi Network
In the bottom left, select ‘Logs’
In the bottom left, select ‘Export to SIEM server’
Select the radial button ‘SIEM server’
For contents, ensure that ‘Firewall Default Policy’ is selected. This is the only type required for the integration. Other log types are not parsed.
Specify the server address and port based on the desired agent to ingest the logs.

Test Ingestion

Running an search within Discover will show new results. Ensure you are on the logs-* data view and run tags:"UnifiNetwork".

break