What is Network Metadata?

Research Article & Video

Why should your team be leveraging network metadata for more mature cybersecurity?

Many cybersecurity vendors use the term network metadata without clearly defining what it is. Generally speaking, metadata is descriptive information about the data, or “data about the data.” Network metadata carries individual traits pertaining to the structure of network protocols and packets. Specifically, it represents telemetry of network connections and the artifacts associated with these connections.

 

Oleg Sinitsin, CEO and Founder of Dynamite Analytics, discusses network metadata used for cyber analytics

Metadata acquisition technologies are generally categorized as Deep Packet Inspection (DPI) or network traffic inspection with many notable differences. Flow data (e.g. NetFlow, IPFIX, sFlow, etc.) is a commonly used set of the OSI Layer 2-4 telemetry, such as source, destination, protocol, bytes sent/received. Various flavors of flow data are produced by network infrastructure devices, and they offer a good start in understanding the basic trends of network traffic. Unfortunately, flow data is not always enough for advanced cyber threat detection within the application-layer context.

As opposed to flow data, network packet capture files (PCAPs) contain the full detail of network traffic. PCAPs are the most detailed historical record of what happened on the network, but they come with exponential storage requirements and inefficiency of data processing. The best of two worlds is now found in traffic inspection technologies that can be customized to extract meaningful Layer 3-7 metadata with an emphasis on Layer 7 application communications. One distinct and specific advantage of this approach is that this metadata can be used effectively for behavior cyber threat detection, while only taking 5% of the representative PCAP volume.

Dynamite has adopted Zeek, formerly Bro, as the best-in-class toolset for network traffic acquisition at high network speeds. Zeek metadata consists of 50+ logs with comprehensive connection records and application-layer transcripts. This information is naturally structured as time-series events corresponding to network conversations, such as: TCP/UDP/ICMP connections, HTTP requests and replies, DHCP leases, SNMP messages, SSH connections, and much more. The richness of Zeek metadata delivers unmatched capabilities for network detection and response, cyber threat hunting and forensic analysis.