Sampled NetFlow: Difference between revisions
(Created page with " == Introduction == Higher traffic volumes mean higer numbers of NetFlow records and increasing load on device's CPU. The increasing number of records may req...") |
No edit summary |
||
| Line 2: | Line 2: | ||
== Introduction == | == Introduction == | ||
Higher traffic volumes mean | Higher traffic volumes mean higher numbers of NetFlow records and increasing load on device's CPU. The increasing number of records may require too much bandwidth when send over the network to the collector. Therefore, it may become necessary to find a balance between reporting needs and device/network efficiency. | ||
Vendors may recommend or even force the use of Sampled NetFlow. | Vendors may recommend or even force the use of Sampled NetFlow. | ||
| Line 10: | Line 10: | ||
== Solution == | == Solution == | ||
=== Bit rate | === Bit rate calculation === | ||
Compared to SFlow, NetFlow had never been designed to accurately report metrics such as bit rate (bps) and packet rate (pps). A single NetFlow record contains duration of the flow and the number of bytes and packets transported during that time interval. The NetFlow reporter calculates bit rate and | Compared to SFlow, NetFlow had never been designed to accurately report metrics such as bit rate (bps) and packet rate (pps). A single NetFlow record contains duration of the flow and the number of bytes and packets transported during that time interval. The NetFlow reporter calculates bit rate and packet rate based on those records. NetFlow Sampling at a rate that is not adapted to the speed of the interface will jeopardize the exactitude of that calculation, even when all other precautionary measures such as "active time-out" are being used. | ||
| | ||
| Line 18: | Line 18: | ||
=== NetFlow Sampling === | === NetFlow Sampling === | ||
A sampling rate of 1 out of 100 may reduce the exports of NetFlow records by as much as 50%. You | A sampling rate of 1 out of 100 may reduce the exports of NetFlow records by as much as 50%. You must keep in mind that the collector will only receive a small percentage of the traffic and will use the sampling rate to "inflate" the metrics for bytes and packets received. | ||
| | ||
| Line 24: | Line 24: | ||
=== Impact === | === Impact === | ||
Tools for flow-based network analysis are used to monitor and baseline network behavior and increasingly by the security team to detect persistent threats. Sampling rates that are not adapted to the speed of the interface may lead to inaccurate reporting of | Tools for flow-based network analysis are used to monitor and baseline network behavior and increasingly by the security team to detect persistent threats. Sampling rates that are not adapted to the speed of the interface may lead to inaccurate reporting of quantitative metrics such as bit rate, packet rate and utilization. It may also lead to some flows no longer being reported at all. When security professionals need to go back in time, the flows they want to investigate may no longer be there. | ||
| | ||
=== | === Recommendations === | ||
#The use of sampling will always come at a cost. Use Sampled NetFlow only if you must. This could be the case when monitoring a high-traffic rate interface (n x Gbps) or when the vendor forces you to use a sampler. | #The use of sampling will always come at a cost. Use Sampled NetFlow only if you must. This could be the case when monitoring a high-traffic rate interface (n x Gbps) or when the vendor forces you to use a sampler. | ||
#Validate the reported data. Compare metrics such as bit rate and packet rate against the same metrics obtained by other means such as SNMP polling. You must find the balance between sampling rate and quality reporting. | #Validate the reported data. Compare metrics such as bit rate and packet rate against the same metrics obtained by other means such as SNMP polling. You must find the balance between sampling rate and quality reporting. | ||
#Please realize that the "raw" flow records database is probably incomplete. Your tool may no longer be | #Please realize that the "raw" flow records database is probably incomplete. Your tool may no longer be suitable for forensic flow analysis by the security team. | ||
| | ||
Revision as of 16:22, 24 November 2020
Introduction
Higher traffic volumes mean higher numbers of NetFlow records and increasing load on device's CPU. The increasing number of records may require too much bandwidth when send over the network to the collector. Therefore, it may become necessary to find a balance between reporting needs and device/network efficiency.
Vendors may recommend or even force the use of Sampled NetFlow.
Solution
Bit rate calculation
Compared to SFlow, NetFlow had never been designed to accurately report metrics such as bit rate (bps) and packet rate (pps). A single NetFlow record contains duration of the flow and the number of bytes and packets transported during that time interval. The NetFlow reporter calculates bit rate and packet rate based on those records. NetFlow Sampling at a rate that is not adapted to the speed of the interface will jeopardize the exactitude of that calculation, even when all other precautionary measures such as "active time-out" are being used.
NetFlow Sampling
A sampling rate of 1 out of 100 may reduce the exports of NetFlow records by as much as 50%. You must keep in mind that the collector will only receive a small percentage of the traffic and will use the sampling rate to "inflate" the metrics for bytes and packets received.
Impact
Tools for flow-based network analysis are used to monitor and baseline network behavior and increasingly by the security team to detect persistent threats. Sampling rates that are not adapted to the speed of the interface may lead to inaccurate reporting of quantitative metrics such as bit rate, packet rate and utilization. It may also lead to some flows no longer being reported at all. When security professionals need to go back in time, the flows they want to investigate may no longer be there.
Recommendations
- The use of sampling will always come at a cost. Use Sampled NetFlow only if you must. This could be the case when monitoring a high-traffic rate interface (n x Gbps) or when the vendor forces you to use a sampler.
- Validate the reported data. Compare metrics such as bit rate and packet rate against the same metrics obtained by other means such as SNMP polling. You must find the balance between sampling rate and quality reporting.
- Please realize that the "raw" flow records database is probably incomplete. Your tool may no longer be suitable for forensic flow analysis by the security team.