Hardware Tuning and Troubleshooting

The underlying operating system beneath pfSense can be fine-tuned in many ways. A few of these tunables are available in pfSense under Advanced Options (See System Tunables Tab). Others are outlined in the FreeBSD main page tuning(7).

The default installation of pfSense software includes a well-rounded set of values tuned for good performance without being overly aggressive. There are cases where hardware or drivers necessitate changing values or a specific network workload requires changes to perform optimally. The hardware sold in the pfSense Store is tuned further since we have detailed knowledge of the hardware, removing the need to rely on more general assumptions.

Common changes along these lines for other hardware can be found in the documentation wiki page for Tuning and Troubleshooting Network Cards.

Note

Changes to /boot/loader.conf.local require a firewall reboot to take effect.

Mbuf Exhaustion

A common problem encountered by users of commodity hardware is mbuf exhaustion. For details on mbufs and monitoring mbuf usage, see Mbuf Clusters. If the firewall runs out of mbufs, it can lead to a kernel panic and reboot under certain network loads that exhaust all available network memory buffers. This is more common with NICs that use multiple queues or are otherwise optimized for performance over resource usage. Also, mbuf usage increases when the firewall is using certain features such as Limiters To increase the amount of mbufs available, add the following to /boot/loader.conf.local:

kern.ipc.nmbclusters="1000000"

Additionally, cards may need other similar values raised such as kern.ipc.nmbjumbop. In addition to the graphs mentioned above, check the output of the command netstat -m to verify if any areas are near exhaustion.

NIC Queue Count

For performance reasons some networks card types use multiple queues for processing packets. On multi-core systems, usually a driver will want to use one queue per CPU core. A few cases exist where this can lead to stability problems, which can be resolved by reducing the number of queues used by the NIC. To reduce the number of queues, specify the new value in /boot/loader.conf.local, such as:

hw.igb.num_queues=1

The name of the sysctl OID varies by network card, but it is usually located in the output of sysctl -a, under hw.<drivername>.

Disable MSIX

Another common issue is a NIC not properly supporting MSIX despite its claims. MSIX can be disabled by adding the following line to /boot/loader.conf.local:

hw.pci.enable_msix=0